Our website would like to use cookies to store information on your computer. You may delete and block all cookies from this site, but parts of the site will not work as a result. Find out more about how we use cookies.

Login or Register

Powered by
Powered by Novacaster
 
FTP weirdness
by Bruce Ure at 16:45 05/06/06 (Forum::Technical Advice::General)
Anyone come across this before?
My dedicated server has started being fussy about FTP connections.

I can connect using active mode via my XP box, but if I use PASV mode it either (a) takes a while after LIST -aL for the listing to come up; or (b) it doesn't come up at all.

That's from work, or home, or round the corner.

If I use Interarchy on the Mac, or the 'Open file from server' feature of Text Wrangler, regardless of Active or PASV mode, it just hangs at LIST or LIST -a. Yet bizarrely, yesterday I am sure got a listing from Interarchy uing active mode (but not PASV).

That was before enabling the Apple firewall, which I have now disabled for FTP, but I don't think it's anything to do with that beause of the flakiness of it from the XP box as well, and the very fact I can't use PASV. And the fact that I can log in and it's only when LIST is used that it falls over.

I can connect to other FTP sites ok from XP and Mac, but not other virtual FTP hosts on the same machine.

Recently a certain Mr Gardner of this parish had a poke around my server and tightened some firewall stuff, but this was happening before then too, and anyway the FTP settings are definitely to Allow All from Anywhere.

The ftp server is proftpd:

======================
20 ProFTPD 1.2.10 Server (ProFTPD) [212.227.62.91]
USER ureorguk2
331 Password required for ureorguk2.
PASS *****
230 User ureorguk2 logged in.
PWD
257 "/" is current directory.
MACB E
500 MACB not understood
TYPE A
200 Type set to A
PORT 192,168,1,11,193,224
200 PORT command successful
LIST
======================

I realise there is a new version of proftpd out there but upgrading it just to see if it fixes it seems a bit of a shot in the dark because it's been working for bleedin' ages with the current version.

Have tried rebooting the machine(s, all of them).

Stumped!

--

<< Guy Kewney and his interview - >>
View Comments (Threaded Mode) Printer Version
FTP weirdness Bruce Ure - 16:45 05/06/06
Re: FTP weirdness Bruce Ure - 16:52 05/06/06
Oh FFS, it's the firewall.

So why is it that when the firewall is ON but FTP is specifically ENABLED, it won't work, but when the firewall is OFF it will work?

Curious.

And that only really fixes half of it, because why won't PASV work from the XP boxes... but that is much less of an issue.

--

Re: FTP weirdness Bruce Ure - 16:57 05/06/06
Definitely weird, this.

Jun 5 16:53:32 slippy ipfw: 12190 Deny TCP 212.227.62.91:20 192.168.1.11:49667 in via en0
Jun 5 16:53:35 slippy ipfw: 12190 Deny TCP 212.227.62.91:20 192.168.1.11:49667 in via en0
Jun 5 16:53:41 slippy ipfw: 12190 Deny TCP 212.227.62.91:20 192.168.1.11:49667 in via en0
Jun 5 16:53:53 slippy ipfw: 12190 Deny TCP 212.227.62.91:20 192.168.1.11:49667 in via en0
Jun 5 16:54:17 slippy ipfw: 12190 Deny TCP 212.227.62.91:20 192.168.1.11:49667 in via en0

That log is with the firewall on but FTP *allowed* in 'Services' and 'Firewall' (which IIUC is how it's telling me to enable an FTP-shaped hole in the firewall).

So why is it denying?

--

Re: FTP weirdness Bruce Ure - 17:16 05/06/06
If I enable PASV mode in the Proxies tab in Network preferences, everything dies again.

So if I want to use FTP to ure.org.uk from this machine I have to leave the firewall swicthed off.

Simon, do you have the firewall on your Mac(s) enabled, AAMOI? It's way less of a security issue than Windows I'm sure, but even so, it seems a bit scary to leave it off.

--

Re: FTP weirdness Bruce Ure - 17:23 05/06/06
Just for good measure here's a terminal session trying to do it manually:

=========================
slippy:~ bruce$ ftp ftp.ure.org.uk
Connected to ftp.ure.org.uk.
220 ProFTPD 1.2.10 Server (ProFTPD) [212.227.62.91]
Name (ftp.ure.org.uk:bruce): ureorguk2
331 Password required for ureorguk2.
Password:
230 User ureorguk2 logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls -al
229 Entering Extended Passive Mode (|||32794|)
[about a 2 minute gap here]
200 EPRT command successful
421 Service not available, remote server timed out. Connection closed
=========================

I'll shut up now but if anyone has any helpful brainwaves I'll buy them a stupidly large pint, or equivalent, next time we meet.

--

Re: FTP weirdness Simon - 18:05 05/06/06
The "FTP Access" entries in the Service and Firewall tabs in the OS X system preferences are for enabling inbound access to an FTP server running on that Mac.

Simply using PASV should be fine for retrieval from a remote FTP server.

--
simon

Re: FTP weirdness Simon - 18:09 05/06/06
Yes, I have my firewall on. I'm also behind two sets of NAT (one on the ADSL router, one on a Smoothwall) and I can still FTP anywhere just fine.

Gimme a username and password and I'll have a go from here if you like.
--
simon

Re: FTP weirdness Bruce Ure - 18:33 05/06/06
Cheers dude, yhm.

--

Re: FTP weirdness Gordon Hundley - 19:04 05/06/06
Not sure I follow all this. You say proftpd and firewall... what's the server OS, where's the firewall - on the same box, or between? It got confusing when you threw Macs and Windows servers into the equation. Do you have a plethora of servers behind a firewall that are not working? The you speak of Mac firewall settings, yet the default ftp server is tnftpd... did you put a proftpd on a Mac, if so which one?

At a guess... there's a proftpd somewhere and the Macs and Windows boxes are red herrings, and are just clients? The proftpd is probably set up not to use passive mode or is masquerading as something. The firewall looks like it doesn't have the ftp data port open which is required for active mode, hence why LIST will time out.

Tell me what the ftp server is, and I'll tell you how to enable PASV (though you may have to throw me a bone with some config lines - eg: masquerade). Tell me what the firewall is and I'll tell you how to put in rules that allow active and passive connections.
--
DrGoon

Re: FTP weirdness Bruce Ure - 19:33 05/06/06
Thanks Gordon.

Server is in Germany somewhere in a rack, running Plesk ("psa v7.5.4_build75060413.12 os_FedoraCore 2") on top of Fedora ("Linux 2.6.11.9-050512a").

There is a software firewall on the server, which is part of the Plesk system, which has:

FTP server -- Allow incoming from all

amongst a load of other stuff.

The firewall seems to be described as "the Plesk firewall" :-/

The FTP server is ProFTPD 1.2.10.

The clients, which are sitting here on my desk, NAT'ed behind my DSL router, and which regularly connect perfectly OK to myriad other ftp sites (well OK the mac is new so only a few times), behave thus:

XP
Active
- Connects immediately.
PASV
- Delay of between a few seconds and half a minute and then connects.
(Both via Windows personal firewall)

Mac
Active
- Connects immediately with Mac personal firewall off.
- Fails to connect with Mac personal firewall on.
PASV
- Won't connect at all, regardless of Mac personal firewall settings.

Ok, I give up, what is masquerade?

--

Re: FTP weirdness Gordon Hundley - 20:19 05/06/06
Okay... briefly - do you only have access to the server through the Plesk interface, or do you have root shell access? I'm pretty sure it's iptables with a GUI. With a bit of luck its even using the default Fedora location of /etc/sysconfig/iptables to store its changes and I won't have to go read up loads about Plesk. :)

ProFTPD can have it masquerade as anotehr host. So if the machine is really bdge.dodgyisp.com but you want it to look like ftp.bdge.com then you'd use masquerading. Unfortunately this knackers passive connections. If all you're doing is using ftp for admin, you probably don't need it.

--
DrGoon

Re: FTP weirdness Bruce Ure - 20:49 05/06/06
No I'm pretty certain there's no masquerading going on. I register the domains, point them at the server, tell the server to deal with them, and I can log in using FTP straight to the domain.

I have root shell access, via a specific middle host, so as to disallow SSH access from anywhere but that one host.

I can confirm that logging on to the ftp site(s) from this route also fails in the same way.

I have just checked and there is no iptables folder but there is a file in there called iptables-config and it looks like this:

(I don't think there's anything foolish to post here here but if you spot anythng perhaps you could let me know asap :-)


# Load additional iptables modules (nat helpers)
# Default: -none-
# Space separated list of nat helpers (e.g. 'ip_nat_ftp ip_nat_irc'), which
# are loaded after the firewall rules are applied. Options for the helpers are
# stored in /etc/modules.conf.
#IPTABLES_MODULES=""

# Unload modules on restart and stop
# Value: yes|no, default: yes
# This option has to be 'yes' to get to a sane state for a firewall
# restart or stop. Only set to 'no' if there are problems unloading netfilter
# modules.
#IPTABLES_MODULES_UNLOAD="yes"

# Save current firewall rules on stop.
# Value: yes|no, default: no
# Saves all firewall rules to /etc/sysconfig/iptables if firewall gets stopped
# (e.g. on system shutdown).
#IPTABLES_SAVE_ON_STOP="no"

# Save current firewall rules on restart.
# Value: yes|no, default: no
# Saves all firewall rules to /etc/sysconfig/iptables if firewall gets
# restarted.
#IPTABLES_SAVE_ON_RESTART="no"

# Save (and restore) rule and chain counter.
# Value: yes|no, default: no
# Save counters for rules and chains to /etc/sysconfig/iptables if
# 'service iptables save' is called or on stop or restart if SAVE_ON_STOP or
# SAVE_ON_RESTART is enabled.
#IPTABLES_SAVE_COUNTER="no"

# Numeric status output
# Value: yes|no, default: no
# Print IP addresses and port numbers in numeric format in the status output.
#IPTABLES_STATUS_NUMERIC="no"

ie. nothing, since it's all commented out.

Hang on, here's this:


[root@p15193706 /]# find -name iptables
./etc/rc.d/init.d/iptables
./lib/modules/2.6.9-1.6_FC2/build/include/config/ip/nf/iptables
./lib/modules/2.6.9-1.6_FC2/build/include/config/ip6/nf/iptables
./lib/modules/2.6.9-1.6_FC2smp/build/include/config/ip/nf/iptables
./lib/modules/2.6.9-1.6_FC2smp/build/include/config/ip6/nf/iptables
./lib/iptables
./sbin/iptables

and this:


[root@p15193706 /]# more /etc/rc.d/init.d/iptables
#!/bin/sh
#
# iptables Start iptables firewall
#
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
#
# config: /etc/sysconfig/iptables
# config: /etc/sysconfig/iptables-config

# Source function library.
. /etc/init.d/functions

IPTABLES=iptables
IPTABLES_DATA=/etc/sysconfig/$IPTABLES
IPTABLES_CONFIG=/etc/sysconfig/${IPTABLES}-config
IPV=${IPTABLES%tables} # ip for ipv4 | ip6 for ipv6
PROC_IPTABLES_NAMES=/proc/net/${IPV}_tables_names
VAR_SUBSYS_IPTABLES=/var/lock/subsys/$IPTABLES

if [ ! -x /sbin/$IPTABLES ]; then
echo -n $"/sbin/$IPTABLES does not exist."; warning; echo
exit 0
fi

if lsmod 2>/dev/null | grep -q ipchains ; then
echo -n $"ipchains and $IPTABLES can not be used together."; warning; echo
exit 0
fi

# Old or new modutils
/sbin/modprobe --version 2>&1 | grep -q module-init-tools \
&& NEW_MODUTILS=1 \
|| NEW_MODUTILS=0

# Default firewall configuration:
IPTABLES_MODULES=""
IPTABLES_MODULES_UNLOAD="yes"
IPTABLES_SAVE_ON_STOP="no"
IPTABLES_SAVE_ON_RESTART="no"
IPTABLES_SAVE_COUNTER="no"
IPTABLES_STATUS_NUMERIC="no"

# Load firewall configuration.
[ -f "$IPTABLES_CONFIG" ] && . "$IPTABLES_CONFIG"

rmmod_r() {
# Unload module with all referring modules.
# At first all referring modules will be unloaded, then the module itself.
local mod=$1
local ret=0
local ref=

# Get referring modules.
# New modutils have another output format.
[ $NEW_MODUTILS = 1 ] \
&& ref=`lsmod | awk "/^${mod}/ { print \\\$4; }" | tr ',' ' '` \
|| ref=`lsmod | grep ^${mod} | cut -d "[" -s -f 2 | cut -d "]" -s -f 1`

# recursive call for all referring modules
for i in $ref; do
rmmod_r $i
let ret+=$?;
done

# Unload module.
# The extra test is for 2.6: The module might have autocleaned,
# after all referring modules are unloaded.
if grep -q "^${mod}" /proc/modules ; then
modprobe -r $mod > /dev/null 2>&1
let ret+=$?;
fi

return $ret
}

flush_n_delete() {
# Flush firewall rules and delete chains.
[ -e "$PROC_IPTABLES_NAMES" ] || return 1

# Check if firewall is configured (has tables)
tables=`cat $PROC_IPTABLES_NAMES 2>/dev/null`
[ -z "$tables" ] && return 1

echo -n $"Flushing firewall rules: "
ret=0
# For all tables
for i in $tables; do
# Flush firewall rules.
$IPTABLES -t $i -F;
let ret+=$?;

# Delete firewall chains.
$IPTABLES -t $i -X;
let ret+=$?;

# Set counter to zero.
$IPTABLES -t $i -Z;
let ret+=$?;
done

[ $ret -eq 0 ] && success || failure
echo
return $ret
}

set_policy() {
# Set policy for configured tables.
policy=$1

# Check if iptable module is loaded
[ ! -e "$PROC_IPTABLES_NAMES" ] && return 1

# Check if firewall is configured (has tables)
tables=`cat $PROC_IPTABLES_NAMES 2>/dev/null`
[ -z "$tables" ] && return 1

echo -n $"Setting chains to policy $policy: "
ret=0
for i in $tables; do
echo -n "$i "
case "$i" in
filter)
$IPTABLES -t filter -P INPUT $policy \
&& $IPTABLES -t filter -P OUTPUT $policy \
&& $IPTABLES -t filter -P FORWARD $policy \
|| let ret+=1
;;
nat)
$IPTABLES -t nat -P PREROUTING $policy \
&& $IPTABLES -t nat -P POSTROUTING $policy \
&& $IPTABLES -t nat -P OUTPUT $policy \
|| let ret+=1
;;
mangle)
$IPTABLES -t mangle -P PREROUTING $policy \
&& $IPTABLES -t mangle -P POSTROUTING $policy \
&& $IPTABLES -t mangle -P INPUT $policy \
&& $IPTABLES -t mangle -P OUTPUT $policy \
&& $IPTABLES -t mangle -P FORWARD $policy \
|| let ret+=1
;;
*)
let ret+=1
;;
esac
done

[ $ret -eq 0 ] && success || failure
echo
return $ret
}

start() {
# Do not start if there is no config file.
[ -f "$IPTABLES_DATA" ] || return 1

echo -n $"Applying $IPTABLES firewall rules: "

OPT=
[ "x$IPTABLES_SAVE_COUNTER" = "xyes" ] && OPT="-c"

$IPTABLES-restore $OPT $IPTABLES_DATA
if [ $? -eq 0 ]; then
success; echo
else
failure; echo; return 1
fi

# Load additional modules (helpers)
if [ -n "$IPTABLES_MODULES" ]; then
echo -n $"Loading additional $IPTABLES modules: "
ret=0
for mod in $IPTABLES_MODULES; do
echo -n "$mod "
modprobe $mod > /dev/null 2>&1
let ret+=$?;
done
[ $ret -eq 0 ] && success || failure
echo
fi

touch $VAR_SUBSYS_IPTABLES
return $ret
}

stop() {
# Do not stop if iptables module is not loaded.
[ -e "$PROC_IPTABLES_NAMES" ] || return 1

flush_n_delete
set_policy ACCEPT

if [ "x$IPTABLES_MODULES_UNLOAD" = "xyes" ]; then
echo -n $"Unloading $IPTABLES modules: "
ret=0
rmmod_r ${IPV}_tables
let ret+=$?;
rmmod_r ${IPV}_conntrack
let ret+=$?;
[ $ret -eq 0 ] && success || failure
echo
fi

rm -f $VAR_SUBSYS_IPTABLES
return $ret
}

save() {
# Check if iptable module is loaded
[ ! -e "$PROC_IPTABLES_NAMES" ] && return 1

# Check if firewall is configured (has tables)
tables=`cat $PROC_IPTABLES_NAMES 2>/dev/null`
[ -z "$tables" ] && return 1

echo -n $"Saving firewall rules to $IPTABLES_DATA: "

OPT=
[ "x$IPTABLES_SAVE_COUNTER" = "xyes" ] && OPT="-c"

ret=0
TMP_FILE=`/bin/mktemp -q /tmp/$IPTABLES.XXXXXX` \
&& chmod 600 "$TMP_FILE" \
&& $IPTABLES-save $OPT > $TMP_FILE 2>/dev/null \
&& size=`stat -c '%s' $TMP_FILE` && [ $size -gt 0 ] \
|| ret=1
if [ $ret -eq 0 ]; then
if [ -e $IPTABLES_DATA ]; then
cp -f $IPTABLES_DATA $IPTABLES_DATA.save \
&& chmod 600 $IPTABLES_DATA.save \
|| ret=1
fi
if [ $ret -eq 0 ]; then
cp -f $TMP_FILE $IPTABLES_DATA \
&& chmod 600 $IPTABLES_DATA \
|| ret=1
fi
fi
[ $ret -eq 0 ] && success || failure
echo
rm -f $TMP_FILE
return $ret
}

status() {
# Do not print status if lockfile is missing and iptables modules are not
# loaded.
# Check if iptable module is loaded
if [ ! -f "$VAR_SUBSYS_IPTABLES" ]; then
echo $"Firewall is stopped."
return 1
fi

# Check if firewall is configured (has tables)
if [ ! -e "$PROC_IPTABLES_NAMES" ]; then
echo $"Firewall is not configured. "
return 1
fi
tables=`cat $PROC_IPTABLES_NAMES 2>/dev/null`
if [ -z "$tables" ]; then
echo $"Firewall is not configured. "
return 1
fi

NUM=
[ "x$IPTABLES_STATUS_NUMERIC" = "xyes" ] && NUM="-n"

for table in $tables; do
echo $"Table: $table"
$IPTABLES -t $table --list $NUM && echo
done

return 0
}

restart() {
[ "x$IPTABLES_SAVE_ON_RESTART" = "xyes" ] && save
stop
start
}

case "$1" in
start)
stop
start
RETVAL=$?
;;
stop)
[ "x$IPTABLES_SAVE_ON_STOP" = "xyes" ] && save
stop
RETVAL=$?
;;
restart)
restart
RETVAL=$?
;;
condrestart)
[ -e "$VAR_SUBSYS_IPTABLES" ] && restart
;;
status)
status
RETVAL=$?
;;
panic)
flush_n_delete
set_policy DROP
RETVAL=$?
;;
save)
save
RETVAL=$?
;;
*)
echo $"Usage: $0 {start|stop|restart|condrestart|status|panic|save}"
exit 1
;;
esac

exit $RETVAL
[root@p15193706 iptables]#

And I can see immediately from that what's wrong: there's a problem.

--

Re: FTP weirdness Bruce Ure - 20:51 05/06/06
Would it help if I just sent you the login details? :)

--

Re: FTP weirdness Gordon Hundley - 21:48 05/06/06
Sure, you have my email, right?
--
DrGoon (atmacdotcom)
Re: FTP weirdness Bruce Ure - 08:33 06/06/06
Wahey! Huge thanks to Gordon for working on this and fixing it.

Turns out to have been that the ip_conntrack_ftp (ftp connection tracking) module was not loaded, so now it is.

It's a mystery to me why it used to work but then stopped. And also how the hell they (Plesk) can distribute their management software so fundamentally broken. I suspect they don't, and that somewhere along the line someone (that'd be me or Paul then) has inadvertently broken it, but lawd knows how.

Simon, thanks to you as well for your help.

--

Re: FTP weirdness Simon - 10:14 06/06/06
No prob.

--
simon