Support NLST command wildcard

[giampaolo]

From stephane...@gmail.com on April 03, 2009 11:19:36

- What steps will reproduce the problem?
1. Connect to a FTP server backed by pyftpdlib with ncftp
2. From the client, try to 'get *.txt' or another wildcard
3. pyftpdlib replies:
127.0.0.1:49098 <== NLST *.txt
[ftp]@127.0.0.1:49098 FAIL NLST "/*.txt". No such file or directory.


- 

What is the expected output?  
What do you see instead?
The RFC 1123 talks about the possibility for the server to implement
wildcards on NLST ( http://www.ietf.org/rfc/rfc1123.txt , page 31)

- What version of pyftpdlib are you using? On what operating system? Which 
Python version? pyftpdlib 0.5.1

Original issue: http://code.google.com/p/pyftpdlib/issues/detail?id=106

[giampaolo]

From billiej...@gmail.com on April 03, 2009 09:05:40

The glob/wildcard functionality was available in pyftpdlib for the STAT command until
I decided to remove it in r358 because it was too complex to maintain.

More than one path traversal vulnerability has been discovered during the 
development
of pyftpdlib (see for example issue 9 and issue 55 ). 
Properly implementing wildcard patterns (not only for NLST but also for LIST and
STAT) making sure that the user does not escape from its home directory is not easy,
so I just decided to give up.

RFC-1123 was suggesting to implement wildcards for NLST because at that time (year
1989) NLST was the less painful choice to do such a thing since the LIST command
output was not standardized, but it wasn't much better than LIST either since NLST
does not distinguish between files directories.
You just receive something like this:

foo.txt\r\n
bar.txt\r\n
apple.txt\r\n

...but you don't know whether those are files, directories, symlinks or 
anything else.

Luckily for us RFC-3659 (year 2007) defined the new MLSD command (which is supported
by pyftpdlib).
Every modern client willing to do anything like the "get *.txt" functionality offered
by ncftp should use MLSD.
LIST and especially NLST commands are obsolete and shouldn't be used anymore.

It's useful to support clients like ncftp, but considering the risk of incurring
other problems and potential security vulnerabilities I don't think it's worth adding
extra functionality to support older, obsolete clients.

Labels: -Type-Defect -OpSys-All Type-Enhancement Version-0.5.1 Usability Compatibility RFC

[giampaolo]

From billiej...@gmail.com on April 12, 2009 07:22:59

Status: Rejected

[giampaolo]

From d...@devicenull.org on July 15, 2011 11:10:40

I've just discovered that the FTP client *built-in* to windows uses this.  If 
you use the command-prompt one (Start->Run 'ftp') and try to use the mget 
command, it passes whatever you input to the FTP server with the NLST command.  
For example:

client log:
ftp> mget *.log
No such file or directory.
200 Type set to: ASCII.
Cannot find list of remote files.

server log:
x.x.x.x:10144 <== NLST *.log
x.x.x.x:10144 ==> 550 No such file or directory.

You might want to reconsider adding this, because I'm testing with the FTP 
client from Windows 7.  I really wouldn't consider this FTP client to be obsolete.

[giampaolo]

From d...@devicenull.org on July 15, 2011 11:22:26

I've also confirmed this is present in the latest version of Net::FTP perl 
module.  ("Latest" is really a joke here, it hasn't been updated since 2007).

I'm going to have to reconsider what I just said though, I'd rather have not 
have to worry about security issues caused by this, rather then have support 
for this.  I'll continue to update this bug if I find any more clients that do 
this, but that's just for reference.