New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libcurl can't download FTP website in 7.57 or later version #2445

Closed
wncboy opened this Issue Apr 2, 2018 · 6 comments

Comments

Projects
None yet
2 participants
@wncboy

wncboy commented Apr 2, 2018

Hi All,

[I did this]
I use CURLOPT_WILDCARDMATCH to download FTP website, but libcurl will report "Unable to parse FTP file list", and this issue always can be reproduced if the folder contains a large number of folders/files.

I found that this issue appears in 7.57 or later version, and it should be caused by the following commits:

=============================================
commit 1d72b5b
Author: Daniel Stenberg daniel@haxx.se
Date: Wed Oct 25 18:19:44 2017 +0200

ftplistparser: follow-up cleanup to remove PL_ERROR()

commit f786d1f
Author: Max Dymond cmeister2@gmail.com
Date: Wed Oct 25 15:52:43 2017 +0100

ftplistparser: free off temporary memory always

When using the FTP list parser, ensure that the memory that's
allocated is always freed.

Detected by OSS-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3682
Closes #2013

=============================================

BTW, I'm using curl's Debian package.

[I expected the following]
This issue can be fixed.

[curl/libcurl version]
libcurl3:amd64 7.57.0

[operating system]
Linux 4.4.98.x86_64.1

Thanks,

@wncboy wncboy changed the title from Curl can't download FTP website in 7.57 or later version to lib curl can't download FTP website in 7.57 or later version Apr 2, 2018

@wncboy wncboy changed the title from lib curl can't download FTP website in 7.57 or later version to libcurl can't download FTP website in 7.57 or later version Apr 2, 2018

@bagder bagder added the FTP label Apr 2, 2018

@bagder

This comment has been minimized.

Member

bagder commented Apr 2, 2018

Thanks for your report. Please tell us (with more details) how to reproduce this problem as otherwise we can't really work on fixing it.

@wncboy

This comment has been minimized.

wncboy commented Apr 2, 2018

Yes, we can use the following test program to reproduce this issue:
https://curl.haxx.se/libcurl/c/ftp-wildcard.html

Running ftp-wildcard in the console, and using the following command:
./ftp-wildcard ftp://ftp.scene.org/music/groups//*

curl_easy_perform function will return an error code, the code number is 87 (Unable to parse FTP file list)

Thanks,

@wncboy

This comment has been minimized.

wncboy commented Apr 9, 2018

Do you have any updates?

Thanks,

@bagder

This comment has been minimized.

Member

bagder commented Apr 9, 2018

If there were updates, they would be posted here.

bagder added a commit that referenced this issue Apr 19, 2018

ftplistparser: keep state between invokes
Regression from f786d1f

Reported-by: wncboy on github
Fixes #2445

@bagder bagder closed this in 8fb78f9 Apr 19, 2018

@bagder

This comment has been minimized.

Member

bagder commented Apr 23, 2018

Re-opened since the fix for this (#2508) was reverted in 10b195d due to memory leaks.

@bagder bagder reopened this Apr 23, 2018

bagder added a commit that referenced this issue Apr 23, 2018

ftplistparser: keep state between invokes
Regression from f786d1f

Reported-by: wncboy on github
Fixes #2445
Closes #2508

@bagder bagder closed this in abbc845 Apr 23, 2018

@bagder

This comment has been minimized.

Member

bagder commented Apr 24, 2018

Had to revert again due to fuzzing problems with the landed code...

@bagder bagder reopened this Apr 24, 2018

bagder added a commit that referenced this issue Apr 24, 2018

ftplistparser: keep state between invokes
Fixes FTP wildcard parsing when doing over a number of read buffers.

Regression from f786d1f

Reported-by: wncboy on github
Fixes #2445
Closes #2519

@bagder bagder closed this in e6c2236 Apr 24, 2018

@lock lock bot locked as resolved and limited conversation to collaborators Jul 23, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.