libcurl can't download FTP website in 7.57 or later version

[wncboy]

Hi All,

[I did this]
I use CURLOPT_WILDCARDMATCH to download FTP website, but libcurl will report "Unable to parse FTP file list", and this issue always can be reproduced if the folder contains a large number of folders/files.

I found that this issue appears in 7.57 or later version, and it should be caused by the following commits:

=============================================
commit 1d72b5b
Author: Daniel Stenberg daniel@haxx.se
Date: Wed Oct 25 18:19:44 2017 +0200

ftplistparser: follow-up cleanup to remove PL_ERROR()

commit f786d1f
Author: Max Dymond cmeister2@gmail.com
Date: Wed Oct 25 15:52:43 2017 +0100

ftplistparser: free off temporary memory always

When using the FTP list parser, ensure that the memory that's
allocated is always freed.

Detected by OSS-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3682
Closes #2013

=============================================

BTW, I'm using curl's Debian package.

[I expected the following]
This issue can be fixed.

[curl/libcurl version]
libcurl3:amd64 7.57.0

[operating system]
Linux 4.4.98.x86_64.1

Thanks,

[bagder]

Thanks for your report. Please tell us (with more details) how to reproduce this problem as otherwise we can't really work on fixing it.

[wncboy]

Yes, we can use the following test program to reproduce this issue:
https://curl.haxx.se/libcurl/c/ftp-wildcard.html

Running ftp-wildcard in the console, and using the following command:
./ftp-wildcard ftp://ftp.scene.org/music/groups//*

curl_easy_perform function will return an error code, the code number is 87 (Unable to parse FTP file list)

Thanks,

[wncboy]

Do you have any updates?

Thanks,

[bagder]

If there were updates, they would be posted here.

[bagder]

Re-opened since the fix for this (#2508) was reverted in 10b195d due to memory leaks.

[bagder]

Had to revert again due to fuzzing problems with the landed code...