misleading/wrong error when using long output filename on Windows

[vszakats]

I did this

When using a long output filename on Windows or WINE, this happens:

> curl -q https://curl.se/ --output aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Warning: bad output glob
curl: (3) URL using bad/illegal format or missing URL

Even though the URL has no glob. It also only triggers with long
output filenames. It's a warning, but makes curl quit. To use the
long filename anyway and avoid the error, --globoff works as
a workaround.

Possible cause:

On Windows and MS-DOS, output filename sanitization is done
within glob_match_url(). When it fails, the caller assumes
a glob issue, shows the warning bad output glob and quits.

Maybe related: 4520534 (2016-02-05)

Ref: #20040 (comment)

I expected the following

• long filenames not to trigger a URL glob warning.
• not to show a glob warning for a URL without globs.
• say 'error' instead of 'warning' if the issue is fatal.
• allow using long filenames on Windows.
• apply the sanitizer also when using --globoff?

curl/libcurl version

master (and possibly earlier stable releases)

operating system

Windows

[jay]

The assumption is that the length of the path to be sanitized will not exceeed PATH_MAX-1 unless UNC:

curl/src/tool_doswin.c

Lines 117 to 130 in 400fffa

	if(flags & SANITIZE_ALLOW_PATH) {
	#ifndef MSDOS
	if(file_name[0] == '\\' && file_name[1] == '\\')
	/* UNC prefixed path \\ (eg \\?\C:\foo) */
	max_sanitized_len = 32767-1;
	else
	#endif
	max_sanitized_len = PATH_MAX-1;
	}
	else
	/* The maximum length of a filename. FILENAME_MAX is often the same as
	PATH_MAX, in other words it is 260 and does not discount the path
	information therefore we should not use it. */
	max_sanitized_len = (PATH_MAX-1 > 255) ? 255 : PATH_MAX-1;

How about this: #20046

[jay]

I think rename_if_reserved_dos will need changes as well.

edit: I will incorporate those changes into #20046

[vszakats]

@jay #20046 fixes the immediate issue, yes.

It also keeps existing (misleading) warning/error when the line exceeds the
new length limit. I wonder if it could be improved?

Makes me think why no such check happens when passing --globoff?
(and why it happens when there is an input URL with no glob, but globbing
enabled (default)).

I'd expect an 'error: too long filename` (and similar) if there is a filename
sanizitizing error, independ of globbing option.

On other platforms there is no similar check, with curl showing a warning
opening the output, and failing when trying to write to it:

Warning: Failed to open the file 
Warning: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Warning: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa: File name too long
curl: (23) client returned ERROR on write of 873 bytes

[bagder]

The reason the error message unexpectedly mentions globbing although there is no glob used in the URL is this:

• When globbing is in use, which it is by default, curl converts every given URL into a glob sequence. If no globbing sequences are used for a URL it stores the full URL as a single glob set. Globbing thus still considered enabled, just with the URL being a single glob.
• The function tool_operate/setup_outfile is used to determine the name of the file to use for the output, where to store the download. Since globbing is enabled, it calls glob_match_url to figure out if there is a #[num] construct to use. In this example case there isn't.
• The setup_outfile function calls sanitize_file_name (on windows only), which in the case of a too long name returns error, which makes setup_outfile return CURLE_URL_MALFORMAT which then causes the bad output glob error message...