Description
I did this
curl --verbose --create-dirs --head --dump-header ./test/headers.txt https://www.example.com/
Result:
Warning: Failed to open ./test/headers.txt
curl: (23) Failed writing received data to disk/application
I expected the following
Headers to be saved in the directory specified, even if it doesn't already exist, due to the --create-dirs option.
Obviously for a simple example like this, the directory can be manually created first. However, when used in a large config file with thousands of URLs, it is not possible to automatically save headers and responses into a custom directory per-URL (without first creating every single directory required, which defeats the value of --create-dirs).
E.g., in a config file such as this:
fail-early
next
url = "https://files.example.com/download?id=abc123"
fail
create-dirs
output-dir = "abc123"
remote-name
remote-header-name
dump-header = "abc123/headers.txt"
next
url = "https://files.example.com/download?id=def456"
fail
create-dirs
output-dir = "def456"
remote-name
remote-header-name
dump-header "def456/headers.txt"
(The use of --next is needed in order to reset --output-dir per URL. Different directories are needed per URL in this use case as different download IDs can have the same filename in the Content-Disposition header and both the download ID and filename need to be preserved in the resulting structure).
Without the dump-header lines, curl correctly uses the --create-dirs option and creates the directories when it comes to saving the actual response bodies. But because the headers come before the body, the directory does not exist at the time the headers are written, and curl errors out.
Without the dump-header lines, the rest of the config works really well for this use case. Thanks!
Suggested change:
In the dump-header code, if the provided value contains leading directories and they don't exist, before returning an error, first check if the --create-dirs option has been provided, and then first try to create the required directory tree (in the same way that --create-dirs works with --output). Only if that fails, then return an error, otherwise continue with the rest of the command.
curl/libcurl version
curl 8.10.0
operating system
Ubuntu 22.04