Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v8.8.0 - problem with --write-out: error 43 - A libcurl function was given a bad argument #13845

Closed
luckman212 opened this issue Jun 1, 2024 · 16 comments
Labels
appleOS specific to an Apple operating system build cmdline tool

Comments

@luckman212
Copy link

luckman212 commented Jun 1, 2024

related: Homebrew/homebrew-core#173294

I did this

I tried to use curl to make an HTTP request to GitHub and output the http response code:

$ /opt/homebrew/opt/curl/bin/curl -w '%{response_code}' https://github.com
curl: (43) A libcurl function was given a bad argument
000

I expected the following

Get http_response code 302 — same as builtin curl 8.6.0 at /usr/bin/curl:

$ /usr/bin/curl -w '%{response_code}' https://github.com
302
$ /usr/bin/curl -V
curl 8.6.0 (x86_64-apple-darwin23.0) libcurl/8.6.0 SecureTransport (LibreSSL/3.3.6) zlib/1.2.12 nghttp2/1.61.0
Release-Date: 2024-01-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

curl/libcurl version (Homebrew version)

$ /opt/homebrew/opt/curl/bin/curl -V
curl 8.8.0 (aarch64-apple-darwin23.4.0) libcurl/8.8.0 SecureTransport (OpenSSL/3.3.0) zlib/1.2.12 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libssh2/1.11.0 nghttp2/1.61.0 librtmp/2.3 OpenLDAP/2.6.8
Release-Date: 2024-05-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

operating system

macOS 14.5

@bagder
Copy link
Member

bagder commented Jun 1, 2024

Nobody has been able to reproduce. Looks like a broken build somehow.

@bagder bagder added build cmdline tool appleOS specific to an Apple operating system labels Jun 1, 2024
@luckman212
Copy link
Author

I also tried compiling it on my own system with

brew install --build-from-source curl

Same error

Anything else I can do to debug?

@luckman212
Copy link
Author

luckman212 commented Jun 1, 2024

I wrote a small test script to check response_code from some popular sites. Out of all the test sites, only the ones below generate the error. I guess there's a clue in here somewhere.

000  https://chatgpt.com
000  https://github.com
000  https://openai.com

Here's the test script in case anyone can check on their systems.

requires Bash 4.x or 5.x because it uses mapfile

#!/usr/bin/env bash

mapfile -t testurls <<EOF
amazon.com
chatgpt.com
cnn.com
example.com
facebook.com
github.com
google.com
instagram.com
microsoft.com
netflix.com
nytimes.com
office.com
openai.com
reddit.com
tiktok.com
whatsapp.com
wikipedia.org
x.com
youtube.com
EOF

case $1 in
	-h|--help|'') echo "${0##*/} [-ut] <url>"; exit;;
	-t|--test)
		for u in "${testurls[@]}"; do rc "$u"; done
		exit
		;;
	-u|--urls)
		echo "type or paste URLs below, press ⌃d when done"
		mapfile -t urls
		echo
		for u in "${urls[@]}"; do rc "$u"; done
		exit
		;;
esac

if [[ -n $1 ]] && [[ ! $1 =~ ^http ]] ; then
	c="https://$1"
	set -- "$c"
fi

/opt/homebrew/opt/curl/bin/curl \
--silent \
--output /dev/null \
--write-out "%{response_code}  $1\n" \
"$1"

@vszakats
Copy link
Member

vszakats commented Jun 1, 2024

Is there any app/machine/network-level firewall running, or is there any DNS filtering done?

edit: Also it may be worth trying with the --disable option to ignore .curlrc.

@bagder
Copy link
Member

bagder commented Jun 1, 2024

@vszakats I think the "A libcurl function was given a bad argument" thing is still the big sign that this is a build related issue.

@luckman212
Copy link
Author

@vszakats No DNS tampering or firewalls MITM'ing etc. I have also tested from several different networks/ISPs. I also get the same result when hitting the IP directly (not using DNS name at all)

@bagder I am working on trying to compile my own build from source to test, it isn't as straightforward as I thought on macOS...

@vszakats
Copy link
Member

vszakats commented Jun 1, 2024

...indeed, firewall/DNS only replicates the 000 output, not the (43) error.

It means I also couldn't replicate it with Homebrew on Intel + Monterey.

By default Homebrew installs pre-built binaries, meaning everyone should be using identical ones for the same CPU + macOS major release.

@vszakats
Copy link
Member

vszakats commented Jun 1, 2024

@luckman212 Here is our pre-built 8.8.0 binary if you want to try it:
https://github.com/curl/curl-for-win/actions/runs/9186135030 artifact 'curl-macos-universal-clang'

It's build differently than the Homebrew ones, but may be useful to see how it behaves in your env.

@luckman212
Copy link
Author

@vszakats Thank you. I grabbed that build, and not sure how to interpret these results... not sure what SSL library it's using, because I can't get it to connect without the insecure -k flag. But:

$ ./curl -s -o /dev/null -k -w '%{response_code}' https://github.com
200

Without -k I get

$ ./curl -s -o /dev/null -w '%{response_code}' https://github.com
000

@vszakats
Copy link
Member

vszakats commented Jun 2, 2024

It uses LibreSSL. To pick up the included CA bundle, you can use --cacert curl-ca-bundle.crt. Or to point it to to the Homebrew one: --cacert $(brew --prefix)/etc/ca-certificates/cert.pem.

@luckman212
Copy link
Author

luckman212 commented Jun 2, 2024

Ah, that works! (both variations actually). Not sure what went wrong with the Brew build but I wish they hadn't closed the issue so quickly. I'll see if they will re-open it based on this...

$ ./curl -s -o /dev/null -w '%{response_code}' --cacert ./curl-ca-bundle.crt https://github.com
200
$ ./curl -s -o /dev/null -w '%{response_code}' --cacert $(brew --prefix)/etc/ca-certificates/cert.pem https://github.com
200
$ /opt/homebrew/opt/curl/bin/curl -s -o /dev/null -w '%{response_code}' --cacert $(brew --prefix)/etc/ca-certificates/cert.pem https://github.com
000  <= ???

@luckman212
Copy link
Author

luckman212 commented Jun 2, 2024

After spending several hours compiling builds with various flags, I found that as long as I had --with-secure-transport enabled, it would fail. I was scratching my head until I finally found this old cobweb in my .bash_profile:

# Fix for `curl: (60) SSL certificate problem: certificate has expired`
export CURL_SSL_BACKEND='secure-transport'

Commenting that out seems to be a valid workaround for now. Very much appreciate the assistance.

@carlocab
Copy link

carlocab commented Jun 2, 2024

I can reproduce the problem after setting CURL_SSL_BACKEND. Based on my reading of https://curl.se/libcurl/c/libcurl-env.html, I don't see why that environment variable should break things.

Is this a bug in curl?

@Bo98
Copy link
Contributor

Bo98 commented Jun 2, 2024

Seems to come from 13ca438, which only a few SSL backends use.

I have a couple of PRs:

#13857 adds the missing OIDs which fixes the specific examples mentioned here but it doesn't really fix the problem in general as there's likely other missing OIDs.
#13858 is a more general fix that falls back to dotted representation if the OID mapping is missing.

@bagder bagder closed this as completed in 9aa1d41 Jun 2, 2024
dscho added a commit to dscho/MINGW-packages that referenced this issue Jun 6, 2024
The `git update-git-for-windows` command is a Unix Shell script that
relies heavily on `curl`; For example, it calls:

  curl --silent --show-error --output /tmp/gfw-httpget-ruMLqjGG.txt \
    --write-out '%{http_code}' \
    https://api.github.com/repos/git-for-windows/git/releases/latest

This worked well until cURL v8.8.0 was integrated. With the Secure
Channel backend, the invocation would fail with this error message:

  curl: (43) A libcurl function was given a bad argument

The same issue had been reported for a different SSL backend:
curl/curl#13845. Conveniently, all we have to
do is to Backport the fix from the PR that addressed that ticket to
address the bug at hand.

This fixes microsoft/git#655.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit to dscho/MINGW-packages that referenced this issue Jun 6, 2024
The `git update-git-for-windows` command is a Unix Shell script that
relies heavily on `curl`; For example, it calls:

  curl --silent --show-error --output /tmp/gfw-httpget-ruMLqjGG.txt \
    --write-out '%{http_code}' \
    https://api.github.com/repos/git-for-windows/git/releases/latest

This worked well until cURL v8.8.0 was integrated. With the Secure
Channel backend, the invocation would fail with this error message:

  curl: (43) A libcurl function was given a bad argument

The same issue had been reported for a different SSL backend:
curl/curl#13845. Conveniently, all we have to
do is to Backport the fix from the PR that addressed that ticket to
address the bug at hand.

This fixes microsoft/git#655.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@YOU54F
Copy link

YOU54F commented Jun 9, 2024

I've been seeing this issue as part of the latest round of windows image updates for GHA.

actions/runner-images#10004 (comment)

curl --version
curl 8.8.0 (x86_6[4](https://github.com/YOU54F/pact-js-cli/actions/runs/9433203515/job/25983873894#step:5:5)-w64-mingw32) libcurl/8.8.0 Schannel zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0
Release-Date: 2024-0[5](https://github.com/YOU54F/pact-js-cli/actions/runs/9433203515/job/25983873894#step:5:6)-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HSTS HTTPS-proxy IDN IPv[6](https://github.com/YOU54F/pact-js-cli/actions/runs/9433203515/job/25983873894#step:5:7) Kerberos Largefile libz NTLM PSL SPNEGO SSL SSPI threadsafe UnixSockets zstd

curl --output foo --write-out "%{http_code}" --location https://example.com/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  1256  100  1256    0     0  3[4](https://github.com/YOU54F/pact-js-cli/actions/runs/9433203515/job/25983873894#step:6:5)722      0 --:--:-- --:--:-- --:--:-- 36941
200

curl --output foo --write-out "%{http_code}" --location https://github.com/pact-foundation/pact-ruby-standalone/releases/download/v2.4.4/pact-2.4.4-windows-x86_64.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: libcurl function was given a bad argument
000

@ksidirop-laerdal
Copy link

ksidirop-laerdal commented Jul 8, 2024

I'm seeing this exact issue in github windows images (windows-2022) starting from early June 2024. Any workarounds for this one?

I've been seeing this issue as part of the latest round of windows image updates for GHA.

actions/runner-images#10004 (comment)

curl --version
curl 8.8.0 (x86_6[4](https://github.com/YOU54F/pact-js-cli/actions/runs/9433203515/job/25983873894#step:5:5)-w64-mingw32) libcurl/8.8.0 Schannel zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0
Release-Date: 2024-0[5](https://github.com/YOU54F/pact-js-cli/actions/runs/9433203515/job/25983873894#step:5:6)-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HSTS HTTPS-proxy IDN IPv[6](https://github.com/YOU54F/pact-js-cli/actions/runs/9433203515/job/25983873894#step:5:7) Kerberos Largefile libz NTLM PSL SPNEGO SSL SSPI threadsafe UnixSockets zstd

curl --output foo --write-out "%{http_code}" --location https://example.com/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  1256  100  1256    0     0  3[4](https://github.com/YOU54F/pact-js-cli/actions/runs/9433203515/job/25983873894#step:6:5)722      0 --:--:-- --:--:-- --:--:-- 36941
200

curl --output foo --write-out "%{http_code}" --location https://github.com/pact-foundation/pact-ruby-standalone/releases/download/v2.4.4/pact-2.4.4-windows-x86_64.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: libcurl function was given a bad argument
000

sergio-nsk pushed a commit to snxd/curl that referenced this issue Jul 9, 2024
dscho added a commit to git-for-windows/MINGW-packages that referenced this issue Jul 24, 2024
Obviously we need to drop the backport of the fix for
curl/curl#13845.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit to git-for-windows/MINGW-packages that referenced this issue Jul 24, 2024
Obviously we need to drop the backport of the fix for
curl/curl#13845.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
appleOS specific to an Apple operating system build cmdline tool
Development

Successfully merging a pull request may close this issue.

7 participants