IP addresses mismatch in `--noproxy` and related env variable (in 7.86.0)

[henning-schild]

I did this

cd /tmp
touch testfile
php -S 127.0.0.1:8000 &
# anything where there is no proxy will do
export http_proxy=http://127.0.0.5:3128
curl -vv -o /dev/null http://127.0.0.1:8000/testfile --noproxy "127.0.0.1"
curl -vv -o /dev/null http://127.0.0.1:8000/testfile --noproxy "127.0.0.1,localhost"

I expected the following

both curls downloading the file and not reaching out to the proxy, but if "127.0.0.1" is not the last part of the string ... it will not work

curl/libcurl version

[curl -V output]

[root@6a5006b5a040 ~]# curl -V
curl 7.86.0 (x86_64-pc-linux-gnu) libcurl/7.86.0 OpenSSL/1.1.1q zlib/1.2.13 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.3 libpsl/0.21.1 (+libidn2/2.3.0) libssh2/1.10.0 nghttp2/1.50.0
Release-Date: 2022-10-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

This came with 7.86.0 and showed up in a CI pipline testing multiple "modern distros". It came with "alpine:edge" and "archlinux:latest" at the same time. Both switched to that new curl yesterday.

operating system

docker containers of "alpine:edge" and "archlinux:latest" ... or Linux with the latest curl

[bagder]

Introduced in #9775 no doubt

[DhruvaSambrani]

Same issue, other applications using curl libs also affected. Is there a temporary workaround? Need to no-proxy on .ts.net, what should I be using instead?

[bagder]

Work-around: if you append /32 on the right side of the 127.0.0.1 in the noproxy string it should work. Like:

127.0.0.1/32,localhost

[DhruvaSambrani]

Work-around works. I assume this would need to be done on all ip addresses?

[bagder]

Yes, and you can now use proper CIDR notation so you can match a larger network in a single string, like 127.0.0.1/8.

[henning-schild]

I would say that is a regression which might even call for a new release rather soon. Nice to see there is a workaround. But if you ever had the honors to having to work with proxies and related env variables, you will know how horrible all of that is.

The variables are used by many tools and they all have their own implementation on dealing with them. While curl is an important player in that game it is not the only one. Just think multiple versions of curl all hoping to use the same NO_PROXY.

In my example i pointed it out via cmdline --noproxy but the env variables are the real deal, that is where users have to find values that work for all their tools and have the same effect in all of them. That is in fact impossible ... but any tool should try its best.

[bagder]

a regression which might even call for a new release rather soon

We will certainly take this into consideration.

[henning-schild]

Thanks for that quick fix but there is more to it. I tried to make that workaround get a whole complex pipeline green where proxies are needed for several things. The IP just being one of several essential parts of no_proxy.

curl 7.86.1-DEV (x86_64-pc-linux-gnu) libcurl/7.86.1-DEV OpenSSL/1.1.1n libidn2/2.3.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS HSTS HTTPS-proxy IDN IPv6 Largefile NTLM NTLM_WB SSL threadsafe TLS-SRP UnixSockets

commit d4fed2a13a81d23e73f1fb491c335a1b1d91e3fb

export https_proxy=http://127.0.0.5:3128
export http_proxy=http://127.0.0.5:3128
export no_proxy=localhost,.google.com,.google.de
curl -vv -o /dev/null http://www.google.com/testfile

And again it tries to go via that proxy where it should not.