git fetch fails .netrc parse with libcurl 8.11.0

[tranzystorekk]

I did this

With a standard .netrc entry like:

machine github.com
login <login>
password <github personal access token>

running git fetch on a github repository results in:

Fetching origin
fatal: unable to access <github repository url>: .netrc parser error

(issue not present on curl 8.10.1)

I expected the following

Git fetch should work correctly

curl/libcurl version

curl 8.11.0 (x86_64-unknown-linux-gnu) libcurl/8.11.0 OpenSSL/3.3.2 zlib/1.3.1 zstd/1.5.6 libidn2/2.3.4 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.64.0
Release-Date: 2024-11-06
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets zstd

operating system

Void Linux

[bagder]

Does the last line end with a newline or not?

[tranzystorekk]

Last line ends with a newline (double checked with a hex viewer)

[bagder]

I wrote up a test 478 in an attempt to reproduce, but it seems to just work...

478.zip

[akrherz]

FWIW, I am hitting this too. Some experimenting:

1. Empty ~/.netrc file, git works
2. No ~/.netrc file, git works
3. Using ~/.netrc from previous comment test, git fails

[yermulnik]

Hitting the same with curl: stable 8.11.0 (installed with Homebrew)
.netrc file contents ("blurred"):

machine tvbox
        login ...
        password ...
        macdef init
                set prompt "[%n@TVBox][%/]> "

[some more of "machine" defs]

default
        macdef init
                set prompt "[%n@%M][%/]> "

Under the hood I guess it's the same git fetch:

> brew update
==> Updating Homebrew...
fatal: unable to access 'https://filippo.io/age/': .netrc parser error
fatal: unable to access 'https://github.com/Homebrew/brew/': .netrc parser error
fatal: unable to access 'https://str4d.xyz/rage/': .netrc parser error
Error: Fetching /home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/filippo.io/homebrew-age failed!
Fetching /home/linuxbrew/.linuxbrew/Homebrew failed!
Fetching /home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/str4d.xyz/homebrew-rage failed!

[yermulnik]

Is there no option to disable use of .netrc with cURL Env var? 😢

[mmcc-arivo]

We're experiencing this too, but with GitLab. It seems specific to libcurl 8.11.0. It's running in a container built with the docker:24-cli image:

https://hub.docker.com/layers/library/docker/24-cli/images/sha256-dba081061b860672b2f50172ee0449b4d0a673586056ee8c12bf7747469e3cbe?context=explore

[BruceZhang1993]

Same issue here using curl/8.11.0 on Arch Linux, removing netrc file fix it.

[ZhongRuoyu]

Bisected to 3b43a05.

When running git clone https://github.com/curl/curl.git with debug build of 8.11.0 (Git version 2.47.0), curl aborts with git-remote-https: dynbuf.c:242: Curl_dyn_ptr: Assertion `s->init == 0xbee51da' failed.. This seems to be the case even without the .netrc file.

Below is a backtrace.

root@6ef6c94c19f2:~/curl# gdb /usr/local/bin/git
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/bin/git...done.
(gdb) set follow-fork-mode child
(gdb) r clone https://github.com/curl/curl.git /tmp/curl
Starting program: /usr/local/bin/git clone https://github.com/curl/curl.git /tmp/curl
warning: Error disabling address space randomization: Operation not permitted
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Cloning into '/tmp/curl'...
[New process 340009]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
process 340009 is executing new program: /usr/local/libexec/git-core/git
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New process 340010]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
process 340010 is executing new program: /usr/local/libexec/git-core/git-remote-https
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
git-remote-https: dynbuf.c:242: Curl_dyn_ptr: Assertion `s->init == 0xbee51da' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7cd4fbc43700 (LWP 340010)]
0x00007e4df6778125 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0  0x00007e4df6778125 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007e4df677b3a0 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007e4df6771311 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007e4df701d547 in Curl_dyn_ptr (s=0x11f5af0) at dynbuf.c:242
#4  0x00007e4df705fef9 in parsenetrc (store=0x11f5af0, host=0x11f7148 "github.com", loginp=0x11f6570, passwordp=0x11f6578,
    netrcfile=0x11f6a48 "/root/.netrc") at netrc.c:123
#5  0x00007e4df7060677 in Curl_parsenetrc (store=0x11f5af0, host=0x11f7148 "github.com", loginp=0x11f6570, passwordp=0x11f6578, netrcfile=0x0) at netrc.c:375
#6  0x00007e4df708d8f3 in override_login (data=0x11f4718, conn=0x11f6388) at url.c:2690
#7  0x00007e4df708f456 in create_conn (data=0x11f4718, in_connect=0x7ffdc89ac6c0, async=0x7ffdc89ac706) at url.c:3427
#8  0x00007e4df7090038 in Curl_connect (data=0x11f4718, asyncp=0x7ffdc89ac706, protocol_done=0x7ffdc89ac707) at url.c:3772
#9  0x00007e4df705b78e in state_connect (multi=0x11f1608, data=0x11f4718, nowp=0x7ffdc89ac840, resultp=0x7ffdc89ac748) at multi.c:2549
#10 0x00007e4df705bbf7 in multi_runsingle (multi=0x11f1608, nowp=0x7ffdc89ac840, data=0x11f4718) at multi.c:2673
#11 0x00007e4df705c69c in curl_multi_perform (m=0x11f1608, running_handles=0x7ffdc89ac8b8) at multi.c:3013
#12 0x000000000040eaf2 in start_active_slot (slot=0x11f46c0) at http.c:1596
#13 0x000000000040f38c in run_one_slot (slot=0x11f46c0, results=0x7ffdc89ac950) at http.c:1894
#14 0x000000000040fed4 in http_request (url=0x11f44a0 "https://github.com/curl/curl.git/info/refs?service=git-upload-pack", result=0x7ffdc89acb20, target=0,
    options=0x7ffdc89aca60) at http.c:2189
#15 0x0000000000410123 in http_request_reauth (url=0x11f44a0 "https://github.com/curl/curl.git/info/refs?service=git-upload-pack", result=0x7ffdc89acb20,
    target=0, options=0x7ffdc89aca60) at http.c:2271
#16 0x00000000004102e0 in http_get_strbuf (url=0x11f44a0 "https://github.com/curl/curl.git/info/refs?service=git-upload-pack", result=0x7ffdc89acb20,
    options=0x7ffdc89aca60) at http.c:2319
#17 0x0000000000407673 in discover_refs (service=0x11f4432 "git-upload-pack", for_push=0) at remote-curl.c:515
#18 0x000000000040a204 in stateless_connect (service_name=0x11f4432 "git-upload-pack") at remote-curl.c:1477
#19 0x000000000040a8a1 in cmd_main (argc=3, argv=0x7ffdc89ace18) at remote-curl.c:1643
#20 0x0000000000413411 in main (argc=3, argv=0x7ffdc89ace18) at common-main.c:64
(gdb)

[psbfahmi]

our workflow using .netrc to go get from private repo and experiencing this issue for the past few hours :(

[dacohen]

Continuing @ZhongRuoyu's work, 3b43a05 adds a call to Curl_netrc_init inside Curl_open.

However, from the above stack trace, we can see git is using curl_multi_perform, which doesn't use Curl_open. Consequently, Curl_netrc_init is never called, and the dynbuf is never initialized.

As a quick test of this, if I add a call to Curl_netrc_init in curl_multi_add_handle, everything works correctly.

[him3010]

our codebuild using .netrc to go get from private repo and experiencing this issue for the past few hours

[imajiddiffa]

for temporary solution, we use alpine3.19 instead of latest on our build image and it works for now.

[Vectorrent]

Same issue here. I just updated my system, and now I can't use any git repos, without first deleting my ~/.netrc file.

[bagder]

we can see git is using curl_multi_perform, which doesn't use Curl_open

That's not quite correct. Curl_open() is called by curl_easy_init() which is always used even with curl_multi_perform()...

[bagder]

Could it be because the involved handle has been duplicated?

[psbfahmi]

for temporary solution, we use alpine3.19 instead of latest on our build image and it works for now.

just want to confirm this. upgraded to golang:1.23.3-alpine3.19 and it works again!
thank you.

[bagder]

Tentative fix:

diff --git a/lib/easy.c b/lib/easy.c
index d16fa8c07..ac8fab342 100644
--- a/lib/easy.c
+++ b/lib/easy.c
@@ -938,10 +938,11 @@ CURL *curl_easy_duphandle(CURL *d)
   /* copy all userdefined values */
   if(dupset(outcurl, data))
     goto fail;
 
   Curl_dyn_init(&outcurl->state.headerb, CURL_MAX_HTTP_HEADER);
+  Curl_netrc_init(&outcurl->state.netrc);
 
   /* the connection pool is setup on demand */
   outcurl->state.lastconnect_id = -1;
   outcurl->state.recent_conn_id = -1;
   outcurl->id = -1;

[bagder]

Can anyone confirm that this fixes the problem seen with git?

[mnasruul-xxi]

Can anyone confirm that this fixes the problem seen with git?

wow, i tried to resolve it for a few hours today, but i couldn't. to resolve it i changed it to non alpine @bagder

[tranzystorekk]

Can anyone confirm that this fixes the problem seen with git?

Tested this patch on Void Linux, the issue got fixed :)

[ncopa]

Thank you all for solving this so quick!

[devTechi]

@at-wat thank you very much for your steps to reproduce the issue. I used your basis to test it further and sadly still have the problem:

# syntax = docker/dockerfile:1
FROM alpine:3.20

RUN apk add --no-cache git curl \
  && apk info libcurl

RUN --mount=type=secret,id=netrc,target=/root/.netrc \
  git clone https://github.com/[...]/private_repository.git

docker build --secret id=netrc,src=./.netrc --progress=plain --no-cache -t libcurl-test .
#0 building with "desktop-linux" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 267B done
#1 DONE 0.0s

#2 resolve image config for docker-image://docker.io/docker/dockerfile:1
#2 DONE 0.6s

#3 docker-image://docker.io/docker/dockerfile:1@sha256:865e5dd094beca432e8c0a1d5e1c465db5f998dca4e439981029b3b81fb39ed5
#3 CACHED

#4 [internal] load metadata for docker.io/library/alpine:3.20
#4 DONE 0.4s

#5 [internal] load .dockerignore
#5 transferring context: 2B done
#5 DONE 0.0s

#6 [stage-0 1/3] FROM docker.io/library/alpine:3.20@sha256:beefdbd8a1da6d2915566fde36db9db0b524eb737fc57cd1367effd16dc0d06d
#6 CACHED

#7 [stage-0 2/3] RUN apk add --no-cache git curl   && apk info libcurl
#7 0.091 fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/aarch64/APKINDEX.tar.gz
#7 0.581 fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/aarch64/APKINDEX.tar.gz
#7 1.398 (1/14) Installing ca-certificates (20240705-r0)
#7 1.465 (2/14) Installing brotli-libs (1.1.0-r2)
#7 1.703 (3/14) Installing c-ares (1.33.1-r0)
#7 1.751 (4/14) Installing libunistring (1.2-r0)
#7 1.957 (5/14) Installing libidn2 (2.3.7-r0)
#7 2.027 (6/14) Installing nghttp2-libs (1.62.1-r0)
#7 2.060 (7/14) Installing libpsl (0.21.5-r1)
#7 2.108 (8/14) Installing zstd-libs (1.5.6-r0)
#7 2.182 (9/14) Installing libcurl (8.11.0-r1)
#7 2.234 (10/14) Installing curl (8.11.0-r1)
#7 2.270 (11/14) Installing libexpat (2.6.3-r0)
#7 2.299 (12/14) Installing pcre2 (10.43-r0)
#7 2.352 (13/14) Installing git (2.45.2-r0)
#7 2.805 (14/14) Installing git-init-template (2.45.2-r0)
#7 2.832 Executing busybox-1.36.1-r29.trigger
#7 2.834 Executing ca-certificates-20240705-r0.trigger
#7 2.849 OK: 22 MiB in 28 packages
#7 2.885 WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.20/main: No such file or directory
#7 2.885 WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.20/community: No such file or directory
#7 2.885 libcurl-8.11.0-r1 description:
#7 2.886 The multiprotocol file transfer library
#7 2.886
#7 2.886 libcurl-8.11.0-r1 webpage:
#7 2.886 https://curl.se/
#7 2.886
#7 2.886 libcurl-8.11.0-r1 installed size:
#7 2.886 732 KiB
#7 2.886
#7 DONE 2.9s

#8 [stage-0 3/3] RUN --mount=type=secret,id=netrc,target=/root/.netrc   git clone https://github.com/[...]/private_repository.git
#8 0.103 Cloning into 'private_repository'...
#8 0.106 fatal: unable to access 'https://github.com/[...]/private_repository.git/': .netrc parser error
#8 ERROR: process "/bin/sh -c git clone https://github.com/[...]/private_repository.git" did not complete successfully: exit code: 128
------
 > [stage-0 3/3] RUN --mount=type=secret,id=netrc,target=/root/.netrc   git clone https://github.com/[...]/private_repository.git:
0.103 Cloning into 'private_repository'...
0.106 fatal: unable to access 'https://github.com/[...]/private_repository.git/': .netrc parser error
------
Dockerfile:7
--------------------
   6 |
   7 | >>> RUN --mount=type=secret,id=netrc,target=/root/.netrc \
   8 | >>>   git clone https://github.com/[...]/private_repository.git
   9 |
--------------------
ERROR: failed to solve: process "/bin/sh -c git clone https://github.com/[...]/private_repository.git" did not complete successfully: exit code: 128

I also made a cat of the mounted .netrc-file and it has the correct content.

Is it possible that your .netrc is not used, because you are cloning a public repository (totally unsure with this 😊 )?

Main difference I can see right now is the platform: x86_64 and for me aarch64

[at-wat]

@devTechi I tried also with my private repository URL and the actual netrc. It works as well both on my local x86_64 and EC2 aarch64 Linux.

[moha-gh]

@devTechi @at-wat : We are also still observing .netrc parse errors. Issue seems to persist if username and/or password are > 128 characters in length. I've filed #15513 for this.

[devTechi]

@moha-gh thank you. My .netrc also contains a big token as password (multiple lines). I will follow #15513

[alexanderjschmidt]

@at-wat whats the chance/process of getting #15513 added into alpine 3.20 as well?

[at-wat]

@alexanderjschmidt I already opened PRs for Alpine 3.20/edge and waiting for the maintainers response
https://gitlab.alpinelinux.org/alpine/aports/-/issues/16599
https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/74836
https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/74837

[BrianInglis]

Applied patches for 3 commits including test2309 and lib2309.c and Makefiles to Cygwin and problem now fixed but running test 2309 still fails?
Ran:
$ PATH="../../src/curl-8.11.0/tests:$PATH" runtests.pl -d -k -p -v 2309
Looks like HTTP server is returning or logger is writing responses with DOS EoLs and not surprisingly that does not match expected output:

test 2309...[HTTP with .netrc using duped easy handle]
./libtest/lib2309.exe http://github.com log/netrc2309 http://127.0.0.1:53387/ > log/stdout2309 2> log/stderr2309
CMD (0): ./libtest/lib2309.exe http://github.com log/netrc2309 http://127.0.0.1:53387/ > log/stdout2309 2> log/stderr2309

 2309: protocol FAILED:
--- log/check-expected	2024-12-05 16:17:37.255721400 -0700
+++ log/check-generated	2024-12-05 16:17:37.255721400 -0700
@@ -1,6 +1,6 @@
-GET http://github.com/ HTTP/1.1[LF]
-Host: github.com[LF]
-Authorization: Basic ZGFuaWVsOiR5JGo5VCRXVVZqaVZ2RGJSQVdhZkRMczZjYWIxJDAxTlgub2FaS2Y1bHc4TVIyTms5WWF4djRDcWJFMElhREYuR3BHeFB1bDE=[LF]
-Accept: */*[LF]
-Proxy-Connection: Keep-Alive[LF]
-[LF]
+GET http://github.com/ HTTP/1.1[CR][LF]
+Host: github.com[CR][LF]
+Authorization: Basic ZGFuaWVsOiR5JGo5VCRXVVZqaVZ2RGJSQVdhZkRMczZjYWIxJDAxTlgub2FaS2Y1bHc4TVIyTms5WWF4djRDcWJFMElhREYuR3BHeFB1bDE=[CR][LF]
+Accept: */*[CR][LF]
+Proxy-Connection: Keep-Alive[CR][LF]
+[CR][LF]
== Contents of files in the log/ dir after test 2309
=== Start of file check-expected
 GET http://github.com/ HTTP/1.1[LF]
 Host: github.com[LF]
 Authorization: Basic ZGFuaWVsOiR5JGo5VCRXVVZqaVZ2RGJSQVdhZkRMczZjYWIxJDAxTlgub2FaS2Y1bHc4TVIyTms5WWF4djRDcWJFMElhREYuR3BHeFB1bDE=[LF]
 Accept: */*[LF]
 Proxy-Connection: Keep-Alive[LF]
 [LF]
=== End of file check-expected
=== Start of file check-generated
 GET http://github.com/ HTTP/1.1[CR][LF]
 Host: github.com[CR][LF]
 Authorization: Basic ZGFuaWVsOiR5JGo5VCRXVVZqaVZ2RGJSQVdhZkRMczZjYWIxJDAxTlgub2FaS2Y1bHc4TVIyTms5WWF4djRDcWJFMElhREYuR3BHeFB1bDE=[CR][LF]
 Accept: */*[CR][LF]
 Proxy-Connection: Keep-Alive[CR][LF]
 [CR][LF]
=== End of file check-generated

Log files attached! ???
check-expected.log
check-generated.log