While Importing, received Error: "Fatal: the remote end hung up unexpectedly"

[kabua]

Select Topic Area

Question / Bug?

Body

While trying to manually import a repository, we got this internal error

error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly
Everything up-to-date

How do we fix this issue? Or can we, as it appears to be an internal bug.

Background

We are in the process of moving from BitBucket to GitHub. Therefore, we have created a GitHub organization (the free version, until we can resolve all of our migration issues).

We have been able to use the import tool to migrate 18 out of 19 repositories. However, for some unknown reason, we can't import the last one.

While searching the web for a solution, we came across this Import of Git repo into GitHub failing discussion. That led us to this GitHub documentation, Importing an external Git repository using the command line.

Following the documentation, we did the following (private information replaced with <company> and <repo> placeholders):

1. Opened a Windows 11 CMD
2. In a test folder, we ran: git clone --bare https://bitbucket.org/<company>/<repo>.git
3. cd <repo>.git
4. git push --mirror https://github.com/<company>/<repo>.git

Results

error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly
Everything up-to-date

More searching led to Github - unexpected disconnect while reading sideband packet. From this we applied

set GIT_TRACE_PACKET=1
set GIT_TRACE=1
set GIT_CURL_VERBOSE=1
git config --global core.compression 0

Still failed.

Full trace output

D:\Sw\<repo>.git>git push --mirror https://github.com/<Company>/<repo>.git
13:54:22.711252 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
13:54:22.717757 git.c:463               trace: built-in: git push --mirror https://github.com/<Company>/<repo>.git
13:54:22.717757 run-command.c:659       trace: run_command: git remote-https https://github.com/<Company>/<repo>.git https://github.com/<Company>/<repo>.git
13:54:22.781760 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:54:22.782761 git.c:749               trace: exec: git-remote-https https://github.com/<Company>/<repo>.git https://github.com/<Company>/<repo>.git
13:54:22.782761 run-command.c:659       trace: run_command: git-remote-https https://github.com/<Company>/<repo>.git https://github.com/<Company>/<repo>.git
13:54:22.839018 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:54:22.864532 http.c:870              == Info: Couldn't find host github.com in the .netrc file; using defaults
13:54:22.885545 http.c:870              == Info:   Trying 140.82.112.3:443...
13:54:22.929772 http.c:870              == Info: Connected to github.com (140.82.112.3) port 443
13:54:22.930772 http.c:870              == Info: ALPN: curl offers h2,http/1.1
13:54:22.930772 http.c:870              == Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
13:54:23.108157 http.c:870              == Info:  CAfile: C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt
13:54:23.108157 http.c:870              == Info:  CApath: none
13:54:23.108157 http.c:870              == Info: TLSv1.3 (IN), TLS handshake, Server hello (2):
13:54:23.108157 http.c:870              == Info: TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
13:54:23.108157 http.c:870              == Info: TLSv1.3 (IN), TLS handshake, Certificate (11):
13:54:23.109158 http.c:870              == Info: TLSv1.3 (IN), TLS handshake, CERT verify (15):
13:54:23.109158 http.c:870              == Info: TLSv1.3 (IN), TLS handshake, Finished (20):
13:54:23.109158 http.c:870              == Info: TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
13:54:23.109158 http.c:870              == Info: TLSv1.3 (OUT), TLS handshake, Finished (20):
13:54:23.109158 http.c:870              == Info: SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
13:54:23.109158 http.c:870              == Info: ALPN: server accepted h2
13:54:23.109158 http.c:870              == Info: Server certificate:
13:54:23.109158 http.c:870              == Info:  subject: C=US; ST=California; L=San Francisco; O=GitHub, Inc.; CN=github.com
13:54:23.109158 http.c:870              == Info:  start date: Feb 14 00:00:00 2023 GMT
13:54:23.109158 http.c:870              == Info:  expire date: Mar 14 23:59:59 2024 GMT
13:54:23.109158 http.c:870              == Info:  subjectAltName: host "github.com" matched cert's "github.com"
13:54:23.109158 http.c:870              == Info:  issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS Hybrid ECC SHA384 2020 CA1
13:54:23.109158 http.c:870              == Info:  SSL certificate verify ok.
13:54:23.109158 http.c:870              == Info: using HTTP/2
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] OPENED stream for https://github.com/<Company>/<repo>.git/info/refs?service=git-receive-pack
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] [:method: GET]
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] [:scheme: https]
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] [:authority: github.com]
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] [:path: /<Company>/<repo>.git/info/refs?service=git-receive-pack]
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] [user-agent: git/2.43.0.windows.1]
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] [accept: */*]
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] [accept-encoding: deflate, gzip, br, zstd]
13:54:23.109158 http.c:870              == Info: [HTTP/2] [1] [pragma: no-cache]
13:54:23.109158 http.c:817              => Send header, 0000000218 bytes (0x000000da)
13:54:23.109158 http.c:829              => Send header: GET /<Company>/<repo>.git/info/refs?service=git-receive-pack HTTP/2
13:54:23.109158 http.c:829              => Send header: Host: github.com
13:54:23.109158 http.c:829              => Send header: User-Agent: git/2.43.0.windows.1
13:54:23.109158 http.c:829              => Send header: Accept: */*
13:54:23.109158 http.c:829              => Send header: Accept-Encoding: deflate, gzip, br, zstd
13:54:23.109158 http.c:829              => Send header: Pragma: no-cache
13:54:23.109158 http.c:829              => Send header:
13:54:23.152771 http.c:870              == Info: TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
13:54:23.152771 http.c:870              == Info: TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
13:54:23.152771 http.c:870              == Info: old SSL session ID is stale, removing
13:54:23.167782 http.c:817              <= Recv header, 0000000013 bytes (0x0000000d)
13:54:23.167782 http.c:829              <= Recv header: HTTP/2 401
13:54:23.167782 http.c:817              <= Recv header, 0000000026 bytes (0x0000001a)
13:54:23.167782 http.c:829              <= Recv header: server: GitHub-Babel/3.0
13:54:23.167782 http.c:817              <= Recv header, 0000000054 bytes (0x00000036)
13:54:23.167782 http.c:829              <= Recv header: content-security-policy: default-src 'none'; sandbox
13:54:23.167782 http.c:817              <= Recv header, 0000000041 bytes (0x00000029)
13:54:23.167782 http.c:829              <= Recv header: content-type: text/plain; charset=UTF-8
13:54:23.167782 http.c:817              <= Recv header, 0000000040 bytes (0x00000028)
13:54:23.167782 http.c:829              <= Recv header: www-authenticate: Basic realm="GitHub"
13:54:23.167782 http.c:817              <= Recv header, 0000000020 bytes (0x00000014)
13:54:23.167782 http.c:829              <= Recv header: content-length: 21
13:54:23.167782 http.c:817              <= Recv header, 0000000037 bytes (0x00000025)
13:54:23.167782 http.c:829              <= Recv header: date: Tue, 16 Jan 2024 19:54:25 GMT
13:54:23.167782 http.c:817              <= Recv header, 0000000055 bytes (0x00000037)
13:54:23.167782 http.c:829              <= Recv header: x-github-request-id: E013:4E25:264732:387513:65A6DEF1
13:54:23.167782 http.c:817              <= Recv header, 0000000023 bytes (0x00000017)
13:54:23.167782 http.c:829              <= Recv header: x-frame-options: DENY
13:54:23.167782 http.c:817              <= Recv header, 0000000002 bytes (0x00000002)
13:54:23.167782 http.c:829              <= Recv header:
13:54:23.167782 http.c:870              == Info: Connection #0 to host github.com left intact
13:54:23.168782 run-command.c:659       trace: run_command: 'git credential-manager get'
13:54:23.306144 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:54:23.306754 git.c:749               trace: exec: git-credential-manager get
13:54:23.306754 run-command.c:659       trace: run_command: git-credential-manager get
13:54:23.544733 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:54:23.544733 git.c:463               trace: built-in: git config --null --list
13:54:23.652562 http.c:870              == Info: Found bundle for host: 0x20698d03d10 [can multiplex]
13:54:23.652562 http.c:870              == Info: Re-using existing connection with host github.com
13:54:23.652562 http.c:870              == Info: Server auth using Basic with user 'kabua'
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] OPENED stream for https://github.com/<Company>/<repo>.git/info/refs?service=git-receive-pack
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [:method: GET]
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [:scheme: https]
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [:authority: github.com]
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [:path: /<Company>/<repo>.git/info/refs?service=git-receive-pack]
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [authorization: Basic <redacted>]
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [user-agent: git/2.43.0.windows.1]
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [accept: */*]
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [accept-encoding: deflate, gzip, br, zstd]
13:54:23.652562 http.c:870              == Info: [HTTP/2] [3] [pragma: no-cache]
13:54:23.652562 http.c:817              => Send header, 0000000305 bytes (0x00000131)
13:54:23.652562 http.c:829              => Send header: GET /<Company>/<repo>.git/info/refs?service=git-receive-pack HTTP/2
13:54:23.652562 http.c:829              => Send header: Host: github.com
13:54:23.652562 http.c:829              => Send header: Authorization: Basic <redacted>
13:54:23.652562 http.c:829              => Send header: User-Agent: git/2.43.0.windows.1
13:54:23.652562 http.c:829              => Send header: Accept: */*
13:54:23.652562 http.c:829              => Send header: Accept-Encoding: deflate, gzip, br, zstd
13:54:23.652562 http.c:829              => Send header: Pragma: no-cache
13:54:23.652562 http.c:829              => Send header:
13:54:23.769539 http.c:817              <= Recv header, 0000000013 bytes (0x0000000d)
13:54:23.769539 http.c:829              <= Recv header: HTTP/2 200
13:54:23.769539 http.c:817              <= Recv header, 0000000026 bytes (0x0000001a)
13:54:23.769539 http.c:829              <= Recv header: server: GitHub-Babel/3.0
13:54:23.769539 http.c:817              <= Recv header, 0000000060 bytes (0x0000003c)
13:54:23.769539 http.c:829              <= Recv header: content-type: application/x-git-receive-pack-advertisement
13:54:23.769539 http.c:817              <= Recv header, 0000000054 bytes (0x00000036)
13:54:23.769539 http.c:829              <= Recv header: content-security-policy: default-src 'none'; sandbox
13:54:23.769539 http.c:817              <= Recv header, 0000000040 bytes (0x00000028)
13:54:23.769539 http.c:829              <= Recv header: expires: Fri, 01 Jan 1980 00:00:00 GMT
13:54:23.769539 http.c:817              <= Recv header, 0000000018 bytes (0x00000012)
13:54:23.769539 http.c:829              <= Recv header: pragma: no-cache
13:54:23.769539 http.c:817              <= Recv header, 0000000053 bytes (0x00000035)
13:54:23.769539 http.c:829              <= Recv header: cache-control: no-cache, max-age=0, must-revalidate
13:54:23.769539 http.c:817              <= Recv header, 0000000023 bytes (0x00000017)
13:54:23.769539 http.c:829              <= Recv header: vary: Accept-Encoding
13:54:23.769539 http.c:817              <= Recv header, 0000000037 bytes (0x00000025)
13:54:23.769539 http.c:829              <= Recv header: date: Tue, 16 Jan 2024 19:54:25 GMT
13:54:23.769539 http.c:817              <= Recv header, 0000000055 bytes (0x00000037)
13:54:23.769539 http.c:829              <= Recv header: x-github-request-id: E013:4E25:264850:387692:65A6DEF1
13:54:23.769539 http.c:817              <= Recv header, 0000000023 bytes (0x00000017)
13:54:23.769539 http.c:829              <= Recv header: x-frame-options: DENY
13:54:23.769539 http.c:817              <= Recv header, 0000000002 bytes (0x00000002)
13:54:23.769539 http.c:829              <= Recv header:
13:54:23.776611 http.c:870              == Info: Connection #0 to host github.com left intact
13:54:23.777610 run-command.c:659       trace: run_command: 'git credential-manager store'
13:54:23.914243 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:54:23.915654 git.c:749               trace: exec: git-credential-manager store
13:54:23.915654 run-command.c:659       trace: run_command: git-credential-manager store
13:54:24.140631 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:54:24.141631 git.c:463               trace: built-in: git config --null --list
13:54:24.252788 pkt-line.c:85           packet:          git< # service=git-receive-pack
13:54:24.252788 pkt-line.c:85           packet:          git< 0000
13:54:24.252788 pkt-line.c:85           packet:          git< 0000000000000000000000000000000000000000 capabilities^{}\0report-status report-status-v2 delete-refs side-band-64k ofs-delta atomic object-format=sha1 quiet agent=github/spokes-receive-pack-302ef7e34db14cc0c1823fcb16b9517d4c29a87b session-id=E013:4E25:264850:387692:65A6DEF1 push-options
13:54:24.252788 pkt-line.c:85           packet:          git< 0000
13:54:24.254291 pkt-line.c:85           packet:          git> +refs/heads/master:refs/heads/master
13:54:24.254291 pkt-line.c:85           packet:          git> 0000
13:54:24.254291 run-command.c:659       trace: run_command: git send-pack --stateless-rpc --helper-status --thin --progress https://github.com/<Company>/<repo>.git/ --stdin
13:54:24.308709 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:54:24.309358 git.c:463               trace: built-in: git send-pack --stateless-rpc --helper-status --thin --progress https://github.com/<Company>/<repo>.git/ --stdin
13:54:24.309861 pkt-line.c:85           packet:          git< +refs/heads/master:refs/heads/master
13:54:24.309861 pkt-line.c:85           packet:          git< 0000
13:54:24.309861 pkt-line.c:85           packet:          git< 0000000000000000000000000000000000000000 capabilities^{}\0report-status report-status-v2 delete-refs side-band-64k ofs-delta atomic object-format=sha1 quiet agent=github/spokes-receive-pack-302ef7e34db14cc0c1823fcb16b9517d4c29a87b session-id=E013:4E25:264850:387692:65A6DEF1 push-options
13:54:24.310033 pkt-line.c:85           packet:          git< 00b50000000000000000000000000000000000000000 028a19c3371ec27f4cd178aa00be4531419153bd refs/heads/master\0 report-status-v2 side-band-64k object-format=sha1 agent=git/2.43.0.windows.10000
13:54:24.309861 pkt-line.c:85           packet:          git< 0000
13:54:24.310033 pkt-line.c:85           packet:          git> 0000000000000000000000000000000000000000 028a19c3371ec27f4cd178aa00be4531419153bd refs/heads/master\0 report-status-v2 side-band-64k object-format=sha1 agent=git/2.43.0.windows.1
13:54:24.310033 pkt-line.c:85           packet:          git> 0000
13:54:24.310033 run-command.c:659       trace: run_command: git pack-objects --all-progress-implied --revs --stdout --thin --delta-base-offset --progress
13:54:24.365297 exec-cmd.c:244          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:54:24.365854 git.c:463               trace: built-in: git pack-objects --all-progress-implied --revs --stdout --thin --delta-base-offset --progress
Enumerating objects: 1149, done.
Counting objects: 100% (1149/1149), done.
Delta compression using up to 28 threads
Compressing objects: 100% (796/796), done.
13:54:24.374196 pkt-line.c:85           packet:          git< PACK ...
Writing objects: 100% (1149/1149), 25.92 MiB | 428.13 MiB/s, done.
Total 1149 (delta 326), reused 1149 (delta 326), pack-reused 0
13:54:24.438880 pkt-line.c:85           packet:          git> 0000
13:54:24.438880 http.c:870              == Info: Found bundle for host: 0x20698d03d10 [can multiplex]
13:54:24.438880 http.c:870              == Info: Re-using existing connection with host github.com
13:54:24.438880 http.c:870              == Info: Server auth using Basic with user 'kabua'
13:54:24.438880 http.c:870              == Info: [HTTP/2] [5] OPENED stream for https://github.com/<Company>/<repo>.git/git-receive-pack
13:54:24.438880 http.c:870              == Info: [HTTP/2] [5] [:method: POST]
13:54:24.438880 http.c:870              == Info: [HTTP/2] [5] [:scheme: https]
13:54:24.438880 http.c:870              == Info: [HTTP/2] [5] [:authority: github.com]
13:54:24.438880 http.c:870              == Info: [HTTP/2] [5] [:path: /<Company>/<repo>.git/git-receive-pack]
13:54:24.438880 http.c:870              == Info: [HTTP/2] [5] [authorization: Basic <redacted>]
13:54:24.439882 http.c:870              == Info: [HTTP/2] [5] [user-agent: git/2.43.0.windows.1]
13:54:24.439882 http.c:870              == Info: [HTTP/2] [5] [accept-encoding: deflate, gzip, br, zstd]
13:54:24.439882 http.c:870              == Info: [HTTP/2] [5] [content-type: application/x-git-receive-pack-request]
13:54:24.439882 http.c:870              == Info: [HTTP/2] [5] [accept: application/x-git-receive-pack-result]
13:54:24.439882 http.c:870              == Info: [HTTP/2] [5] [content-length: 27181056]
13:54:24.439882 http.c:817              => Send header, 0000000384 bytes (0x00000180)
13:54:24.439882 http.c:829              => Send header: POST /<Company>/<repo>.git/git-receive-pack HTTP/2
13:54:24.439882 http.c:829              => Send header: Host: github.com
13:54:24.439882 http.c:829              => Send header: Authorization: Basic <redacted>
13:54:24.439882 http.c:829              => Send header: User-Agent: git/2.43.0.windows.1
13:54:24.439882 http.c:829              => Send header: Accept-Encoding: deflate, gzip, br, zstd
13:54:24.439882 http.c:829              => Send header: Content-Type: application/x-git-receive-pack-request
13:54:24.439882 http.c:829              => Send header: Accept: application/x-git-receive-pack-result
13:54:24.439882 http.c:829              => Send header: Content-Length: 27181056
13:54:24.439882 http.c:829              => Send header:
13:54:25.824506 http.c:817              <= Recv header, 0000000013 bytes (0x0000000d)
13:54:25.824506 http.c:829              <= Recv header: HTTP/2 500
13:54:25.824506 http.c:817              <= Recv header, 0000000026 bytes (0x0000001a)
13:54:25.824506 http.c:829              <= Recv header: server: GitHub-Babel/3.0
13:54:25.824506 http.c:817              <= Recv header, 0000000054 bytes (0x00000036)
13:54:25.824506 http.c:829              <= Recv header: content-security-policy: default-src 'none'; sandbox
13:54:25.824506 http.c:817              <= Recv header, 0000000041 bytes (0x00000029)
13:54:25.824506 http.c:829              <= Recv header: content-type: text/plain; charset=UTF-8
13:54:25.824506 http.c:817              <= Recv header, 0000000040 bytes (0x00000028)
13:54:25.824506 http.c:829              <= Recv header: www-authenticate: Basic realm="GitHub"
13:54:25.824506 http.c:817              <= Recv header, 0000000020 bytes (0x00000014)
13:54:25.824506 http.c:829              <= Recv header: content-length: 21
13:54:25.824506 http.c:817              <= Recv header, 0000000037 bytes (0x00000025)
13:54:25.824506 http.c:829              <= Recv header: date: Tue, 16 Jan 2024 19:54:27 GMT
13:54:25.824506 http.c:817              <= Recv header, 0000000055 bytes (0x00000037)
13:54:25.824506 http.c:829              <= Recv header: x-github-request-id: E013:4E25:2649A4:38788C:65A6DEF1
13:54:25.824506 http.c:817              <= Recv header, 0000000023 bytes (0x00000017)
13:54:25.824506 http.c:829              <= Recv header: x-frame-options: DENY
13:54:25.824506 http.c:870              == Info: HTTP error before end of send, stop sending
13:54:25.824506 http.c:817              <= Recv header, 0000000002 bytes (0x00000002)
13:54:25.824506 http.c:829              <= Recv header:
13:54:25.824506 http.c:870              == Info: Connection #0 to host github.com left intact
error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly
Everything up-to-date

[GauravScripts]

Increase the Git buffer size by using git config --global http.postBuffer 524288000
try shallow clone also if issue not resolved

[kabua]

@GauravScripts thanks. But that didn't work. Using the SO question we ran git fsck, and discovered that someone had checked in the .git folder a long time ago. Therefore, we tried to remove it using git filter-branch but got this error.

'Rewrite 8522d815a3c2b83ed56d5f8f8b7f26cf9095d3ab (1/20) (0 seconds passed, remaining 0 predicted) error: invalid path '<repo>/.git/COMMIT_EDITMSG' Could not initialize the index.'

Any ideas on how to solve this new issue? thanks.

[GauravScripts]

you can use the BFG Repo-Cleaner, an alternative to git filter-branch
after that do that
git reflog expire --expire=now --all && git gc --prune=now --aggressive
let me know is issue resolved @kabua

[SmartwebsOwner]

Yes we finally got it working. Thanks.

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬