Push fails when pushed with mirror option and repo contains 1000s of refs in 'refs/replace'

[thejewdude]

My team and I are about to move all our repos to Github but I’m having trouble pushing our main repo. As part of the migration we shrunk our repo size by rewriting the git history to remove large files that we no longer need. The tool we used, git-filter-repo, created a 1000s of replace refs under refs/replace and when I try pushing the repo with git push origin --mirror I get a remote: Internal Server Error. I attached an image that shows the error.

Any help would be appreciated, and it would be great if I can tell what the internal server error is

[canuckjacq]

👋 Welcome!

500 is a hard one to troubleshoot! Have you resolved this yet?

[thejewdude]

Sort of.
I reached out to support by email and one of your colleagues indicated that its a known limitation that GitHub will timeout when trying to push too many refs at once. Their suggestion was to push the refs in smaller chunks. I accomplished this with a wee bit of commandline hackery

git for-each-ref --format="%(refname)" --sort='authordate' refs/replace | xargs git push origin

[xshahbazintel]

Hi @thejewdude

I am facing same error while migrating code from gerrit to github, can you please elaborate what have you done.

TIA

[thejewdude]

Basically the command line I posted above pushes the refs in smaller chunks. I’m using the Linux program xargs to automatically chunk pushing the refs (I think it was about 500 at a time) In my case I had > 10,000 replace refs, that’s why my command line mentions refs/replace.

Let me know if something still isn’t clear!

[xshahbazintel]

Thanks, it worked for some repos but unfortunately for some its failing
using this command for push
git for-each-ref --format="%(refname)" --sort='authordate' | xargs git push origin
before executing the above command i am executing this
git config --unset remote.origin.mirror

error820×167 9.07 KB

[thejewdude]

What does unsetting the mirror do?
Is it possible that you’re pushing refs that already exist on the remote? If you’re pushing to the same repo that tried to push to earlier (and failed), it be causing problems. You might want to try my command line with a fresh repo.
You should also try changing the amount of args that xargs will run at a time. For me it worked with < 500 but YMMV

[xshahbazintel]

I am unsetting the mirror because i faced this
error: --mirror can't be combined with refspecs, when i googled some one suggested to unset mirror.
anyhow i pushed the tags and branches separately.
Thanks for response

[bhalbright]

FWIW, I added -P 3 to the command from @thejewdude to run some of the pushes in parallel, got my repo to push in about 30 minutes vs 2 hours before. And thanks so much @thejewdude, your script saved me hours of frustration :)

git for-each-ref --format="%(refname)" --sort='authordate' refs/replace | xargs -P 3 git push origin