New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Git push has some rejections #36

Open
kazoompa opened this Issue Mar 17, 2014 · 7 comments

Comments

Projects
None yet
7 participants
@kazoompa

kazoompa commented Mar 17, 2014

Hi,

I recently used BFG to remove some old files and change a few texts in my test files. The clean operations went all OK without a problem. When I tried to push the changes I had the following error message at the end:

{code}
error: failed to push some refs to '<MY_REPOSITORY>'
{code}

These are some of the rejected messages:

{code}
! [remote rejected] refs/pull/1/head -> refs/pull/1/head (deny updating a hidden ref)
! [remote rejected] refs/pull/1/merge -> refs/pull/1/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/10/head -> refs/pull/10/head (deny updating a hidden ref)
! [remote rejected] refs/pull/10/merge -> refs/pull/10/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/11/head -> refs/pull/11/head (deny updating a hidden ref)
{code}

Am I in trouble? Did I just mess up my repository? Is there anyway to correct this error?

I would really appreciate any help, thanks.

@rtyley

This comment has been minimized.

Show comment
Hide comment
@rtyley

rtyley Mar 17, 2014

Owner

You're not in trouble! However this is something I could cover with better documentation - issue #16 relates to this. The refs beginning 'refs/pull' are synthetic read-only refs created by GitHub - you can't update (and therefore 'clean') them, because they reflect branches that may well actually come from other repositories - ones that submitted pull-requests to you.

So, while you've pushed all your real refs, the pull requests don't get updated. There's no real way to fix them with your existing repo with the GitHub admin tools. The severity of an issue depends on whether you were trying to remove any 'private' data:

  • Private data : the pull requests still have that private data in their history - they will remain as a way for other people to retrieve that data. The only ways to recover from this are to a) delete the repo using the GitHub admin tools, and then recreate it, or b) contact GitHub support and ask them to delete the PRs for you (they're pretty good about it)
  • Nothing Private : the pull requests still reference your old history - if any of them are still open, you don't want to merge them into your new fresh history! Get the submitters to rebase them on your new fresh history before merging.
Owner

rtyley commented Mar 17, 2014

You're not in trouble! However this is something I could cover with better documentation - issue #16 relates to this. The refs beginning 'refs/pull' are synthetic read-only refs created by GitHub - you can't update (and therefore 'clean') them, because they reflect branches that may well actually come from other repositories - ones that submitted pull-requests to you.

So, while you've pushed all your real refs, the pull requests don't get updated. There's no real way to fix them with your existing repo with the GitHub admin tools. The severity of an issue depends on whether you were trying to remove any 'private' data:

  • Private data : the pull requests still have that private data in their history - they will remain as a way for other people to retrieve that data. The only ways to recover from this are to a) delete the repo using the GitHub admin tools, and then recreate it, or b) contact GitHub support and ask them to delete the PRs for you (they're pretty good about it)
  • Nothing Private : the pull requests still reference your old history - if any of them are still open, you don't want to merge them into your new fresh history! Get the submitters to rebase them on your new fresh history before merging.
@isomorphisms

This comment has been minimized.

Show comment
Hide comment
@isomorphisms

isomorphisms Jul 23, 2015

So if you get this error does that mean BFG has fixed the problem, or not?

isomorphisms commented Jul 23, 2015

So if you get this error does that mean BFG has fixed the problem, or not?

@algarecu

This comment has been minimized.

Show comment
Hide comment
@algarecu

algarecu Aug 17, 2015

Hey there, could this be possibly related to the fact that BFG has duplicated most of my commit history after deleting a file from it? I have the bfg-report for reference.

algarecu commented Aug 17, 2015

Hey there, could this be possibly related to the fact that BFG has duplicated most of my commit history after deleting a file from it? I have the bfg-report for reference.

@rtyley

This comment has been minimized.

Show comment
Hide comment
@rtyley

rtyley Aug 17, 2015

Owner

Hey there, could this be possibly related to the fact that BFG has duplicated most of my commit history after deleting a file from it? I have the bfg-report for reference.

In order to change Git history (ie to change a repo to make it look like that file was never added) brand new commits have to be created. This is how Git works - every commit-id reflects a hash of it's history - change the history, change the commit id. To delete a file, you have to create new 'clean' commit history, and throw away the old 'dirty' history.

So yes, your commit history has been duplicated, but that's essential. And this can be awkward if you have old pull requests lying around, pointing to the old history.

Owner

rtyley commented Aug 17, 2015

Hey there, could this be possibly related to the fact that BFG has duplicated most of my commit history after deleting a file from it? I have the bfg-report for reference.

In order to change Git history (ie to change a repo to make it look like that file was never added) brand new commits have to be created. This is how Git works - every commit-id reflects a hash of it's history - change the history, change the commit id. To delete a file, you have to create new 'clean' commit history, and throw away the old 'dirty' history.

So yes, your commit history has been duplicated, but that's essential. And this can be awkward if you have old pull requests lying around, pointing to the old history.

@takecare

This comment has been minimized.

Show comment
Hide comment
@takecare

takecare Aug 11, 2016

does this mean that bfg can't "clean" your history on any github repository that has had pull requests "touching" any file you're trying to remove?

what i mean is, you look at the history (on github) and you can see that the history has been re-written and the file(s) you're cleaning out cannot be seen on the commits but if you go through the pull request history or if you still have the hash for the commit the file was introduced in (as an example), you can still see it (and its contents). unless i'm wrong.

so in order for someone to remove a file for sure they'd have to:

  1. run bfg, re-writing history
  2. push (get this rejected error but history has been re-written)
  3. ask github staff to delete all the PRs involving this file (?)
  4. what about the commits?

i have a public repo in which (i believe) this issue is easily seen, in case i didn't explain myself appropriately.

takecare commented Aug 11, 2016

does this mean that bfg can't "clean" your history on any github repository that has had pull requests "touching" any file you're trying to remove?

what i mean is, you look at the history (on github) and you can see that the history has been re-written and the file(s) you're cleaning out cannot be seen on the commits but if you go through the pull request history or if you still have the hash for the commit the file was introduced in (as an example), you can still see it (and its contents). unless i'm wrong.

so in order for someone to remove a file for sure they'd have to:

  1. run bfg, re-writing history
  2. push (get this rejected error but history has been re-written)
  3. ask github staff to delete all the PRs involving this file (?)
  4. what about the commits?

i have a public repo in which (i believe) this issue is easily seen, in case i didn't explain myself appropriately.

@jochenwezel

This comment has been minimized.

Show comment
Hide comment
@jochenwezel

jochenwezel Oct 6, 2016

I just try to run bfg without deleting any files, without any open push requests, and it fails, too:

This is what I did:

D:\temp-cleanup-cwm>git clone --mirror https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
Cloning into bare repository 'cammIntegrationPortal.git'...
remote: Counting objects: 5979, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 5979 (delta 1), reused 0 (delta 0), pack-reused 5976 eceiving objects: 100% (5979/5979), 14.99 MiB | 2.27
Receiving objects: 100% (5979/5979), 15.78 MiB | 2.27 MiB/s, done.

Resolving deltas: 100% (3801/3801), done.
Checking connectivity... done.

D:\temp-cleanup-cwm>cd temp-cleanup-cwm
Das System kann den angegebenen Pfad nicht finden.

D:\temp-cleanup-cwm>cd cammIntegrationPortal.git

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 5979, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5871/5871), done.
Writing objects: 100% (5979/5979), done.
Total 5979 (delta 3984), reused 1806 (delta 0)

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>

After that I tried to follow the suggestions at http://stackoverflow.com/questions/34265266/remote-rejected-errors-after-mirroring-a-git-repository and did following:

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git config -e

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git remote update
Fetching origin

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 5979, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5871/5871), done.
Writing objects: 100% (5979/5979), done.
Total 5979 (delta 3984), reused 1992 (delta 0)

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>

with the following config with updated fetches:

[core]
        repositoryformatversion = 0
        filemode = false
        bare = true
        symlinks = false
        ignorecase = true
        hideDotFiles = dotGitOnly
[remote "origin"]
        url = https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
        fetch = +refs/heads/*:refs/heads/*
        fetch = +refs/tags/*:refs/tags/*
        fetch = +refs/change/*:refs/change/*
        mirror = true

But still without effect (see output above) :-(

The repository in this case is:
image

No (open) push requests, only 1 additional branch - so it should be very simply, isn't it?

jochenwezel commented Oct 6, 2016

I just try to run bfg without deleting any files, without any open push requests, and it fails, too:

This is what I did:

D:\temp-cleanup-cwm>git clone --mirror https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
Cloning into bare repository 'cammIntegrationPortal.git'...
remote: Counting objects: 5979, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 5979 (delta 1), reused 0 (delta 0), pack-reused 5976 eceiving objects: 100% (5979/5979), 14.99 MiB | 2.27
Receiving objects: 100% (5979/5979), 15.78 MiB | 2.27 MiB/s, done.

Resolving deltas: 100% (3801/3801), done.
Checking connectivity... done.

D:\temp-cleanup-cwm>cd temp-cleanup-cwm
Das System kann den angegebenen Pfad nicht finden.

D:\temp-cleanup-cwm>cd cammIntegrationPortal.git

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 5979, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5871/5871), done.
Writing objects: 100% (5979/5979), done.
Total 5979 (delta 3984), reused 1806 (delta 0)

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>

After that I tried to follow the suggestions at http://stackoverflow.com/questions/34265266/remote-rejected-errors-after-mirroring-a-git-repository and did following:

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git config -e

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git remote update
Fetching origin

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 5979, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5871/5871), done.
Writing objects: 100% (5979/5979), done.
Total 5979 (delta 3984), reused 1992 (delta 0)

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>

with the following config with updated fetches:

[core]
        repositoryformatversion = 0
        filemode = false
        bare = true
        symlinks = false
        ignorecase = true
        hideDotFiles = dotGitOnly
[remote "origin"]
        url = https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
        fetch = +refs/heads/*:refs/heads/*
        fetch = +refs/tags/*:refs/tags/*
        fetch = +refs/change/*:refs/change/*
        mirror = true

But still without effect (see output above) :-(

The repository in this case is:
image

No (open) push requests, only 1 additional branch - so it should be very simply, isn't it?

@javabrett

This comment has been minimized.

Show comment
Hide comment
@javabrett

javabrett Oct 8, 2016

Contributor

No (open) push requests, only 1 additional branch - so it should be very simply, isn't it?

I think you mean Pull Requests. You are distinguishing open PRs from closed ones, however GitHub is retaining a (read-only) ref for each PR, regardless of whether it is open or closed. This allows you to later view the PR and the diffs it represents, even if that history would have been otherwise lost/garbage-collected if it were not merged into an ongoing branch.

Since you originally followed the BFG documentation and used --mirror, your config would have had both mirror = true and a complete-wildcard fetch refspec of fetch = +refs/*:refs/*. You have since amended to fetch config, but the mirror config remains. This means your git push will behave like it has the --mirror switch added. Assuming you had already fetched and rewritten all your refs/pull refs, this will account for the read-only error.

How you deal with this depends on what you are using BFG for and what you want to achieve. If you want sensitive or large files completely removed from the GitHub repo, then you need to contact GitHub support according to https://help.github.com/articles/remove-sensitive-data/ (emphasis mine):

This article tells you how to make commits with sensitive data unreachable from any branches or tags in your GitHub repository. However, it's important to note that those commits may still be accessible in any clones or forks of your repository, directly via their SHA-1 hashes in cached views on GitHub, and through any pull requests that reference them. You can't do anything about existing clones or forks of your repository, but you can permanently remove all of your repository's cached views and pull requests on GitHub by contacting GitHub Support.

In other words, GitHub Support might be able to provide further info on how you can discard the old history referenced by PRs.

If you just want bulk removed from what default (non-mirror) clones see, then you don't have to worry about these, as they aren't fetched by default, but be aware that the size of the repo on GitHub will actually increase slightly as a result.

Contributor

javabrett commented Oct 8, 2016

No (open) push requests, only 1 additional branch - so it should be very simply, isn't it?

I think you mean Pull Requests. You are distinguishing open PRs from closed ones, however GitHub is retaining a (read-only) ref for each PR, regardless of whether it is open or closed. This allows you to later view the PR and the diffs it represents, even if that history would have been otherwise lost/garbage-collected if it were not merged into an ongoing branch.

Since you originally followed the BFG documentation and used --mirror, your config would have had both mirror = true and a complete-wildcard fetch refspec of fetch = +refs/*:refs/*. You have since amended to fetch config, but the mirror config remains. This means your git push will behave like it has the --mirror switch added. Assuming you had already fetched and rewritten all your refs/pull refs, this will account for the read-only error.

How you deal with this depends on what you are using BFG for and what you want to achieve. If you want sensitive or large files completely removed from the GitHub repo, then you need to contact GitHub support according to https://help.github.com/articles/remove-sensitive-data/ (emphasis mine):

This article tells you how to make commits with sensitive data unreachable from any branches or tags in your GitHub repository. However, it's important to note that those commits may still be accessible in any clones or forks of your repository, directly via their SHA-1 hashes in cached views on GitHub, and through any pull requests that reference them. You can't do anything about existing clones or forks of your repository, but you can permanently remove all of your repository's cached views and pull requests on GitHub by contacting GitHub Support.

In other words, GitHub Support might be able to provide further info on how you can discard the old history referenced by PRs.

If you just want bulk removed from what default (non-mirror) clones see, then you don't have to worry about these, as they aren't fetched by default, but be aware that the size of the repo on GitHub will actually increase slightly as a result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment