Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Git push has some rejections #36

Open
kazoompa opened this issue Mar 17, 2014 · 11 comments
Open

Git push has some rejections #36

kazoompa opened this issue Mar 17, 2014 · 11 comments

Comments

@kazoompa
Copy link

@kazoompa kazoompa commented Mar 17, 2014

Hi,

I recently used BFG to remove some old files and change a few texts in my test files. The clean operations went all OK without a problem. When I tried to push the changes I had the following error message at the end:

{code}
error: failed to push some refs to '<MY_REPOSITORY>'
{code}

These are some of the rejected messages:

{code}
! [remote rejected] refs/pull/1/head -> refs/pull/1/head (deny updating a hidden ref)
! [remote rejected] refs/pull/1/merge -> refs/pull/1/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/10/head -> refs/pull/10/head (deny updating a hidden ref)
! [remote rejected] refs/pull/10/merge -> refs/pull/10/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/11/head -> refs/pull/11/head (deny updating a hidden ref)
{code}

Am I in trouble? Did I just mess up my repository? Is there anyway to correct this error?

I would really appreciate any help, thanks.

@rtyley
Copy link
Owner

@rtyley rtyley commented Mar 17, 2014

You're not in trouble! However this is something I could cover with better documentation - issue #16 relates to this. The refs beginning 'refs/pull' are synthetic read-only refs created by GitHub - you can't update (and therefore 'clean') them, because they reflect branches that may well actually come from other repositories - ones that submitted pull-requests to you.

So, while you've pushed all your real refs, the pull requests don't get updated. There's no real way to fix them with your existing repo with the GitHub admin tools. The severity of an issue depends on whether you were trying to remove any 'private' data:

  • Private data : the pull requests still have that private data in their history - they will remain as a way for other people to retrieve that data. The only ways to recover from this are to a) delete the repo using the GitHub admin tools, and then recreate it, or b) contact GitHub support and ask them to delete the PRs for you (they're pretty good about it)
  • Nothing Private : the pull requests still reference your old history - if any of them are still open, you don't want to merge them into your new fresh history! Get the submitters to rebase them on your new fresh history before merging.
@isomorphisms
Copy link

@isomorphisms isomorphisms commented Jul 23, 2015

So if you get this error does that mean BFG has fixed the problem, or not?

@algarecu
Copy link

@algarecu algarecu commented Aug 17, 2015

Hey there, could this be possibly related to the fact that BFG has duplicated most of my commit history after deleting a file from it? I have the bfg-report for reference.

@rtyley
Copy link
Owner

@rtyley rtyley commented Aug 17, 2015

Hey there, could this be possibly related to the fact that BFG has duplicated most of my commit history after deleting a file from it? I have the bfg-report for reference.

In order to change Git history (ie to change a repo to make it look like that file was never added) brand new commits have to be created. This is how Git works - every commit-id reflects a hash of it's history - change the history, change the commit id. To delete a file, you have to create new 'clean' commit history, and throw away the old 'dirty' history.

So yes, your commit history has been duplicated, but that's essential. And this can be awkward if you have old pull requests lying around, pointing to the old history.

@takecare
Copy link

@takecare takecare commented Aug 11, 2016

does this mean that bfg can't "clean" your history on any github repository that has had pull requests "touching" any file you're trying to remove?

what i mean is, you look at the history (on github) and you can see that the history has been re-written and the file(s) you're cleaning out cannot be seen on the commits but if you go through the pull request history or if you still have the hash for the commit the file was introduced in (as an example), you can still see it (and its contents). unless i'm wrong.

so in order for someone to remove a file for sure they'd have to:

  1. run bfg, re-writing history
  2. push (get this rejected error but history has been re-written)
  3. ask github staff to delete all the PRs involving this file (?)
  4. what about the commits?

i have a public repo in which (i believe) this issue is easily seen, in case i didn't explain myself appropriately.

@jochenwezel
Copy link

@jochenwezel jochenwezel commented Oct 6, 2016

I just try to run bfg without deleting any files, without any open push requests, and it fails, too:

This is what I did:

D:\temp-cleanup-cwm>git clone --mirror https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
Cloning into bare repository 'cammIntegrationPortal.git'...
remote: Counting objects: 5979, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 5979 (delta 1), reused 0 (delta 0), pack-reused 5976 eceiving objects: 100% (5979/5979), 14.99 MiB | 2.27
Receiving objects: 100% (5979/5979), 15.78 MiB | 2.27 MiB/s, done.

Resolving deltas: 100% (3801/3801), done.
Checking connectivity... done.

D:\temp-cleanup-cwm>cd temp-cleanup-cwm
Das System kann den angegebenen Pfad nicht finden.

D:\temp-cleanup-cwm>cd cammIntegrationPortal.git

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 5979, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5871/5871), done.
Writing objects: 100% (5979/5979), done.
Total 5979 (delta 3984), reused 1806 (delta 0)

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>

After that I tried to follow the suggestions at http://stackoverflow.com/questions/34265266/remote-rejected-errors-after-mirroring-a-git-repository and did following:

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git config -e

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git remote update
Fetching origin

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 5979, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5871/5871), done.
Writing objects: 100% (5979/5979), done.
Total 5979 (delta 3984), reused 1992 (delta 0)

D:\temp-cleanup-cwm\cammIntegrationPortal.git>git push
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
 ! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/6/head -> refs/pull/6/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/CompuMasterGmbH/cammIntegrationPortal.git'

D:\temp-cleanup-cwm\cammIntegrationPortal.git>

with the following config with updated fetches:

[core]
        repositoryformatversion = 0
        filemode = false
        bare = true
        symlinks = false
        ignorecase = true
        hideDotFiles = dotGitOnly
[remote "origin"]
        url = https://github.com/CompuMasterGmbH/cammIntegrationPortal.git
        fetch = +refs/heads/*:refs/heads/*
        fetch = +refs/tags/*:refs/tags/*
        fetch = +refs/change/*:refs/change/*
        mirror = true

But still without effect (see output above) :-(

The repository in this case is:
image

No (open) push requests, only 1 additional branch - so it should be very simply, isn't it?

@javabrett
Copy link
Contributor

@javabrett javabrett commented Oct 8, 2016

No (open) push requests, only 1 additional branch - so it should be very simply, isn't it?

I think you mean Pull Requests. You are distinguishing open PRs from closed ones, however GitHub is retaining a (read-only) ref for each PR, regardless of whether it is open or closed. This allows you to later view the PR and the diffs it represents, even if that history would have been otherwise lost/garbage-collected if it were not merged into an ongoing branch.

Since you originally followed the BFG documentation and used --mirror, your config would have had both mirror = true and a complete-wildcard fetch refspec of fetch = +refs/*:refs/*. You have since amended to fetch config, but the mirror config remains. This means your git push will behave like it has the --mirror switch added. Assuming you had already fetched and rewritten all your refs/pull refs, this will account for the read-only error.

How you deal with this depends on what you are using BFG for and what you want to achieve. If you want sensitive or large files completely removed from the GitHub repo, then you need to contact GitHub support according to https://help.github.com/articles/remove-sensitive-data/ (emphasis mine):

This article tells you how to make commits with sensitive data unreachable from any branches or tags in your GitHub repository. However, it's important to note that those commits may still be accessible in any clones or forks of your repository, directly via their SHA-1 hashes in cached views on GitHub, and through any pull requests that reference them. You can't do anything about existing clones or forks of your repository, but you can permanently remove all of your repository's cached views and pull requests on GitHub by contacting GitHub Support.

In other words, GitHub Support might be able to provide further info on how you can discard the old history referenced by PRs.

If you just want bulk removed from what default (non-mirror) clones see, then you don't have to worry about these, as they aren't fetched by default, but be aware that the size of the repo on GitHub will actually increase slightly as a result.

@markkolich
Copy link

@markkolich markkolich commented Feb 6, 2019

Since this is a top search result for "deny updating a hidden ref", I figured I'd leave this here to help out the next person...

If you've used BFG to clean up a repo, and you're trying to force push back to GitHub, make sure you're not trying to force push back to a protected branch. I wasted several hours staring at [remote rejected] ... (deny updating a hidden ref) thinking the discussion in this issue was the cause of my grief, only to realize it's a total red herring.

Buried deep in the deny updating a hidden ref noise, was a "protected branch hook declined" failure right in my terminal.

If you're in the same boat, disable/delete branch protection, and try another git push -f --mirror:

[mark (master)]~/workspace/foobar.mirror$ git push -f --mirror
...
remote: Resolving deltas: 100% (34037/34037), done.
To https://github.com/foo/bar.git
 + a5fb13391...9b66d3db1 master -> master (forced update)    <<<---!!!!!!
 + 0978357d5...f987b9bd2 release -> release (forced update)    <<<---!!!!!!
...
 ! [remote rejected]     refs/pull/100/head -> refs/pull/100/head (deny updating a hidden ref)

Note the familiar (forced update) from Git indicating the git push -f --mirror worked fine. Subsequent clones of this repo after the force push also confirmed success.

TL;DR — I could be totally mistaken, but the [remote rejected] ... (deny updating a hidden ref) complaints appear to be just warnings (for the reasons covered earlier in this thread) and do not seem to impact the ultimate success of a force push.

@rtyley perhaps something to clarify in future documentation?

@antoniocascais
Copy link

@antoniocascais antoniocascais commented Oct 17, 2019

We faced the same problem described in this issue. We had an existent PR in our Repo, so we assume that was the root cause of the push rejection.

Here's what we did (let's say that our repo was named awesome-tool):

  1. Created a new empty repo with name awesome-tool-tmp;
  2. In our cloned repo, we changed the URL with git set-url to the url of the new awesome-tool-tmp repo;
  3. git push (at this point, both awesome-tool and awesome-tool-tmp have the same content and commits in master;
  4. used the tool as described here https://rtyley.github.io/bfg-repo-cleaner/. git push now worked (because there was no PR in the tmp repo, we assume)
  5. renamed awesome-tool to awesome-tool-tmp2
  6. renamed awesome-tool-tmp to awesome-tool
  7. deleted awesome-tool-tmp2 (the repo with the passwords)
  8. asked the ppl to delete their folders and clone the repo again

And we were able to successfully remove passwords from git history.

Might not be the best solution for everyone, but in our case was enough.

@rodrigolece
Copy link

@rodrigolece rodrigolece commented Sep 8, 2020

I think that I accidentally found a solution (I'm not sure I can explain why it works). Same thing, I had PRs (closed) on GitHub and git push was being rejected.

  1. I cloned the local copy of my repo (using git clone --mirror but providing the path to my local directory).
  2. I cleaned this copy using BFG.
  3. git remote was showing origin as pointing to my local repo, so I removed origin and added it but this time pointing to my repo hosted on GitHub.
  4. From the clean mirror copy, I forced a push using git push -f --set-upstream origin master.

As far as I can tell, my repo in GitHub is now clean. Git pulling from my original repo doesn't work, but now I can easily delete that folder and clone a new copy.

Any ideas for/against doing it this way?

@calebTree
Copy link

@calebTree calebTree commented Jun 3, 2021

Mirror cloning from a local copy path apparently doesn't include pull request refs. This can be verified using git ls-remote. BFG still doesn't apparently seem to be built to clean pull requests at all. Here is a tl;dr blog I found with some more details https://www.secjuice.com/github-complete-cleaning-sensitive-secrets/.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet