Allow for default branch change on custom tap

[q0rban]

A detailed description of the proposed feature

Over at Tugboat, we're working on updating our various GitHub repositories to switch the default branch from master to main, per https://tools.ietf.org/html/draft-knodel-terminology-01. As such, we went to update the default branch for the Homebrew Tap for our CLI. After adding a new main branch and deleting the master branch, it seems that existing installs of our CLI with Homebrew fail with the following message:

==> Unshallowing tugboatqa/tugboat
fatal: Couldn't find remote ref refs/heads/master
Error: Failure while executing; `git fetch --unshallow` exited with 128.

Would it be feasible to allow for branch renaming on taps without breaking existing installs?

The motivation for the feature

Allowing for Homebrew developers to change their default branch without breaking existing installations.

How the feature would be relevant to at least 90% of Homebrew users

Allowing for Homebrew developers to change their default branch without breaking existing installations.

What alternatives to the feature have been considered

I cannot think of alternatives.

[MikeMcQuaid]

it seems that existing installs of our CLI with Homebrew fail with the following message:

What command are you running to produce this output?

[MikeMcQuaid]

Thinking about this more:

After adding a new main branch and deleting the master branch, it seems that existing installs of our CLI with Homebrew fail with the following message:

I'm 99% sure you're seeing that Git/GitHub do not support "renaming branches". As far as all local Git checkouts of your repository (which is what a Homebrew tap is) are concerned: you have deleted the master branch and it doesn't know where to look instead.

[q0rban]

Wow, yeah, this is tricky with git. Renaming the branch is easy with git branch -m [old] [new], however if you delete the old branch on origin (GitHub), existing checkouts are broken with no easy way to recover. Even trying to checkout origin/HEAD directly doesn't work.

$ git fetch origin && git checkout origin/HEAD
warning: ignoring dangling symref refs/remotes/origin/HEAD.
error: pathspec 'origin/HEAD' did not match any file(s) known to git.

And cleaning up doesn't work.

$ git gc
fatal: bad object refs/remotes/origin/HEAD
error: failed to run repack

Most suggestions online say you should, "just delete the .git folder and recheck out the repo," which isn't very helpful for our purposes.

Reading through man git-fetch, I found a few other things to try, but those didn't help either.

$ git fetch --update-head-ok --prune
From .
 * branch            origin     -> FETCH_HEAD

$ git checkout origin/HEAD
warning: ignoring dangling symref refs/remotes/origin/HEAD.
error: pathspec 'origin/HEAD' did not match any file(s) known to git.

I can think of some hacky ways around this for Homebrew, but I'm beginning to think this is not your problem to solve.

I'll leave this open in case others have ideas.

[MikeMcQuaid]

however if you delete the old branch on origin (GitHub), existing checkouts are broken with no easy way to recover. Even trying to checkout origin/HEAD directly doesn't work.

Yup 😭.

I can think of some hacky ways around this for Homebrew, but I'm beginning to think this is not your problem to solve.

Sadly, I agree. This is something probably best raised with GitHub and/or Git directly.

I'll leave this open in case others have ideas.

As per-usual on our issue tracker: I'm going to close this out as there's no actionable work for Homebrew maintainers or contributors. I'll pin this (closed) issue, though, so others can see it.

[sjackman]

Hi, James. I created PR #7758 to fetch origin/HEAD, when it hasn't already been fetched. That PR has been merged.
I've opened PR #7769, which fixes brew update-reset when the default upstream branch is not named master. After that PR is merged, you can fix an existing clone of your repo using

brew update-reset $(brew --repo tugboatqa/tugboat)

You could also simply brew untap tugboatqa/tugboat; brew tap tugboatqa/tugboat.
I hope that helps!

[sjackman]

@q0rban PRs #7758 and #7769 have been merged, and Homebrew 2.4.1 including this fix has been tagged and released. https://github.com/Homebrew/brew/releases/tag/2.4.1

Does brew update-reset $(brew --repo tugboatqa/tugboat) work for you to change the default branch from master to main?

[q0rban]

@sjackman wow, fantastic work! We ended up sticking with master for now, but I'll notify our developer that we have a path forward. Perhaps we can put a script in the master branch that outputs the help text on how to upgrade.

[sjackman]

😁Thanks! It's possible that brew update could be modified to catch this particular branch renaming error and fix it. I'll defer that investigation until your developer has a chance to look into it.

[q0rban]

We gave this a shot and were still getting errors.

$ brew --version
Homebrew 2.4.1

$ brew update-reset $(brew --repo tugboatqa/tugboat)
==> Fetching /usr/local/Homebrew/Library/Taps/tugboatqa/homebrew-tugboat...
fatal: Couldn't find remote ref refs/heads/master
error: Not a valid ref: refs/remotes/origin/main

==> Resetting /usr/local/Homebrew/Library/Taps/tugboatqa/homebrew-tugboat...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

$ brew update
fatal: Couldn't find remote ref refs/heads/master
Error: Fetching /usr/local/Homebrew/Library/Taps/tugboatqa/homebrew-tugboat failed!

So, I think we will keep the master branch around for a while and ship a new version that outputs a warning message on how to update.

[sjackman]

😢 A PR to fix this issue is welcome if you'd like to open one. If you have a halfway working fix, and want to discuss it, you can open a draft PR, and we can chat there.

[nebhale]

@MikeMcQuaid I believe that this feature does work for deep clones:

➜  ~ brew tap nebhale/test https://github.com/nebhale/test.git
==> Tapping nebhale/test
Cloning into '/usr/local/Homebrew/Library/Taps/nebhale/homebrew-test'...
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), 4.69 KiB | 961.00 KiB/s, done.
Tapped (31 files, 41.3KB).

➜  ~ cd /usr/local/Homebrew/Library/Taps/nebhale/homebrew-test && git branch -a && cd -
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
~

➜  ~ brew update-reset $(brew --repo nebhale/test)                                     
==> Fetching /usr/local/Homebrew/Library/Taps/nebhale/homebrew-test...
From https://github.com/nebhale/test
 * [new branch]      main       -> origin/main

==> Resetting /usr/local/Homebrew/Library/Taps/nebhale/homebrew-test...
Branch 'main' set up to track remote branch 'main' from 'origin'.
Switched to a new branch 'main'

➜  ~ cd /usr/local/Homebrew/Library/Taps/nebhale/homebrew-test && git branch -a && cd -
* main
  master
  remotes/origin/HEAD -> origin/main
  remotes/origin/main
  remotes/origin/master
~

There does appear to be a problem with shallow though:

➜  ~ brew tap --shallow nebhale/test https://github.com/nebhale/test.git
==> Tapping nebhale/test
Cloning into '/usr/local/Homebrew/Library/Taps/nebhale/homebrew-test'...
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), 4.69 KiB | 961.00 KiB/s, done.
Tapped (32 files, 41.4KB).
Time: 0h:00m:05s                                                                                                                                                                                                                                                                         

➜  ~ cd /usr/local/Homebrew/Library/Taps/nebhale/homebrew-test && git branch -a && cd -
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
~

➜  ~ brew update-reset $(brew --repo nebhale/test)                      
==> Fetching /usr/local/Homebrew/Library/Taps/nebhale/homebrew-test...
fatal: couldn't find remote ref refs/heads/master
error: Not a valid ref: refs/remotes/origin/main

==> Resetting /usr/local/Homebrew/Library/Taps/nebhale/homebrew-test...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

➜  ~ cd /usr/local/Homebrew/Library/Taps/nebhale/homebrew-test && git branch -a && cd -
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
~

[sjackman]

Thank you for the reproducible example, Ben, and how strange!

[nebhale]

@sjackman I think the explanation is that when you shallow clone, it adds a bunch of restrictions to that clone. For example, the git fetch --unshallow documentation uses the following prose:

If the source repository is complete, convert a shallow repository to a complete one, removing all the limitations imposed by shallow repositories.

So my guess is that one of those limitations is that you can't swap the origin/HEAD in the local shallow clone because there's no way for that clone to ever learn about another branch.

I was doing a bunch of experimentation this morning and there are a couple of reasonable strategies for making this migration, but what holds me up from suggesting or PR-ing any of them is that I haven't yet found how to use the Git command line to determine what a remote's default HEAD is, without using sed.

[sjackman]

I had to read the git source code to find the answer to this question!

$ git remote show origin | grep HEAD
  HEAD branch: develop

[nebhale]

Yeah, so extracting that and doing something useful requires sed and is pretty brittle. Not sure if that's the way to go about this effort.

[sjackman]

Based on my searching of the Git codebase, I don't believe there is another way to fetch the default branch other than git remote show origin and git remote set-head origin -a. You can also fetch it using the GitHub API. That only works for third-party taps stored on GitHub.

$ curl -s https://api.github.com/repos/brewsci/homebrew-bio | jq -r .default_branch
develop

sed or awk is just fine in this case.

$ git remote show origin | awk '/^  HEAD branch: / {print $3}'
develop

[q0rban]

Our workaround, in case it is helpful for others, is to add a script to the master branch that outputs a warning and error with instructions on how to upgrade

https://github.com/TugboatQA/homebrew-tugboat/blob/master/Formula/tugboat-cli.rb

class TugboatCli < Formula
  desc "Tugboat CLI"
  homepage "https://tugboat.qa"
  url "https://dashboard.tugboat.qa/cli/macos/tugboat.tar.gz"
  version "2.20.33"

  ohai "Language Matters"

  opoo "Tugboat is updating its language in an effort to be more inclusive."
  opoo "Part of that effort includes renaming our default git branches from"
  opoo "'master' to 'main'. Read more at"
  opoo ""
  opoo "  https://www.tugboat.qa/language-matters"
  opoo ""

  odie """To upgrade tugboat-cli, retap it with:
    brew untap tugboatqa/tugboat
    brew tap tugboatqa/tugboat
    brew upgrade tugboat-cli"""

end

Which then results in:

[sjackman]

Tricksy! Nice hack.