Skip to content

Latest commit

聽

History

History
282 lines (218 loc) 路 10 KB

Homebrew-linuxbrew-core-Maintainer-Guide.md

File metadata and controls

282 lines (218 loc) 路 10 KB
Raw

Homebrew linuxbrew-core Maintainer Guide

Merging formulae updates from Homebrew/homebrew-core

Linuxbrew-core is a fork of Homebrew-core and, therefore, it has to periodically merge changes made by Homebrew developers and contributors. Below we describe the steps required to merge Homebrew/homebrew-core into Linuxbrew/homebrew-core, possible conflicts and ways to resolve them. Note, that instructions below have been written for a "clean" environment and you might be able to skip some of the steps if you have done them in the past.

Preparation

First of all, we want to enable developer commands and prevent automatic updates while we do the merge:

export HOMEBREW_DEVELOPER=1
export HOMEBREW_NO_AUTO_UPDATE=1

Once we've done that, we need to get access to the merge-homebrew command that will be used for the merge. To do that we have to tap the linuxbrew/homebrew-developer repo:

brew tap linuxbrew/developer

Next, we have to navigate to the repository where we want to do the merge and make sure that there are 3 remotes:

  • a remote named origin pointing to Linuxbrew-core,
  • a remote named homebrew pointing to Homebrew-core, and
  • a remote pointing to your GitHub fork of Linuxbrew-core.

Remote names origin and homebrew are hard-coded in merge-homebrew, while the remote pointing to your fork must be the same as your GitHub username, as it will be used to submit a pull request for the merge. Set the name to the $GITHUB_USER environment variable, or let hub fork add a remote for you.

brew install hub
cd $(brew --repo homebrew/core)
git remote add homebrew https://github.com/Homebrew/homebrew-core.git
hub fork --remote-name=$GITHUB_USER

Now, let's make sure that our local branch master is clean and that your fork is up-to-date with Homebrew/linuxbrew-core:

git checkout master
git fetch origin master
git reset --hard origin/master
git push --force $GITHUB_USER master

Strictly speaking, there is no need for git reset --hard origin/master and simple git merge origin/master would have been sufficient if you didn't mess with your local master branch. However, hard reset makes sure that these instructions are correct even if you did mess something up. The same is true for the --force flag for the git push command above.

By default, the following command will attempt to merge all the changes that the upstream Homebrew developers have made.

brew merge-homebrew --core

Merging all the changes from upstream in one go is usually undesireable since our build servers will time out. Instead, attempt to only merge 8-10 modified formulae.

git log --oneline master..homebrew/master will show a list of all the upstream commits since the last merge, from oldest to newest.

Pick a commit SHA-1 that will merge between 8-10 formulae (16-20 commits including bottles). Once you're satisfied with the list of updated formulae, begin the merge:

brew merge-homebrew --core --skip-style <sha>

The --skip-style argument skips running brew style, which saves time and in some cases avoids errors. The style errors can be fixed in bottle PRs later in the process when CI flags them.

Simple Conflicts

Once you issue the above command, the merge will begin and in the very end you will see the list of (conflicting) formulae that merge-homebrew could not merge automatically:

==> Conflicts
Formula/git-lfs.rb Formula/gnutls.rb Formula/godep.rb

Note, that you can also get a list of unmerged files (i.e. files with conflicts) using:

git diff --name-only --diff-filter=U

Of course, conflicts will be different every merge. You have to resolve these conflicts either manually in a text editor, or by using tools like diffuse, tkdiff, or meld, some of which are available from Homebrew. Frequently, conflicts are caused by the new versions of macOS bottles and look like:

<<<<<<< HEAD
    sha256 "bd66be269cbfe387920651c5f4f4bc01e0793034d08b5975f35f7fdfdb6c61a7" => :sierra
    sha256 "7071cb98f72c73adb30afbe049beaf947fabfeb55e9f03e0db594c568d77d69d" => :el_capitan
    sha256 "c7c0fe2464771bdcfd626fcbda9f55cb003ac1de060c51459366907edd912683" => :yosemite
    sha256 "95d4c82d38262a4bc7ef4f0a10ce2ecf90e137b67df15f8bf8df76e962e218b6" => :x86_64_linux
=======
    sha256 "ee6db42174fdc572d743e0142818b542291ca2e6ea3c20ff6a47686589cdc274" => :sierra
    sha256 "e079a92a6156e2c87c59a59887d0ae0b6450d6f3a9c1fe14838b6bc657faefaa" => :el_capitan
    sha256 "c334f91d5809d2be3982f511a3dfe9a887ef911b88b25f870558d5c7e18a15ad" => :yosemite
>>>>>>> homebrew/master

For such conflicts, simply remove the "HEAD" (Linuxbrew's) part of the conflict along with <<<<<<< HEAD, =======, and >>>>>>> homebrew/master lines. Later, we will submit a request to rebuild bottles for Linux for such formulae.

The merge-homebrew script will stage resolved conflicts for you.

Complex Conflicts

Of course, from time to time conflicts are more complicated and you have to look carefully into what's going on. An example of a slightly more complex conflict is below:

<<<<<<< HEAD
    if OS.mac?
      lib.install "out-shared/libleveldb.dylib.1.19" => "libleveldb.1.19.dylib"
      lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.dylib"
      lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.1.dylib"
      system "install_name_tool", "-id", "#{lib}/libleveldb.1.dylib", "#{lib}/libleveldb.1.19.dylib"
    else
      lib.install Dir["out-shared/libleveldb.so*"]
    end
=======
    lib.install "out-shared/libleveldb.dylib.1.19" => "libleveldb.1.19.dylib"
    lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.dylib"
    lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.1.dylib"
    MachO::Tools.change_dylib_id("#{lib}/libleveldb.1.dylib", "#{lib}/libleveldb.1.19.dylib")
>>>>>>> homebrew/master

Note, that in the "HEAD" (Linuxbrew's) part we see previous code of the Homebrew's formula wrapped in if OS.mac?. To resolve such a conflict you have to replace the contents of if OS.mac? part up until else with the contents of the bottom part of the conflict ("homebrew/master"). You also have to check if there are any obvious modifications that have to be made to the else part of the code that deals with non-macOS-related code.

Finishing the merge

Once all the conflicts have been resolved, a text editor will open with pre-populated commit message title and body:

Merge branch homebrew/master into linuxbrew/master

# Conflicts:
#       Formula/git-lfs.rb
#       Formula/gnutls.rb
#       Formula/godep.rb

Leave the title of the message unchanged and uncomment all the conflicting files. Your final commit message should be:

Merge branch homebrew/master into linuxbrew/master

Conflicts:
        Formula/git-lfs.rb
        Formula/gnutls.rb
        Formula/godep.rb

Submitting a PR

The merge-homebrew command will create a pull-request for you, using hub.

Once the PR successfully passes the tests and/or is approved by other Homebrew developers, you can finalize the merge with:

brew pull --clean <PR-NUMBER>
git push origin master

The merge is now complete. Don't forget to update your GitHub fork by running git push your-fork master

Building bottles for updated formulae

After merging changes, we must rebuild bottles for all the PRs that had conflicts.

To do this, tap Linuxbrew/homebrew-developer and run the following command where the merge commit is HEAD:

for formula in $(brew find-formulae-to-bottle); do
  brew build-bottle-pr --remote=$GITHUB_USER $formula
done

The find-formulae-to-bottle command outputs a list of formulae parsed from the merge commit body. It also performs some checks against the formulae:

And it skips formulae if any of the following are true:

  • it doesn't need a bottle
  • it already has a bottle
  • the formula depends on macOS to build
  • the formula's tap is Homebrew/homebrew-core (the upstream macOS repo)
  • there is already an open PR for the formula's bottle
  • the current branch is not master

If a formula you are expecting to bottle is skipped, there may be an error; by default, this script won't output the errors. To see them, run brew find-formulae-to-bottle --verbose separate to the for loop above.

The build-bottle-pr script creates a branch called bottle-<FORMULA>, adds # Build a bottle for Linuxbrew to the top of the formula, pushes the branch to GitHub at the specified remote (default: origin), and opens a pull request using hub pull-request.

Common build failures and how to handle them

Bottling errors

Handling brew bump-formula-pr PRs

Formulae that exist in Homebrew/homebrew-core

The brew bump-formula-pr command will raise PRs against the Linux formula repo for upstream Mac formulae when running on Linux. This isn't how version bumps are done for most formulae. Until Homebrew/brew issue 6341 is implemented - a feature that will raise PRs against the correct repo for macOS formulae bumps on Linux - we have to redirect users to raise their PRs in Homebrew/homebrew-core:

Thanks for your PR.

However, this formula is not Linux-specific. Its new versions are merged from the Homebrew/homebrew-core repository daily as documented in CONTRIBUTING.md. Please submit this change as a PR to that repository.

We look forward to your PR against Homebrew/homebrew-core for the next version bump!

Linux-only formulae

If the formula is a Linux-only formula, it either:

  • will contain the line # tag "linuxbrew"
  • won't have macOS bottles

These formulae are fine for users to bump with brew bump-formula-pr, but you should request that they remove the existing x86_64_linux bottle SHA line so that CI will build a bottle for the new version correctly. If the bottle SHA isn't removed, CI will fail with the following error:

--keep-old was passed but there are changes in sha256 => x86_64_linux