Core/formula separation

[xu-cheng]

Replace PR Homebrew/legacy-homebrew#49752

To deliver this change, one should follow below procedures:

• merge this PR
• merge the 142a58f9ef5fe44c8731c00cd9b74d71bc15f449 and 09e2f41dfc7cfa72a133d342562ed0b0b903faa5 in this PR to Homebrew/homebrew. i.e. tap_migrations hack and new update script.

After these changes, brew --version will report 0.9.9 to help us differentiate pre and post migration.

How the migration works?

For usres who have old Homebrew installed, running brew update will
set motion for following events:

• Old brew update logic will update Homebrew to the latest commit, i.e. 7cc83ac0006a7125c558b01130a36a6dbbe6757d.
• When generating update report, tap_migrations.rb will be loaded. A
  hack introduced in the first commit will swap the process to run
  brew update in second time.
• The new brew update logic will set upstream remote to Homebrew/brew and
  perform all the necessary migrations.

What's new file system structure?

• Library/Formula and Library/Aliases will be replaced as symlinks pointed to the new tap to keep backward compatibility.
• Library/tap_migrations.rb will be replaced as a hack explained above. It has been migrated to tap_migrations.json in the Homebrew/core tap.
• Library/formula_renames.rb will be removed. It has been migrated to formula_renames.json in the Homebrew/core tap.
• New tap homebrew/core will be installed as core tap.
• Shell completion scripts will be moved to /etc/bash_completion.d/brew and /share/zsh/site-functions/_brew, so they can be picked up automatically by bash and zsh respected. The main thought is to use this migration as opportunity to avoid git conflict.

Test

I set up https://github.com/xu-cheng/homebrew-test as test repo. See https://travis-ci.org/xu-cheng/homebrew-test/builds/114016549

Remaining Works (read: to be addressed in future PRs)

• update all the document in core.
• set up document for new core tap.
• set up new CI flow.
  • code changes
  • create brew legacy-pull https://gist.github.com/545163571f6c6ae74d6a It can pull homebrew formulae pull request to the new core tap.
  • BrewTestBot forks Homebrew/homerbew-core
  • add Travis CI job to Homebrew/homebrew-core using same .travis.yml from other taps.
  • add new Jenkins job
• update brew doctor to include check for new core tap.
• update install script.
• update bash-completion formula because of shell completion script relocation.

[sjackman]

When I merge Homebrew into Linuxbrew first as a pull request, all the modified and new formula will be built on CI and will either be bottled or marked as Fails.

What CI are you using for this and what's the build timeout, out of interest?

I'm considering using Docker Cloud on Amazon EC2, so the build timeout would be unlimited. I've got twelve months of free service with Amazon for now using the free tier. If it goes well, I'll try to estimate the the number of machine hours and the cost of keeping up with the builds, and hopefully find a sponsor.

See Automated repository testing and CI/CD the Docker Way

[MikeMcQuaid]

Could these changes be pulled back into a Homebrew/homebrew PR?

[xu-cheng]

These changes are to relocate shell completion script which cannot be pulled back to Homebrew as it may create git conflicting.