Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve hermeticity build system #70

Closed
mvgijssel opened this issue Apr 6, 2022 · 1 comment
Closed

Improve hermeticity build system #70

mvgijssel opened this issue Apr 6, 2022 · 1 comment

Comments

@mvgijssel
Copy link
Member

mvgijssel commented Apr 6, 2022

Currently the Bazel setup requires a couple of host installed tools:

  • Ruby 3.0.2
  • qemu-system-x86_64
  • sha512 / sha256
  • xcode-based c/cpp compilers

Let's try to make these tools part of the build so we don't have to install them externally.

Ruby

Rules Ruby https://github.com/bazelruby/rules_ruby supports compiling Ruby, but the problem is that this particular Ruby is not cached in any way into the disk_cache / repository_cache. This means for the CI that every time Ruby is compiled from scratch, greatly increasing the CI execution time.

Qemu

Currently the tools/packer/runner.sh.tpl adds host specific folders to the PATH before calling packer to get access to the brew installed qemu binaries:

# TODO: this is a hacky wat to get the qemu binary in here
# this can also be solved by specifying the qemu binary in the packer template!
export PATH="$PATH:/usr/local/bin"

Instead of relying on host installed tools, make everything hermetic in include qemu into the build system!

For example we can download the qemu source code here https://www.qemu.org/download/#source and compile this using Bazel and https://github.com/bazelbuild/rules_foreign_cc.

First attempt in #92, but was unable to get it to compile on macOS using Bazel (compiling without Bazel in macOS works just fine 🤔).

sha512 / sha256

The rules_packer relies on these being installed (using brew install coreutils. Maybe there's a way we can fetch these dependencies directly from Homebrew (or from somewhere else) so we know these tools are always available. For example by implementing a homebrew rule like tools/homebrew/homebrew.build_defs or using https://github.com/tmc/rules_homebrew.

@mvgijssel mvgijssel changed the title Include qemu in build system Improve hermeticity build system Apr 17, 2022
@mvgijssel
Copy link
Member Author

Because everything is run within BuildBuddy it’s hard to have external dependencies so this is solved for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant