Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(packaging/docker): Support multiple architectures #762

Merged
merged 22 commits into from
Aug 1, 2019

Conversation

sh0rez
Copy link
Member

@sh0rez sh0rez commented Jul 16, 2019

✔️ Ready to be merged ✔️


This is take 2 on multiple architectures. It includes the following:

  • clean loki-build-image that is based on golang:1.11.4-alpine
  • Dockerfile for promtail that is capable of being built for multiple architectures, based on alpine
  • Dockerfile for loki that is capable of being built for multiple architectures, based on alpine
  • Makefile adaptations that allow building the Dockerfiles as cross-platform, but only when run in CI
    • set CI=true to trigger the cross-platform run
    • uses img instead of docker under the hood, because of better support for --platform.
  • Adapt CircleCI to build and push the images for loki and promtail for amd64, arm64 and armv7
    • install img and binfmt-support
    • use it

Notes

BuildKit

One might have noticed that Docker has introduced a new architecture for building OCI container images called BuildKit. It comes with a lot of improvements over the traditional docker build including a DAG-Solver (builds only what's required, parallelizes as much as possible), better caching, frontends (not only Dockerfile, dynamic syntax extensions) and especially cross-platform support using --platform. As long as the host is capable of executing the ELF binaries, BuildKit can build every arch

binfmt

A Linux kernel feature that allows the kernel to dynamically recognize what to use to execute an ELF binary. This is especially useful with buildkit and qemu. By running sudo docker run --privileged linuxkit/binfmt:v0.6, qemu-static-ARCH executors are registered with the kernel and suddenly it is possible to execute a armhf/ arm64 / whatever binary as if it was native.

img

Although buildkit is usable as a standalone daemon, it is a really handy library as well. The awesome folks over at https://github.com/genuinetools/img used this to build a daemonless cli that builds Dockerfiles using buildkit without root privileges. Because it is still BuildKit, we get all the awesome features above include --platform.

Although BuildKit is going to be fully integrated into Docker Engine in 19.03, it will still be experimental, so it is easier for the moment to use it standalone.

@sh0rez sh0rez changed the title feat(build): Support multiple architectures WIP feat(build): Support multiple architectures Jul 16, 2019
@sh0rez sh0rez mentioned this pull request Jul 17, 2019
@sh0rez sh0rez force-pushed the multiarch-docker branch 2 times, most recently from a3256f3 to 79ad067 Compare July 29, 2019 14:28
@sh0rez sh0rez self-assigned this Jul 30, 2019
@sh0rez sh0rez added component/packaging type/enhancement Something existing could be improved labels Jul 30, 2019
Cleans the grafana/loki-build-image container image, removes unnecessary parts
and modularizes it. Switches to alpine for performance and compatibility reasons
Adds multiarch support to the docker build process:
- make Dockerfiles multiarch compatible
- make Makefile aware (build using `img` in ci, cross-build)
Re-adds docker-cli to loki-build-image, to support TCP daemons
@sh0rez sh0rez force-pushed the multiarch-docker branch 3 times, most recently from 9ee62d7 to 011e3dc Compare July 30, 2019 10:40
@sh0rez
Copy link
Member Author

sh0rez commented Jul 30, 2019

Okay, this finally comes to an an end here.

  • 🏘️ multiarch support for
    • 📄 loki
    • 🦀 promtail
    • 👀 loki-canary
  • 📦 rework of CircleCI to actually build those

Please take a look @slim-bean @cyriltovena @rfratto, thanks :)

Copy link
Member

@rfratto rfratto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice job! my only big concerns are two things:

  • the regression with systemd headers being removed from the build image due to the switch to alpine
  • shouldn't circle CI have picked up the new config.yml and ran some kind of tests against the PR?

loki-build-image/Dockerfile Outdated Show resolved Hide resolved
loki-build-image/Dockerfile Outdated Show resolved Hide resolved
cmd/docker-driver/Dockerfile Outdated Show resolved Hide resolved
Makefile Show resolved Hide resolved
@rfratto
Copy link
Member

rfratto commented Jul 30, 2019

If the minimum required version of Docker for building locally changed, it'd be nice to see one of the documents (maybe this one) updated to specify how to build w/ Docker and which version one needs.

I wouldn't be surprised if a good amount of people are running macOS so using img directly for building is out of the question

cmd/loki/Dockerfile Outdated Show resolved Hide resolved
cmd/promtail/Dockerfile Outdated Show resolved Hide resolved
cmd/promtail/Dockerfile Outdated Show resolved Hide resolved
@slim-bean
Copy link
Collaborator

Wondering if it's possible to centralize the build-image version since we use it in half a dozen docker files and the makefile now, updating it in that many places is certainly prone to error

@slim-bean
Copy link
Collaborator

Nice work on this @sh0rez there are a lot of details to keep track of here and this is very good, almost there!

@sh0rez sh0rez changed the title WIP feat(build): Support multiple architectures feat(build): Support multiple architectures Jul 31, 2019
@sh0rez
Copy link
Member Author

sh0rez commented Jul 31, 2019

Wondering if it's possible to centralize the build-image version since we use it in half a dozen docker files and the makefile now, updating it in that many places is certainly prone to error

Yeah, done that using build-args and the makefile

@sh0rez sh0rez requested review from slim-bean and rfratto and removed request for slim-bean July 31, 2019 10:39
@sh0rez sh0rez changed the title feat(build): Support multiple architectures chore(packaging/docker): Support multiple architectures Jul 31, 2019
Contrary to what has been done before, this reverts to something close to the
old behavior:

If a git tag is built, the name of the tag is used as the docker tag.
If the master branch is built, it is tagged as master-SHA, master and latest

If a regular branch is built, it is not pushed.
Copy link
Collaborator

@slim-bean slim-bean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we got it, LGTM!

@sh0rez sh0rez merged commit b887602 into grafana:master Aug 1, 2019
@sh0rez sh0rez deleted the multiarch-docker branch August 1, 2019 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/packaging type/enhancement Something existing could be improved
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants