devxp: Replace forem/ruby image with open-source Debian-based analogue. #19632
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This builds a Debian-derived base image that targets Ruby 3.0.2 (our current version lock) to replace `quay.io/forem/ruby`, which is based on Fedora 35 (and as such, lacks ARM64 support). The building of this image is not yet automated, I intend to add that as a follow-up some time vaguely soon-ish. This has been built and pushed to a temporary tag, [ghcr.io/forem/ruby:klardotsh-test](https://github.com/orgs/forem/packages/container/ruby/103882793?tag=klardotsh-test). Once this merges, I'll build and push the result in main (pending any revisions that come up in PR review) with the following incantation, which will generate a multiarch manifest: ```sh docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile.base . -t ghcr.io/forem/ruby:latest -t ghcr.io/forem/ruby:$(git rev-parse --short HEAD) --push ``` This refs, but does not complete, #19626, and is one of several blockers on the path to getting #19603 merged.
|
Uffizzi Preview |
klardotsh
commented
Jun 24, 2023
| echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \ | ||
| apt update && \ | ||
| apt install -y \ | ||
| imagemagick \ |
There was a problem hiding this comment.
IM's inclusion here is because the quay.io/forem/ruby this image is replacing, installed it as part of the base image. I'm open to the idea that this actually belongs in the app-specific containerfile - honestly I suspect I'll be moving various dependencies between the two as time goes on (as I prepare the app-specific containerfile's rebase onto this image, I'm discovering dependencies that probably need moved into the base, too)
klardotsh
added a commit
that referenced
this pull request
Jun 24, 2023
This "rebases" our application images off of the new Ruby base image added in #19632, and fixes numerous problems and quirks with how the images were built along the way. Notably: - Issues where layers attempted to delete files in prior layers have been resolved (this caused build failures on some Docker filesystem drivers, notably overlay2). - Bundler is no longer allowed to deviate from or modify the lockfile (`BUNDLE_FROZEN` is now `true`). - `git(1)` is no longer required to live inside the container and `.git` is no longer required to be copied into the Docker build context, as these were only used to calculate `FOREM_BUILD_SHA`, which is now passed in as a Build Argument to the container build context. - The entire source tree is no longer `chmod` in one giant swing, which ran so long on my system (as just one example) that I gave up after 15-20 minutes and issued it a `SIGTERM`. Instead, `COPY --chown` is used more heavily and ensures the `APP_USER` will have access to the requisite files. This new container image appears to build successfully for `linux/arm64`, which refs (but does not complete) #19626. Currently, such builds aren't automated , and must be built on a developer workstation. For example: ```sh docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile . -t ghcr.io/forem/forem:klardotsh-test --push --build-arg VCS_REF=$(git rev-parse --short HEAD) ``` In the meantime, the existing `linux/amd64`-only BuildKite scripts have been updated to allow this PR to merge as a separate unit, and CI refactors to enable the multiarch builds of `linux/arm64,linux/amd64` can come later when more time is available. This is one of several blockers on the path to getting #19603 merged. The next step in that chronology will be rebasing that work on top of this work, which *should* be, on the containerization side, as straightforward as bumping `Containerfile.base` to reference the new upstream image, rebuilding the base container, and then bumping the reference in `Containerfile`.
|
Un-assigning myself as I don't have enough context how to review this PR. |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## main #19632 +/- ##
==========================================
+ Coverage 89.27% 94.94% +5.66%
==========================================
Files 1208 913 -295
Lines 27456 18409 -9047
Branches 2117 0 -2117
==========================================
- Hits 24511 17478 -7033
+ Misses 2792 931 -1861
+ Partials 153 0 -153
Flags with carried forward coverage won't be shown. Click here to find out more.
☔ View full report in Codecov by Sentry. |
klardotsh
added a commit
that referenced
this pull request
Jun 28, 2023
This "rebases" our application images off of the new Ruby base image added in #19632, and fixes numerous problems and quirks with how the images were built along the way. Notably: - Issues where layers attempted to delete files in prior layers have been resolved (this caused build failures on some Docker filesystem drivers, notably overlay2). - Bundler is no longer allowed to deviate from or modify the lockfile (`BUNDLE_FROZEN` is now `true`). - `git(1)` is no longer required to live inside the container and `.git` is no longer required to be copied into the Docker build context, as these were only used to calculate `FOREM_BUILD_SHA`, which is now passed in as a Build Argument to the container build context. - The entire source tree is no longer `chmod` in one giant swing, which ran so long on my system (as just one example) that I gave up after 15-20 minutes and issued it a `SIGTERM`. Instead, `COPY --chown` is used more heavily and ensures the `APP_USER` will have access to the requisite files. This new container image appears to build successfully for `linux/arm64`, which refs (but does not complete) #19626. Currently, such builds aren't automated , and must be built on a developer workstation. For example: ```sh docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile . -t ghcr.io/forem/forem:klardotsh-test --push --build-arg VCS_REF=$(git rev-parse --short HEAD) ``` In the meantime, the existing `linux/amd64`-only BuildKite scripts have been updated to allow this PR to merge as a separate unit, and CI refactors to enable the multiarch builds of `linux/arm64,linux/amd64` can come later when more time is available. This is one of several blockers on the path to getting #19603 merged. The next step in that chronology will be rebasing that work on top of this work, which *should* be, on the containerization side, as straightforward as bumping `Containerfile.base` to reference the new upstream image, rebuilding the base container, and then bumping the reference in `Containerfile`.
klardotsh
added a commit
that referenced
this pull request
Jun 28, 2023
This "rebases" our application images off of the new Ruby base image added in #19632, and fixes numerous problems and quirks with how the images were built along the way. Notably: - Issues where layers attempted to delete files in prior layers have been resolved (this caused build failures on some Docker filesystem drivers, notably overlay2). - Bundler is no longer allowed to deviate from or modify the lockfile (`BUNDLE_FROZEN` is now `true`). - `git(1)` is no longer required to live inside the container and `.git` is no longer required to be copied into the Docker build context, as these were only used to calculate `FOREM_BUILD_SHA`, which is now passed in as a Build Argument to the container build context. - The entire source tree is no longer `chmod` in one giant swing, which ran so long on my system (as just one example) that I gave up after 15-20 minutes and issued it a `SIGTERM`. Instead, `COPY --chown` is used more heavily and ensures the `APP_USER` will have access to the requisite files. This new container image appears to build successfully for `linux/arm64`, which refs (but does not complete) #19626. Currently, such builds aren't automated , and must be built on a developer workstation. For example: ```sh docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile . -t ghcr.io/forem/forem:klardotsh-test --push --build-arg VCS_REF=$(git rev-parse --short HEAD) ``` In the meantime, the existing `linux/amd64`-only BuildKite scripts have been updated to allow this PR to merge as a separate unit, and CI refactors to enable the multiarch builds of `linux/arm64,linux/amd64` can come later when more time is available. This is one of several blockers on the path to getting #19603 merged. The next step in that chronology will be rebasing that work on top of this work, which *should* be, on the containerization side, as straightforward as bumping `Containerfile.base` to reference the new upstream image, rebuilding the base container, and then bumping the reference in `Containerfile`.
|
LGTM! |
benhalpern
approved these changes
Jun 28, 2023
klardotsh
added a commit
that referenced
this pull request
Jun 28, 2023
This "rebases" our application images off of the new Ruby base image added in #19632, and fixes numerous problems and quirks with how the images were built along the way. Notably: - Issues where layers attempted to delete files in prior layers have been resolved (this caused build failures on some Docker filesystem drivers, notably overlay2). - Bundler is no longer allowed to deviate from or modify the lockfile (`BUNDLE_FROZEN` is now `true`). - `git(1)` is no longer required to live inside the container and `.git` is no longer required to be copied into the Docker build context, as these were only used to calculate `FOREM_BUILD_SHA`, which is now passed in as a Build Argument to the container build context. - The entire source tree is no longer `chmod` in one giant swing, which ran so long on my system (as just one example) that I gave up after 15-20 minutes and issued it a `SIGTERM`. Instead, `COPY --chown` is used more heavily and ensures the `APP_USER` will have access to the requisite files. This new container image appears to build successfully for `linux/arm64`, which refs (but does not complete) #19626. Currently, such builds aren't automated , and must be built on a developer workstation. For example: ```sh docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile . -t ghcr.io/forem/forem:klardotsh-test --push --build-arg VCS_REF=$(git rev-parse --short HEAD) ``` In the meantime, the existing `linux/amd64`-only BuildKite scripts have been updated to allow this PR to merge as a separate unit, and CI refactors to enable the multiarch builds of `linux/arm64,linux/amd64` can come later when more time is available. This is one of several blockers on the path to getting #19603 merged. The next step in that chronology will be rebasing that work on top of this work, which *should* be, on the containerization side, as straightforward as bumping `Containerfile.base` to reference the new upstream image, rebuilding the base container, and then bumping the reference in `Containerfile`.
dukegreene
pushed a commit
that referenced
this pull request
Jun 28, 2023
…e. (#19632) This builds a Debian-derived base image that targets Ruby 3.0.2 (our current version lock) to replace `quay.io/forem/ruby`, which is based on Fedora 35 (and as such, lacks ARM64 support). The building of this image is not yet automated, I intend to add that as a follow-up some time vaguely soon-ish. This has been built and pushed to a temporary tag, [ghcr.io/forem/ruby:klardotsh-test](https://github.com/orgs/forem/packages/container/ruby/103882793?tag=klardotsh-test). Once this merges, I'll build and push the result in main (pending any revisions that come up in PR review) with the following incantation, which will generate a multiarch manifest: ```sh docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile.base . -t ghcr.io/forem/ruby:latest -t ghcr.io/forem/ruby:$(git rev-parse --short HEAD) --push ``` This refs, but does not complete, #19626, and is one of several blockers on the path to getting #19603 merged.
klardotsh
added a commit
that referenced
this pull request
Jun 29, 2023
…#19633) This "rebases" our application images off of the new Ruby base image added in #19632, and fixes numerous problems and quirks with how the images were built along the way. Notably: - Issues where layers attempted to delete files in prior layers have been resolved (this caused build failures on some Docker filesystem drivers, notably overlay2). - Bundler is no longer allowed to deviate from or modify the lockfile (`BUNDLE_FROZEN` is now `true`). - `git(1)` is no longer required to live inside the container and `.git` is no longer required to be copied into the Docker build context, as these were only used to calculate `FOREM_BUILD_SHA`, which is now passed in as a Build Argument to the container build context. - The entire source tree is no longer `chmod` in one giant swing, which ran so long on my system (as just one example) that I gave up after 15-20 minutes and issued it a `SIGTERM`. Instead, `COPY --chown` is used more heavily and ensures the `APP_USER` will have access to the requisite files. This new container image appears to build successfully for `linux/arm64`, which refs (but does not complete) #19626. Currently, such builds aren't automated , and must be built on a developer workstation. For example: ```sh docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile . -t ghcr.io/forem/forem:klardotsh-test --push --build-arg VCS_REF=$(git rev-parse --short HEAD) ``` In the meantime, the existing `linux/amd64`-only BuildKite scripts have been updated to allow this PR to merge as a separate unit, and CI refactors to enable the multiarch builds of `linux/arm64,linux/amd64` can come later when more time is available. This is one of several blockers on the path to getting #19603 merged. The next step in that chronology will be rebasing that work on top of this work, which *should* be, on the containerization side, as straightforward as bumping `Containerfile.base` to reference the new upstream image, rebuilding the base container, and then bumping the reference in `Containerfile`.
rt4914
pushed a commit
that referenced
this pull request
Jun 29, 2023
…#19633) This "rebases" our application images off of the new Ruby base image added in #19632, and fixes numerous problems and quirks with how the images were built along the way. Notably: - Issues where layers attempted to delete files in prior layers have been resolved (this caused build failures on some Docker filesystem drivers, notably overlay2). - Bundler is no longer allowed to deviate from or modify the lockfile (`BUNDLE_FROZEN` is now `true`). - `git(1)` is no longer required to live inside the container and `.git` is no longer required to be copied into the Docker build context, as these were only used to calculate `FOREM_BUILD_SHA`, which is now passed in as a Build Argument to the container build context. - The entire source tree is no longer `chmod` in one giant swing, which ran so long on my system (as just one example) that I gave up after 15-20 minutes and issued it a `SIGTERM`. Instead, `COPY --chown` is used more heavily and ensures the `APP_USER` will have access to the requisite files. This new container image appears to build successfully for `linux/arm64`, which refs (but does not complete) #19626. Currently, such builds aren't automated , and must be built on a developer workstation. For example: ```sh docker buildx build --platform linux/amd64,linux/arm64 -f Containerfile . -t ghcr.io/forem/forem:klardotsh-test --push --build-arg VCS_REF=$(git rev-parse --short HEAD) ``` In the meantime, the existing `linux/amd64`-only BuildKite scripts have been updated to allow this PR to merge as a separate unit, and CI refactors to enable the multiarch builds of `linux/arm64,linux/amd64` can come later when more time is available. This is one of several blockers on the path to getting #19603 merged. The next step in that chronology will be rebasing that work on top of this work, which *should* be, on the containerization side, as straightforward as bumping `Containerfile.base` to reference the new upstream image, rebuilding the base container, and then bumping the reference in `Containerfile`.
jaw6
added a commit
that referenced
this pull request
Jun 30, 2023
…o-comment-notifications * origin/main: Standalone "admin" page for each comment (#19644) devxp: Base the application container images off new Ruby base image. (#19633) devxp: Base the application container images off new Ruby base image. (#19633) Add index to billboard placement area (#19648) Follow-ups for suggested organizations follows during onboarding (#19619) Fix bug: Unable to unfollow topics (#19637) Remove unnecessary fragment cache (#19653) [ruby] Update carrierwave 2.2.3 → 2.2.4 (patch) (#19614) devxp: Replace forem/ruby image with open-source Debian-based analogue. (#19632) Remove API dependency on fetching tweets and shift to frontend (#19641)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This builds a Debian-derived base image that targets Ruby 3.0.2 (our current version lock) to replace
quay.io/forem/ruby, which is based on Fedora 35 (and as such, lacks ARM64 support).The building of this image is not yet automated, I intend to add that as a follow-up some time vaguely soon-ish. This has been built and pushed to a temporary tag,
ghcr.io/forem/ruby:klardotsh-test. Once this merges, I'll build and push the result in main (pending any revisions that come up in PR review) with the following incantation, which will generate a multiarch manifest:
This refs, but does not complete, #19626, and is one of several blockers on the path to getting #19603 merged.