-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Alpine package upgrades #23
Comments
I am working on documentation and am adding a section on security. The way that this issue is resolved will have a big impact on that documentation. I think that it is essential to provide a way to use the latest packages. Currently, there is no way to do so without editing the I wrote that adding Since timing is already a factor, I also propose that we make the I pushed both of these changes to my What do you think? |
It mostly looks good to me @TravisCardwell . But I think we shouldn't update I guess the term
So, I think we only disagree on the third item. I am happy to provide that tag, but given that we also provide tags that are pinned to a specific Alpine version. A major issue with both mine and your approaches is that we do not really maintain our previous version. Someone can depend on, say I am thinking like publishing all below tags:
So, every week we have a CI task rebuilding the image, and publishes the tag 1 as an immutable pointer, and updates the 2. and 3. mutable pointers. Another improvement would be that we keep building the project on last 3 Alpine versions so they keep getting security updates. But this might be too much maintenance burden for our scale. What do you think @TravisCardwell ? |
Thank you very much, @utdemir, for the explanation! The term reproducibility does indeed have various interpretations. The one that I had in mind when I wrote the above is the ability to produce an image with the same versions of packages given a static What you say about minor version upgrades is a very good point. Does Alpine make any guarantees about the change of versions of packages between point releases of Alpine? I read that Alpine stable releases are "point-in-time snapshots of the package archives" that are tested to ensure inter-package compatibility, but I have been unable to find details about the how major and minor versions are managed. I think that it is unlikely that changes in Alpine will break I like your suggested tagging strategy. It sounds good to me! When building images in CI, the Alpine version will be set by passing an Providing updated images for the last three Alpine branches sounds good to me as well. I am still curious about how frequently relevant packages have upgrades available. Perhaps we can try it out and see if it is suitable or not. |
I just pushed a commit to my For example, the following command builds image
The image is saved with the most specific tag (1). Do you know of a way to also tag this image with the mutable pointers (2 and 3) from within the The date is formatted using the UTC time zone. Since I named the When using The name:tag arguments to I am thinking about how to add support for building images for three Alpine versions as well as conditional building of images (only if there are upgrades available). I am pretty confident that I can do this within the |
By defining the |
By the way, I am fine with setting the default |
Quick update: I'm going on a holiday for the next week, so I won't be able to look at this project (probably the next week too as I'll be busy catching up with other stuff). But I am happy with your suggestions above,
I'm unsure :(. Multiple
Doesn't look super pretty, but I don't have any better alternative. Happy to leave it to :).
As long as we also provide pushed images that are tied to a specific Alpine version, I'm happy with any default. Regarding other Earthfile refactors or introducing another shellscript, I trust your judgement :). Earthly was mostly an experiment for me, and it would be okay for me even if we replace it with Dockerfile's and shell scripts. When I'm away, please feel free to do any improvements to the codebase, and feel free to merge them to (but of course I'd be happy to review them). As I said, I'll be away next week, but after that we can cut a new release with the changes if we have them merged by then. |
No problem, mate! Have a great holiday! I started logging available upgrades to packages in our Alpine 3.15 images from the 18th. There are none so far. I will continue to log this, and we can use the results to decide how to do the CI.
Good idea! I will test that. If I have the time, I will see if I can implement everything within the |
This works, but there is one strange artifact. Here are the
For some reason the Docker image IDs are not all the same:
Inspecting, I see that some empty layers are added:
This is unfortunate, but I do not think it causes any problems. I have only been able to test saving to multiple tags locally, but I doubt that there will be any issues when also pushing to Docker Hub. |
I realized that there is another issue with this multi-tagging. If we want to provide images for a given version for GHC using multiple versions of Alpine, then building an image for an older version of Alpine would push the image with the GHC-only tag. This is not desired, as that tag should point to the image using the latest (supported) version of Alpine. I attempted to fix this by adding new flag arguments to the
The idea was to allow use of command-line arguments to specify which tags are created/updated, as follows:
Unfortunately, this does not work! Earthly gives the following error message:
Conditional tagging of multiple images is not supported. If you have any ideas about how this issue could be resolved, please let me know. I will remove the GHC-only tag for now, until we can figure out a solution. |
I thought of a solution while documenting the issue in a comment. Multiple conditional tags are not supported due to the above limitation, but we can have a single conditional tag as long as it comes first! I updated the
The images are now saved as follows:
This works! |
When building an image, the Alpine package index is updated in order to install various dependencies. The versions installed depend on the state of the package index at that time. One therefore does not necessarily get the same packages when building `ghc-musl` images at different times; images are not reproducible in this way. This commit adds a command to upgrade any already-installed packages after the package index is updated. This ensures that the latest packages are used, and that the already-installed packages are consistent with the versions of newly added packages.
Previously, each image was tagged like the following: * `ghc-musl:v24-ghc924` With this commit, each image is now tagged like the following: * `ghc-musl:ghc9.2.4-alpine3.16.1-20220730` * `ghc-musl:ghc9.2.4-alpine3.16.1` * `ghc-musl:ghc9.2.4` This allows users to specify image names according to their needs. An image tagged with the GHC version, Alpine version, and (UTC) build date never changes. An image tagged with just the GHC version and Alpine version may be updated to include minor/security package releases, but the fixed Alpine version means that major upgrades should not break builds. An image tagged with just the GHC version may be updated to include new package releases, both major and minor. The Earthly targets are renamed to match the new tags. For example, previous target `ghc924` is now named `ghc9.2.4`. Note that updating the GHC-only tag should be done with caution, so that it always points to an image using the latest (supported) Alpine version. To facilitate this, the `image` target has a `TAG_GHC` argument that defaults to `0` so that the GHC-only tag is not saved by default. This argument should be set to `1` only when updating the image using the latest (supported) Alpine version. The following command is an example of using this argument when building locally: ``` $ earthly --allow-privileged --build-arg TAG_GHC=1 +ghc9.2.4 ``` The `ALPINE_VERSION` variable is still set to the latest (supported) Alpine version, *not* set to `latest`. Users who would like to use the `latest` image are still able to do so by specifying it in a build argument.
During discussion in #21, I ran lsupg on
utdemir/ghc-musl:v24-ghc922
and was surprised to see that many packages have updates for an image that was built so recently. I assume thatghc-musl
containers are not public-facing, but statically linking old versions of libraries could have security implications for the static executables built usingghc-musl
.The image that I am testing was built within the past two days.
There are a number of packages with updates available. In this case,
ghc-musl
users may be concerned with the old crypto, TLS, and SSL packages.The problem is that
apk update
is run to update the package index, butapk upgrade
is not run to upgrade the packages that are already installed. The parent image (alpine:3.15.0
) is over four months old, so the packages really should be upgraded, IMHO.Using a specific version (
3.15.0
) of the alpine image makes it clear exactly which image was used to build aghc-musl
image. Addingapk upgrade
, however, means that the packages used depends on the timing of the build. Perhaps this negates the point of using a specific version? Would it be worthwhile to usealpine:latest
instead? Note that I recommend runningapk upgrade
even when using thelatest
image.The text was updated successfully, but these errors were encountered: