images should be rebuilt altogether

[chulkilee]

Found that https://hub.docker.com/r/library/erlang/tags/ lists images for old Erlang versions, and they haven't been updated for a while.

This will result in missing security fix on base image!

I don't know how all things work together... but
based on https://github.com/docker-library/official-images/blob/master/library/erlang the manifest file is generated by https://github.com/erlang/docker-erlang-otp/blob/master/generate-stackbrew-library.sh

[getong]

@chulkilee
Only the latest version will be built, and I don’t know what security problem you met.

[chulkilee]

I didn't say I had particular security problems.

However, if you check out "Security Scanning service" of any images (not only erlang) (doc), you will see many images anyway contains vulnerable packages. For example, alpine/ubuntu package hasn't been released, or base image is not updated, or image is not rebuilt after the baes image is updated.

To minimize the issue, "official" images should be rebuilt periodically to pick up those changes.

[c0b]

I see you're asking a infrastructure level periodically rebuilding service, which may better ask to https://github.com/docker-library/official-images/issues ??

Found that https://hub.docker.com/r/library/erlang/tags/ lists images for old Erlang versions, and they haven't been updated for a while.

it's true that old Erlang versions remain there, but 1) we do not have permission to delete any old image ( you may ask to official-images ) ; 2) they do exist for a reason, for example the tag 20.2.4-slim 's last update was 4 months ago ( in March 2018 ), the security vulnerabilities may come from two sources: either its base image, or the Erlang version, if your concern is from the Erlang version, the Erlang upstream gives you update to 20.3 and here provide such tags 20.3*, if your concern is from its base image, and you want to stick on erlang:20.2.4 and can't upgrade to erlang:20.3 the only option you have will be to get copy of the Dockerfile and build images locally

20.2.4-slim                          117 MB                            4 months ago

I don't feel the official-images infrastructure team would like to periodically rebuild all historical versions, just think about all available images' all historical versions, that could be too heavy computation and they would need to invest a lot of money on hardware;
the alternative is you may ask them to have delete all the old tags which are not the current Repo Info but I also doubt they want to do that, because it's totally possible a user might have deployed some app based on this 20.2.4-slim image and did not upgrade at all in this 4 months, until the moment it's proved contains vulnerabilities the user may choose to continue with that approach, and I feel the tag 20.2.4-slim is better being kept for cases need reproducing of some issues,; and if Erlang 20.2.4 is proven with security issue, the upstream may make a 20.2.5 fix and here will update; if the vulnerability is found in its base image, you may do a dummy change ( like comment with # force rebuild 2018-07-28 ) and make a PR to official-images for a force rebuild, will need a lot of effort to track if each base image has newly discovered vulnerabilities and necessary to make a dummy change, and this vulnerability checking work won't be interesting; I feel it's still better to ask official-images repo to have a periodically scanning service and only force rebuild when new vulnerabilities discovered.

To minimize the issue, "official" images should be rebuilt periodically to pick up those changes.

I agree it should , but who is going to do such monitoring and force rebuild work, I do believe it's beyond someone's spare time can do; if you are one from some software security company, you will be welcome to contribute your effort to keep all older historical version images also up to date and free from vulnerabilities, it would need extra computing resources (= money) or software engineer's dedicated time ( also = money ); So either ask Official-images (the Docker Inc ) or your company donate it, or wait this repo has more active contributors ( so far the number is 2 )

from this repo's current status, the best advice I can give is to keep image built time, and keep upgrading, in the examples above, do not stick on 20.2.4 but upgrade to 20.3, and if possibly upgrade to 21, keep updating your base image to its latest stable is always preferred.
I understand the latest major version might have more risks from its new features, but the <previous mjaor>.<latest minor> (the 20.3) should be a good trade-off between features and stability.

[chulkilee]

First of all, thanks a lot for maintaining this. If it sounds like I "demanded" it, then it's my fault. I'm just not familiarwith Docker official image process (I only have several automated images on docker hub, which does all jobs for me).

According to Library definition files section

Tags that are removed from a library file do not get removed from the Docker Hub, so that old versions can continue to be available for use, but are not maintained by upstream or the maintainer of the official image. Tags in the library file are only built through an update to that library file or as a result of its base image being updated (ie, an image FROM debian:jessie would be rebuilt when debian:jessie is built). Only what is in the library file will be rebuilt when a base has updates.

So, official image maintainers doesn't do anything to trigger rebuild. The manifest file just need to keep containing those tags. Good news!

And here are findings:

• It's common to just keep the latest version of each major versions (like 1.7.3 and 1.6.1), not all recent versions of latest branch (e.g. all 1.7.x), in official images
• Docker hub actually keeps all images as past snapshots anyway
• Docker hub tags pages just do not show all images (meh)

And here is the implicit policy from it..

• By showing tags for latest versions in the image page, it implies only latest versions will be updated
• To get updates from base image, users MUST use tags which will be kept: latest will stay forever, 21 and 21.0 will stay more while 21.0.3 will not get updated once 21.0.4 is released.
• To keep the specific version image with base update, users MUST build it or run base package update in their dockerfile.

What's unfortunate is.. this "policy" is not documented anywhere in Docker Hub or doc. I'm not blaing this specific erlang image - don't get me wrong!

I'm going to close this as it doesn't need any further actions (unless we want to keep all images listed in docker image page :) )

However, for future work, it would be interesting to see how many people use version-specific image (therefore already out-dated or soon-to-be outdated images), and how quick people want to (and actualy do) upgrade erlang version as new version is released.