WIP New approach for up-to-date images

[mikaelarguedas]

Add a cron job to update Dockerfiles and submit upstream PR if new syncs require rebuild:

• Parse images.yaml.em and retrieve the sha256 of the package to install
• Pull Packages from packages.ros.org and check if sha matches
• if sha doesnt match: update dockerfile with the new sha for cache break
• submit PR upstream for rebuild

[mathias-luedtke]

Why do you need to check the SHA of the package?
Wouldn't it be easier to pin the full version?
E.g. ros-melodic-ros-core=1.4.1-0bionic.20200821.054643 instead of ros-melodic-ros-core=1.4.1-0*?

It is simple to check for updates and it would invalidate the cache..
And it is human-readable ;)

[mikaelarguedas]

It would be nicer indeed. The issue is that that version number includes the build timestamp that is specific to each deb so you would need a different version number for each architecture (amd64, armhf, etc).

There is more context / history on #112 that explored a number of approaches for keeping the images up-to-date and we ended up on the approach described here as this was supposed to be the one creating the least number of invalid cache breaks without requiring us to have multiple dockerfiles.

Although this issue description if definitely not complete enough as I dont remember how this was meant to handle the different sha for different debs of the same OS but using different architectures :|

Edit: I guess the assumption was that as syncs happen for all architectures at once, we could use the sha of say amd64 and if that one changed that means that it changed for all platform as the packages was rebuilt for all arches

[mathias-luedtke]

is specific to each deb so you would need a different version number for each architecture (amd64, armhf, etc).

Ah, I see :-/

guess the assumption was that as syncs happen for all architectures at once, we could use the sha of say amd64 and if that one changed that means that it changed for all platform as the packages was rebuilt for all arches

Or the build version of amd64..
Or a label with latest snapshot tag, e.g. 2020-08-27 for foxy.

[mikaelarguedas]

Or the build version of amd64..
Or a label with latest snapshot tag, e.g. 2020-08-27 for foxy.

This is very human readable but could be misleading as snapshots have not been very consistent in the past and it could give the impression that the packages are coming from snapshots.ros.org which they will not be otherwise users would not be able to update to more recent versions without changing their sourcelist.

One idea with the sha was that it was not explicitly tied to a specific deb (or source of debs) but more a black blob and would avoid confusing users and avoid questions like:

• (if I'm on armhf) why do I have a version number that doesn't exist on my system (because timestamp doesnt match)?
• why does it point to snapshots while my debs come from package.ros.org ?

But to be fair I'm not very attached to the sha and am very happy someone is looking into it as it's been on my back burner forever 🙇‍♂️ . So if you have a clear preference for one, I'm fine with it (I'll let @ruffsl give a 2nd approval though)

[mathias-luedtke]

This is very human readable but could be misleading as snapshots have not been very consistent in the past and it could give the impression that the packages are coming from snapshots.ros.org

So we could name the label "sync date"? Instead of retrieving it from the snapshot server, we can ask Jenkins: http://build.ros.org/job/Mrel_sync-packages-to-main/
http://build.ros.org/job/Mrel_sync-packages-to-main/lastSuccessfulBuild/api/json?tree=timestamp

[mikaelarguedas]

That sounds fine too.
Taking that epoch timestamp and displaying it as "%Y-%m-%d" would be both clear and accurate 👍

[mikaelarguedas]

Remembering of corner cases

the sha was also supposed to cover cases where:

• docker images were getting debs from multiple apt repositories, for example the ros1-bridge images getting both ROS 1 and ROS 2 packages. But maybe its not a big deal because one would assume if the ROS1 packages used by the ros1-bridge image didnt change MAJOR.MINOR.PATCH then they should still work event if they were rebuilt (is it sure though?)
• the fact that packages were coming from different build farms (in the proposed solution you need to know to go get the sync date from build.ros.org if ROS 1, build.ros2.org if ROS 2)
• distros without syncs (e.g. Gazebo and Ignition). Although these projects always bump versions when things change so maybe it's not that big a problem but it would stil be nice to keep these image up-to-date along the others

[mathias-luedtke]

Good points!

The sync date could be just informational, and will updated after a SHA change was detected.
(Only for ROS1/2)

I still have to dive into the (template) code to see how this could be done in a cron job.

[mathias-luedtke]

I have been playing around a bit.
The Dockerfiles could be written like this:

[...]
ENV ROS_DISTRO foxy

LABEL packages.ros-foxy-ros-core=0.9.2-1focal.20200826.181817 

# install ros2 packages
RUN apt-get update && apt-get install -y --no-install-recommends \
    ros-foxy-ros-core=0.9.2-1* \
    && rm -rf /var/lib/apt/lists/*

[...]

One label would be added per installed package.

[mathias-luedtke]

Or

LABEL sha256.ros-foxy-ros-core=d5be9ef976036e621c7bcd8dd626af39da465c906a30585247537d3c987d557b

Might be better for Gazebo, because package version do not have the build time appended.

[ruffsl]

LABEL sha256.ros-foxy-ros-core=d5be9ef976036e621c7bcd8dd626af39da465c906a30585247537d3c987d557b

That's not too bad. A label is still a Dockerfile directive that can be used to break the build cache, but innocuous enough to include in the image env downstream. It'd only be indicative of changes from the amd64 architecture, but I think that's good enough.

I still have to dive into the (template) code to see how this could be done in a cron job.

If you add this sha label to the template code, then our current CI will automatically open a PR here with changes to the Dockerfile after a sync is released. We don't have any GH actions for opening a merged PR here to upstream, but that could be worked on separately.

The template code of interest lives here, and I think you'd just need to modify this function here to include the sha inferred from the package_index that was scraped from the apt repo.

https://github.com/osrf/docker_templates/blob/9b2550acc2cf2c1d064ded5fc30f4ac814dc0fc1/docker_templates/packages.py#L128

Then just extend the empy template to include the label from the value in the data dictionary, e.g. for ros2

https://github.com/osrf/docker_templates/blob/9b2550acc2cf2c1d064ded5fc30f4ac814dc0fc1/docker_templates/templates/docker_images_ros2/create_ros_core_image.Dockerfile.em#L69

For an example on generating dockerfiles with the templates, see the docs and Dockerfile for CI

https://github.com/osrf/docker_images/blob/master/doc/generating_dockerfiles.md
https://github.com/osrf/docker_images/tree/master/docker

[mikaelarguedas]

Addressed by osrf/docker_templates#90, example or resulting automatic PR: #459