apt-get update and upgrade in container during build process

[redshiftzero]

Description

We currently only run apt-get update and upgrade when we build the builder image, and then we pull that down from quay when we do a build. We have a test to flag if we need to update the image for security updates. This builder update has been done maybe 40+ times for security updates, and is done manually by developers.

Advantages:

1. Reproducible build environment
2. Faster build

Disadvantages:

1. Dev overhead to do the builder updates. Again, it is done manually by developers.
2. Because of 1, we don't end up updating most of the time except when we do a build of a package to go onto one of our apt servers, meaning that in practice we're running containers potentially with security vulnerabilities on developer machines.

Also bear in mind we are also apt-get update and upgrading in prod right now (when we run securedrop-admin install). It seems like expending this effort is not a wise allocation of time given the current team resources, but open to discuss. An alternative path is discussing how we can safely automate this task.

[emkll]

Another advantage in controlling the builder image using the current process is that we know for each release what versions of packages were used in the build process.

One way to preserve this property would be to tag an image on release day or whenever we build the prod debs.

[eloquence]

Suggestion from 2020-10-08 tech meeting: Include package version dump as part of build logs.

[conorsch]

After group discussion, we're going to defer immediate action related to the build image pending introduction of reproducible builds in for the packages on SecureDrop servers (that's #308 ). We've got a solid approach to reproducible builds for the Workstation over in https://github.com/freedomofpress/securedrop-debian-packaging , but we're not using that same logic here yet. We should, but right now higher priority remains the ongoing Focal migration (#4768) and the upcoming Workstation audit. See the roadmap for more detail https://github.com/freedomofpress/securedrop/wiki/Development-Roadmap

In practice, maintainers are updating the build image as part of the release procedures (e.g. #5563, from yesterday), which means it's not a constant irritant to update frequently. We'll hold to that procedure going forward, and revisit when we've got reproducible builds. Since no changes are planned until that happens, I'm closing this issue.

[eloquence]

Adding this to discussion notes for the release retro for 1.7.0/1.7.1, as the builder image dependency check still does seem to cause a fair amount of friction, and it may be worth working on this issue as a near-term quality-of-life improvement for the release process.

[conorsch]

Cross-linking relevant discussion: #4764