On recommending Docker

[GuilloteauQ]

Hello,

In its current state, I don't really think there enough "warnings" about the use of Docker(files) for authors...
Based on recent studies¹², it really seems like docker(files) is not suitable for long-term reproducibility.
One point missing for the packaging page³ would be to mention tools such as Nix⁴ and Guix⁵ which are made for (long-term) reproducibility, and which can produce reproducible containers (for example⁶).

If you are interested, I can make a PR with some text about Nix/Guix and explaining the limitations of Docker(files).

Footnotes

1. https://arxiv.org/pdf/2601.12811 ↩

2. https://dl.acm.org/doi/pdf/10.1145/3736731.3746146 ↩

3. https://sysartifacts.github.io/packaging-guide.html ↩

4. https://nixos.org/ ↩

5. https://guix.gnu.org/ ↩

6. https://nix.dev/tutorials/nixos/building-and-running-docker-images.html ↩

[SolalPirelli]

Happy to review a PR to the packaging guide.

That being said, there's a tradeoff between full reproducibility and usability, at the end of the day the goal is mostly that someone can reproduce an artifact with some effort, not that it will definitely work in 100 years (will there still be a compiler that can compile Nix at that point? who knows).

Framing it as "here are some important considerations for your Dockerfiles, and you could consider using Nix" is good IMHO, but telling people to not use Docker due to reproducibility issues would probably be counterproductive.

[GuilloteauQ]

I'll draft something then :)

That being said, there's a tradeoff between full reproducibility and usability, at the end of the day the goal is mostly that someone can reproduce an artifact with some effort, not that it will definitely work in 100 years (will there still be a compiler that can compile Nix at that point? who knows).

Note that i am not talking about 100 years, but merely months before a dockerfile creates a different environment and threatens reproducibility...
But I guess this is still "the" question: who are we doing artifact evaluation for ? for the authors or for the future researchers wanting to build their research upon the artifact ?

Framing it as "here are some important considerations for your Dockerfiles, and you could consider using Nix" is good IMHO

That's what I had in mind yes :)

[SolalPirelli]

But I guess this is still "the" question: who are we doing artifact evaluation for ? for the authors or for the future researchers wanting to build their research upon the artifact ?

IMHO, both:

1. for the authors to make sure their results make sense and they can convince others of this (e.g., the outcome of the study doesn't depend on a mysterious hardcoded variable)
2. for future researchers to reproduce and build upon the artifact

(1) is very short term, (2) is medium term. Long-term is still a big unknown -- good luck connecting to the modern Internet with a 20-year-old VM or container or whatever perfectly reproducible image you have, for instance.

I think it's reasonable to assume that major images aren't going to disappear for a while, for instance, and that even if they do it the effort to port a Dockerfile based on [linux distro from 10 years ago] is still much less than trying to reproduce the entire paper from scratch which is what you'd have to do without an artifact. And it's possible to upload an entire Docker image to Zenodo or similar.

On the other hand, yes, we should discourage people from building their artifact on top of some random image that might disappear in a year and whose contents aren't clear. It falls under the "precise list of dependencies" point from the checklist, though I'm happy to take feedback on better wording or more specific points (there's still time for the SOSP'26 AE ;) )