dockerTools.streamLayeredImage: expose conf / streamScript in passthru

[lbpdt]

Motivation for this change

This is useful to create wrappers around streamLayeredImage and tweak
conf after it has been generated.

Specifically, we want to update the image tag (repo_tag in conf)
with a unique suffix based on the short hash of the
streamLayeredImage derivation.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[lbpdt]

@GrahamcOfBorg test docker-tools

[roberth]

I don't think we should expose those implementation details. The format of this "conf" file is likely to change, for example when new layering solutions become available (#48462) or when other changes are made, for example to support all buildImage features.
Perhaps you could add support for your use case directly? That way we can also have a test, ensuring that future changes to dockerTools don't break the tagging solution.

[lbpdt]

I can understand how exposing internals is not ideal but I didn't find another way of solving my problem. Maybe you will have an idea though...

Here's my problem: similar to how each derivation maps to a unique hash in the store, I would like to ensure that an image tag uniquely identifies an image build. If that property holds, it implies that images can be immutable on our docker registry, as you would never have a valid reason for overwriting an image tag on the registry. Indeed if the image tag you want to push is already on the registry, you have the guarantee that it is the same as your local one.

To implement this, we expose conf and stream from streamLayeredImage and write a wrapper around it. That wrapper updates conf by suffixing repo_tag with a unique string based on the hash of the streamLayeredImage derivation in the store. That way, each unique invocation of streamLayeredImage creates a unique image tag.

Would you see another way of implementing this without exposing the internals of streamLayeredImage?

[roberth]

We could probably bake your solution into streamLayeredImage with a flag to enable it.

[lbpdt]

I wasn't sure how reusable it could be, but happy to send a PR with the feature if you think it is.

[roberth]

Actually this sounds a lot like the default imageTag behavior. Is that not sufficient?

nix-repl> i = dockerTools.streamLayeredImage { name = "test"; config.Cmd = hello; }             

nix-repl> i.imageTag                                                                
"cp3qpvzqfz05rx3kvj04lxpdkpaxzr0p"

nix-repl> i = dockerTools.streamLayeredImage { name = "test"; config.Cmd = [hello "--help"]; }

nix-repl> i.imageTag                                                                           
"wpvyhqr2gs3jq3rxv7yfpqdi37kq12d6"

It's currently based only on the conf derivation output hash, so indeed it does not capture changes in the streaming script. We could fix that by moving the conf build (the custom json) into the result (stream script wrapper) derivation. It may also be useful to prefix a human-understandable string.

These changes can be made without increasing the interface surface area too much, they're testable and seem quite useful. Do you need something else?

[stale]

I marked this as stale due to inactivity. → More info

[roberth]

No answer in months. Will reopen if answered.