Stamping with image tags not updating when volatile-status.txt changes

[SQUIDwarrior]

I'm trying to use stamping to generate dynamic image tags for oci_push using the stamp_tags function from here. One of the variables I want to use is BUILD_TIMESTAMP which is stored in volatile-status.txt. However after the first attempt to use the stamped target as shown below, the value for BUILD_TIMESTAMP no longer updates. As I've played around with this, I think its it because the _stamped_stamp.json file produced by the jq function already exists, and therefore the action does not execute and it does not get recreated on subsequent builds, even if volatile-status.txt changes. This is problematic because many things change in volatile-status.txt between builds, even when there are no code changes. For example, we add a -dirty postfix to our tag (via a stamp variable) if you build the image from a dirty Git tree. Right now if I build the image from a dirty tree, I get the -dirty postfix. If I commit my code and build from a non-dirty tree, it still says -dirty.

stamp_tags(
    name = "stamped",
    remote_tags = [
        '($stamp.BUILD_USER) + "-" + ($stamp.BUILD_SCM_REVISION_SHORT) + "-" + ($stamp.BUILD_TIMESTAMP) + "-" + ($stamp.BUILD_SCM_STATUS)',
    ],
)

[alexeagle]

That's the expected behavior of the volatile-status.txt keys. They don't trigger a re-build as they're not considered in the action key. You should use STABLE-prefixed keys if you want the latest value.

https://docs.aspect.build/rules/aspect_bazel_lib/docs/stamping/

NOTE keys that start with STABLE_ will cause a re-build when they change. Other keys will NOT cause a re-build, so stale values can appear in your app. Non-stable (volatile) keys should typically be things like timestamps that always vary between builds.

[SQUIDwarrior]

@alexeagle Right, I've read that. But then how do you even use a value like BUILD_TIMESTAMP in your rules if it won't ever be updated after the first time you use it? Just create a bunch of variables that are prefixed with STABLE_ even though they most definitely aren't?

[SQUIDwarrior]

More context, we are trying to migrate away from rules_docker and this doesn't match the behavior we had before.

[alexeagle]

When you change any other input to the action, like a source file, you get a new image and it will be stamped with the current timestamp. The cache hit only happens when all other inputs are identical aside from volatile-status.txt

[alex-torok]

I believe that we traced this down to the fact that rules_docker's container_push doesn't use the stamp files in a build action, while rules_oci does.

rules_docker passes the status txt files in as arguments to the script that it defines to be run under bazel run.

This means that every time you bazel run the container_push target, it will see the most recent {stable,volatile}-status.txt files from that bazel invocation symlinked into the bazel run runfiles.

On the other hand, oci_push expects an action to write a file for the tags, which requires using "real" bazel stamping in an action.

This is what caused the behavior of rules_docker's container_push to be subtly different from rules_oci's oci_push when it came to not always seeing the latest values of volatile workspace status variables. rules_docker isn't doing "real" action stamping, so we always saw changes in volatile status variables in the pushed image from container_push.

[alexeagle]

@alex-torok thanks for that analysis, that's familiar. I think rules_oci is correct, in the sense that a file in bazel-out should be stamped, and the *-status.txt files should be considered bazel-internal and not part of arguments passed to an external program.

Do you have time to send a docs PR to oci_push? I'll mention in our migration guide as well.