Reset cache when workflow file changes

[benfrankel]

While working on bevy-template, we noticed the CI was constantly recompiling. It turns out that when we modified the ci.yaml file with different cargo commands, the cache became outdated and every "cache hit" was a false positive. But the cache never updated because we never changed Cargo.toml or Cargo.lock.

[benfrankel]

I'm not 100% on whether the workflow file hash changing should restore from fallback or fully reset the cache. Set to fully reset for now.

[benfrankel]

May want to consider allowing cache-group to override the workflow file hash as well, so that cache-group can still work between different workflow files -- but that also means different versions of the same workflow file.

[benfrankel]

May want to consider allowing cache-group to override the workflow file hash as well

Did this.

Also fixed matrix jobs all being in the same cache-group by adding strategy.job-index (the index of the matrix job) to the default cache-group.

[alice-i-cecile]

I'm going to largely defer to Tim here: he knows this tool better than I do :)

[TimJentzsch]

@benfrankel How about this:

• We add strategy.job-index to the cache-group default as you suggested, but move the workflow file hash out of it
• We place the workflow file hash at the end of the key (not sure if before the Cargo.toml hashes or after... probably after?) and add an additional fallback such that if the workflow file didn't change that specific command we keep the old cache (but overwrite after the command finished)

I guess the disadvantage of this approach would be that cache-group cannot be effectively shared between different workflow files, but that should be fine.

[benfrankel]

Sounds good to me. For best of both worlds, there could be a separate input to overwrite the workflow file hash. Not sure what to call it etc. though, so I'll defer that for now.

[benfrankel]

I moved cache-group to fallbacks because it was a smaller change from the previous iteration. Let me know if I should split out the workflow file hash etc. as described before merge.

[TimJentzsch]

Ok one last thought from my side:

What would you think about hashing the entire workflow folder and adding it to the end of the key, with a fallback.

Modifying the workflows is realtively rare so it should be fine to hash the whole thing. The fallback ensures that we still have a cash if the change is unrelated.
And hashing the whole workflow folder allows caches to be reused across multiple workflows.

What do you think about that?
If you dont like it thats fine as well, then I will merge the current iteration

[benfrankel]

That's an interesting option. One drawback I can think of: Same job name + matrix index but in a different workflow file => same cache key and forced to share a cache, which causes recompiles until you rename one of the jobs.

Same job name in a different workflow file may not be common, but when it happens it'd be confusing.

Also I had the thought that putting cache-group on the other end of the fallback order would be better. Like you said, workflow files don't change very often. So, if Cargo.toml or Cargo.lock change, which is more common, we should prefer to fall back to different Cargo.toml but same workflow file / job. However, if the workflow file changes, there's no need to be picky about the exact Cargo.toml of the fallback.

[TimJentzsch]

In order to keep things moving, lets merge this iteration for now and refine it down the road as we test it in practice.

I cant do a release today, but will try tomorrow.
Ill do a v2 version bump, this probably qualifies as breaking change.