Ability to save a build and restore it for faster bisecting

[shinglyu]

Rational:

When working on layout/css, I usually need to compare the reftest result before/after a patch, sometimes even comparing between multiple branches. Needing to rebuild before we run a test is not very efficient. We could workaround this by saving the test result, but you never know which test is required.

Proposal:

Let's build a mach command like this (just a rough idea)

• ./mach cache save copies the target/debug folder and give it a unique identifier, could be the commit hash
• ./mach cache restore <hash> overrides the current target/debug with the specified version previously saved
• ./mach cache restore list the saved versions and let you select which one to restore
• ./mach cache list only list but not restore
• --release flag does the above with the target/release folder
• ./mach cache clean deletes all the saved cache

Use case

• We can ./mach cache save two versions before and after a patch, then we can run test on them and compare the results back and forth
• We can save a known good version for demonstration or dogfooding while still coding on the bleeding edge.

Any feedback is welcome, especially on naming :)

[highfive]

cc @wafflespeanut

[nox]

Was any progress made on that?

[jdm]

No, but this sounds useful.

[tigercosmos]

Looks interesting.

[tigercosmos]

I think we can add a command

# Use current branch to test previous build
./mach test --cache <hash>

[ysimonson]

This would be so useful for my rust projects as well! I wonder if it might make sense to implement this as a custom cargo command, rather than via mach? Would y'all be opposed to that?

[jdm]

A custom cargo command sounds useful, but it might be tricky to allow it to work with Servo due to our peculiarities. I don't think we would spend any time porting our solution to Rust and making it generic, as a result.

[ysimonson]

If it's not too much trouble, could you expound on what those peculiarities are? I'm assuming it'd be useful to know them regardless of how this might be implemented.

[jdm]

We have DMG and MSI nightlies for macOS and Windows, so there are special ways to unpack those so the binaries can actually be executed.