obsolete snaps not deleted

[sbdchd]

Currently when moving files around the old snapshots with the old filenames aren't deleted.

reproduce

1. Create a snapshot in a file foo.rs
2. Rename that file to bar.rs
3. run the tests again and accept the snapshots
4. now there are two snapshots, the unused snap and the one used for bar.rs

[wgoodall01]

I ran into this issue earlier today---I was considering either:

• A cargo insta prune [--check] subcommand to remove the unused snapshots. If --check is passed, exit with failure if any unused snapshots exist, so we can make sure there aren't any unused snapshots in a CI flow
• Or, some way of marking .snap files as unused as the tests run, and incorporating "this snapshot file isn't used, delete it?" into cargo insta review.
  • To do this, every snapshot could touch the timestamp on its .snap file, and we'd figure out which files were untouched when reviewing changes
• Or some combination of the above

Thoughts?

[sbdchd]

Another example is Jest which deletes the obsolete tests when it's passed an update flag, pretty much cargo insta accept.

Seems Jest determines obsolete snaps by first reading the all the snapshots in the directory and then checking off whether it was actually used when evaluating the tests:

• reading the data: https://github.com/facebook/jest/blob/63ebaa6e3848ddf228b1acc6f870c7eaca34e46c/packages/jest-snapshot/src/utils.ts#L91
• checking off the files: https://github.com/facebook/jest/blob/63ebaa6e3848ddf228b1acc6f870c7eaca34e46c/packages/jest-snapshot/src/State.ts#L97

[max-sixty]

To the extent you're familiar with Jest, do you know how it evaluates whether the full test suite is being run, rather than a subset?

[sbdchd]

sorry I'm not really familiar of jest's inner workings

But I get what you're saying: how does jest avoid deleting snapshots for tests that weren't run

[wgoodall01]

Jest controls the entire test harness---it knows which tests are running and which aren't, because Jest is the thing running the tests.

Is there a way to replicate this in libtest, without messing with the internals of the harness?

[mitsuhiko]

I toyed around with this earlier and I couldn't find a good way to deal with running only a subset of tests unfortunately.

[mitsuhiko]

Going to close this for now. While still not perfect, at least you can hack it together now: #136