Test snapshots

[wezm]

This is an initial pass at adding and verifying test output via snapshots. snap files currently capture expected stdout and stderr for each test. A rudimentary diff is shown when a mismatch occurs. To update snapshots/create missing snap shots run the test with FATHOM_UPDATE_SNAP set (any value will do).

It's likely that there are further extensions/improvements that can be done to this. I figured these can be driven by the experience of using this initial version.

[brendanzab]

Really cool stuff!

I wonder if we could add a hint to .gitattributes to render the *.snap files as TOML?

[wezm]

I wonder if we could add a hint to .gitattributes to render the *.snap files as TOML?

I wondered exactly this too! But my searches didn't yield anything useful. Are you aware if it's actually possible?

Edit: I tried again and found out how to do it.

[brendanzab]

Hmm weirdly Github doesn't seem to be using it in the diffs. Maybe it needs to be committed first, or a caching issue? Anyway, not a massive deal.

[brendanzab]

Looks great!

[wezm]

Hmm weirdly Github doesn't seem to be using it in the diffs. Maybe it needs to be committed first, or a caching issue?

It's done on a low priority background job so might take a bit to update:

https://github.com/github/linguist/blob/master/docs/how-linguist-works.md#how-linguist-works-on-githubcom

[wezm]

@brendanzab any idea why the output would be different on different Rust versions?

The cube.stl test is failing the snapshot because at least the initial line is different ({ / [):

        1| - 0 = {

vs.

        96| + 0 = [

[brendanzab]

Ohhhh this seems like it might be testing the merge commit using #322? Where I now have a list of parsed expressions per position (we might have to revisit this approach to rendering these). So we might need to regenerate the snapshot. Not sure why it is different per rust version though - that is very strange 😵‍💫

[wezm]

Righto finally got a passing build

[brendanzab]

Yay :)