[subset] No need to keep actual expected output in tree; just the hash

[behdad]

So. We keep the expected output, then compare hashes. That's slower than just comparing the files directly.

That said, I suggest we remove expected binaries and make the .tests files just list the expected hash. If hash mismatch happens, we suggest regenerating and while doing that we compare TTX to fonttools to approve of the generated output.

That, or remove the hashing.

[khaledhosny]

We can remove the hashing and just compare the file content, but I’m worried about removing expectation from the repo as they help in debugging CI failures that are not reproducible locally that happen from time to time by being able to see the ttx diff.

[behdad]

We can remove the hashing and just compare the file content, but I’m worried about removing expectation from the repo as they help in debugging CI failures that are not reproducible locally that happen from time to time by being able to see the ttx diff.

Sure. We can do that for now. When things are much more stable we can reconsider.

[khaledhosny]

I tried comparing file contents instead of hashes, and I’m not seeing any measurable difference, so I’m inclined to leave the code as it until we later store hashes instead of the actual files.

[behdad]

SGTM.

[behdad]

On second thought, it's just wrong to check hashes when we have both actual and expected readily. :)