Conversation
|
I like it except we still don't know how many tests to expect to be ran here. If the files are missing as from a crate then, if I read this correctly, silently no tests are generated or ran. Correct? |
|
Yes, though in the specific case of running from a The other case that isn't handled is files that are present under tests/images but not under tests/reference. There's currently a couple dozen such images that are used by other tests though, so we can't just treat them as failing tests. I suppose that listing them as ignored would be an option? |
197g
left a comment
There was a problem hiding this comment.
Alright let's worry about packaging at a later time.
This PR uses the
libtest-mimicandwalkdircrates to simplify/improve our reference tests. It recursively walks the entire directory tree under the reference images directory to find tests, and tests that cannot be run because of a disabled feature are now marked as "ignored" rather than silently not run.I removed the "render_images" test because I didn't see what purpose it was serving that we didn't get by just having the normal tests write to
tests/output/with the observed contents of any image that didn't match the reference.Also corrects several reference images because due to a bug in the old implementation we were apparently not testing the images under
tests/reference/jpg...Old output
New output