Large diffs from font rendering

[wtgee]

Hello, thanks for the module.

I know that you are aware of the issues but I'm trying to understand what I can do about them. In particular I see a large difference between two images that are similar, with some apparent minor difference in the font causing the failure.

I've inspected the generated images and they are the same (physical) size, dpi, etc.

I see some discussion (e.g. astropy/astropy#7150) about maybe just choosing the font appropriately to minimize this. Is that the current recommended solution?

I also gave the perceptual hash (#21) a go and it does generate identical hashes for the two images, which is nice. However I haven't yet had time to look into that more to really understand what it is doing so don't necessarily want to rely on it.

Baseline:

Generated. This is missing the title as remove_text=True is enabled (#68 would be relevant here):

Diff:

This is coming from panoptes/POCS#735

(Note that even getting these images out of travis was a huge pain as travis doesn't allow artifacts to be uploaded on pull-requests...but that's a different issue).

[astrofrog]

@wtgee - the easiest solution which I've settled for in astropy & related packages is to generate the baseline images and run the image tests in docker containers on CircleCI, using e.g.:

https://hub.docker.com/r/astropy/image-tests-py35-mpl202/

Incidentally, CircleCI has much better treatment of artifacts, so that helps too. I then run the tests as normal on Travis as part of the usual test suite, without --mpl, and just add the --mpl flag in the CircleCI builds. You could even consider just re-using the docker containers.

See http://docs.astropy.org/en/stable/development/testguide.html#image-tests-with-pytest-mpl for more details.

I'll open an issue regarding an easier way to get the image results out of Travis (I have a way I've used in the past).

[wtgee]

Great, thanks @astrofrog for the response. I'll check out this solution as well. I was just hoping there was some (as yet undocumented) quick trick to make it happen.

I'll open an issue regarding an easier way to get the image results out of Travis (I have a way I've used in the past).

I ended up just writing a small script that zips up the directory and posts to a hosting service (https://transfer.sh), which worked well enough except for the fact that travis was hiding the output from curl. The whole experience made me wish we already were on CircleCI, which I've used for other projects.

Feel free to close or leave open. Cheers!