Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use different baselines images depending on freetype version #247

Merged
merged 4 commits into from
Jul 1, 2017

Conversation

TimoRoth
Copy link
Member

Updating the images is not too complicated, on a system with a more recent freetype it's just doing

pip install --no-deps -I --no-binary :all: matplotlib
vs.
pip install --no-deps -I matplotlib

As the pre-built mpl comes with freetype 2.6.1, a self built one uses the system version.

Additionally this contains a small change for the workflow CPU count, now taking into account a potential Slurm CPU allocation.

@fmaussion
Copy link
Member

Thanks for looking into this! I'm still not sure that having different baselines is a good idea. If you do this for the freetype version, why not doing it for the gdal version also? (we should check this, but I still think that the conda gdal output differs from a pip install which uses the system lib -- or are there manylinux wheels for rasterio/fiona now?)

So after reading the mpl thread there is no best way out, but we have a few possibilities:

  • treat the travis builds as the "blessed" version and test with zero tolerance on Travis, with higher tolerance (or even skipping tests) elsewhere. This would imply that we have access to Travis' outputs (https://github.com/matplotlib/pytest-mpl#test-failure-example)
  • have a higher tolerance everywhere (this is what we've done until now)
  • have different baseline versions for all kind of setups, and lower tolerance everywhere

I don't really have a preference, although I think that option 3 is a bit overkill. The image tests are mostly qualitative, and are a way to check if the model more or less produces a reasonable output.

@TimoRoth
Copy link
Member Author

The appveyor checks are running on conda, and work just fine.
Conda/conda-forge are both up to date with gdal by now.
I don't think there is a need for old-gdal baseline images anymore, as even Ubuntu 16.04 comes with gdal 1.11, which is recent enough to produce compatible results.

The problem with this freetype situation is that there is no easy way to work around it. If your system has 2.7 or more recent, there's not much you can do about it. Disabling the graphics-based tests would be an option, but just having compatible baseline images really doesn't seem a lot of work to me.

The needed high tolerance for the freetype change would make the tests entirely useless. Pretty much every pixel of the graph changes, due to it being shifted upwards by a few pixels.

I'd prefer the multi-baseline approach, as implemented by this PR. The baselines don't change very frequently, and if they do, it's really not that much work to generate the two needed versions.

@fmaussion
Copy link
Member

OK, let's go for it then. Feel free to merge!

@TimoRoth TimoRoth merged commit 11d522d into OGGM:master Jul 1, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants