Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Varying results depending on freetype version #8796

Closed
TimoRoth opened this issue Jun 24, 2017 · 6 comments

Comments

Projects
None yet
5 participants
@TimoRoth
Copy link

commented Jun 24, 2017

Bug report

Bug summary

This is probably not really a bug, but more a huge annoyance caused by the freetype version used by matplotlib.

For a visualization of the issue, you can take a look at this commit:
OGGM/oggm-sample-data@a9888f5

freetype 2.7.0 changed the way fonts are rendered, which is most likely the root cause of the issue.
That version is out for quite a while now, but Ubuntu and Debian never updated from 2.6.3 for some reason, so it's only slowly creeping up on us now.

The difference is big enough to fail our ImageComparison tests. To make matters worse, even if we disable rendering text for our tests, the freetype version still affects the overall layout of the graph enough to fail a lot of our tests.

There seems to be no good way to detect the used freetype version, specially as matplotlib from pip can come with its own bundled version that's independent from the system one.
I'm lost how to tackle this issue. Just accepting that our tests fail depending on the used freetype version seems unacceptable.

Matplotlib version

  • Operating System: Linux
  • Matplotlib Version: 2.0.2
  • Python Version: 3.6.1
  • Other Libraries: freetype 2.6.3 vs. 2.8.0

Installed via pip install --no-binary :all:

@anntzer

This comment has been minimized.

Copy link
Contributor

commented Jun 24, 2017

The "remove_text" kwarg to the image_comparison decorator may be helpful for you.
You can get the freetype version from matplotlib.ft2font.__freetype_version__.

@tacaswell

This comment has been minimized.

Copy link
Member

commented Jun 25, 2017

The way we handle this is there is a 'blessed' version we use on travis (and can optionally build as part of the Matplotlib install process) which lets us use 0-tolerance tests. You can put tolerances on comparing images, but picking a global threshold is basically impossible (we used to have a non-zero threshold and the tests kept passing despite completely wrong text coming out in some cases, a 1px shift generates about the same number of 'wrong' pixel as swapping out a glyph).

You might see if switching to svg, pdf, or ps for your test images helps? Store the vector version under version control and rasterize the baseline images as-needed for the tests.

@TimoRoth

This comment has been minimized.

Copy link
Author

commented Jun 25, 2017

The "remove_text" kwarg to the image_comparison decorator may be helpful for you.

I'm already using that, and it has no effect on the issue. Even though the text is not rendered anymore, the minor layout changes still persist, and break the tests.

You can get the freetype version from matplotlib.ft2font.freetype_version.

Nice, that's what I was looking for. So we can at least automatically detect which set of reference images to use.

You might see if switching to svg, pdf, or ps for your test images helps? Store the vector version under version control and rasterize the baseline images as-needed for the tests.

So we can just output some vector format, and compare that? That sounds like the best solution, but I'm not sure of pytest-mpl supports that. It looks pretty hard-coded for png images.

@jkseppan

This comment has been minimized.

Copy link
Member

commented Aug 2, 2017

So we can just output some vector format, and compare that? That sounds like the best solution,
but I'm not sure of pytest-mpl supports that. It looks pretty hard-coded for png images.

I'm not sure if pytest-mpl does this, but the matplotlib tests convert pdf and svg files to png at comparison time. The pdf conversion is just a call to Ghostscript, while the svg conversion does something interactive with Inkscape:

def _update_converter():

@moto-timo

This comment has been minimized.

Copy link

commented Oct 16, 2017

Is it possible for the 'blessed' version be updated to freetype 2.8.0?

@anntzer

This comment has been minimized.

Copy link
Contributor

commented Oct 16, 2017

We would prefer not, because this is an ever moving target (freetype rendering changes even between patchlevel changes) and regenerating all the images further would increase the size of the already fairly heavy repo by ~15Mb (IIRC) every time.
On Linux (and, by setting the DYLD_FORCE_FLAT_NAMESPACE envvar to 1, on OSX too) you can force the use of freetype 2.6.1 by compiling your own libfreetype.so (2.6.1) and calling ctypes.CDLL("/path/to/your/libfreetype.so", ctypes.RTLD_GLOBAL) (essentially telling the dynamic linker "please look for symbols HERE first"). Probably could trick Windows too...

I'm going to close the issue but feel free to suggest better solutions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.