Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Building Matplotlib on Ubuntu #1874

Merged
merged 2 commits into from

4 participants

@pwuertz
Collaborator

Hi,
I experienced a problem when compiling matplotlib on Ubuntu 13.04. The setup determined that the freetype2 headers are not installed (although they are). The reason for this is a function in setupext.py script that searches for "ft2build.h" in the freetype2 include directories ("/usr/include/freetype2" and below), whereas the file resides in "/usr/include/ft2build.h". As a quick workaround, I linked /usr/include/ft2build.h to /usr/include/freetype2/ft2build.h and matplotlib compiled just fine.

I don't know it this has been caused by a change in the ubuntu package or by a modification in setupext.py though..

@pwuertz
Collaborator

Although #1632 reported a problem with the freetype headers as well, I don't think these problems are related to this.

@dmcdougall
Collaborator

The real question is why pkg-config isn't finding them. What does the output of pkg-config freetype2 --cflags say?

@pwuertz
Collaborator

Nono, pkg-config IS finding freetype2, but matplotlib restricts its search for ft2build.h only to the directories reported by pkg-config, whereas ft2build.h is in the standard /usr/include folder.

@dmcdougall
Collaborator

Hmm, my setup is exactly the same and it works fine. Both on OS X and on Ubuntu.

$ pkg-config freetype2 --cflags
-I/opt/local/include/freetype2 -I/opt/local/include

All my freetype stuff is in /opt/local/include/freetype2 except ft2build.h, which is in /opt/local/include. My point is that pkg-config should still be picking up the ft2build.h file in the parent directory, so that matplotlib restricting to the paths returned by pkg-config is exactly what it should be doing.

@pwuertz
Collaborator

Your setup seems to be far from being exactly the same. It appears that you manually installed freetype2, which ended up in /opt/local/include. Your pkg-config reflects that. The default package managed setup is to have the headers installed in /usr/include. The headers installed in the subdirectory /usr/include/freetype2 require an include directive as reported by pkg-config

$ pkg-config freetype2 --cflags
-I/usr/include/freetype2

The ft2build.h is in /usr/include which doesn't need to be added to the cflags as it is the default header search path. The way setupext.py is trying to verify the freetype2 setup by looking at the additional include directories only doesn't work in this case.

One could either keep the system as it is and look for another freetype header that is located in /usr/include/freetype2, or try to compile a small code snippet that includes freetype (like the autoconf tools do).

@pwuertz
Collaborator

Ah, the reason why this is showing up now is because I switched from matplotlib 1.2 to master, to which @mdboom applied some major restructuring in 6aa7b29.

@mdboom
Owner

@pwuertz: What's the exact error message you're getting?

What the new setup code (in master) does is look in both the standard directories (as reported by Python) and what pkg-config returns. The old version only looked in pkg-config, and displayed a warning (but continued) if a header file wasn't to be found there. So I'd think the new version to be more robust to this kind of issue. But perhaps on Ubuntu, Python doesn't explicitly provide the standard directories? I agree that the only way to really do this right is to probably do a test compile, but I was trying to avoid that due to how complex distutils makes that (since we'd have to do the test build with the compiler distutils chose and the same environment etc...)

@dmcdougall
Collaborator

@pwuertz Aha, it only adds the extra flag for nondefault locations. I see now. Then yes, this is a problem. I see this behaviour on a separate machine where the package was installed into the default location.

@dmcdougall
Collaborator

@mdboom How would one check the standard directories as reported by Python? For me, the 'obvious' places don't have the default /usr/local listed:

In [2]: print sys.prefix
/opt/apps/ossw/applications/python/python-2.7.3/sl6

In [3]: print sys.path
['', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/bin', '/h1/damon/python/lib/docopt-0.5.0-py2.7.egg', '/h1/damon', '/h1/damon/python/lib', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python27.zip', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/plat-linux2', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/lib-tk', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/lib-old', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/lib-dynload', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/site-packages', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/site-packages/gtk-2.0', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/site-packages/IPython/extensions']
@mdboom
Owner

Sorry -- I was wrong -- it's not actually extracting the directories that distutils sends to the compiler by the default -- we're using a set of hardcoded paths in addition to what pkg-config returns -- but /usr/include is among those, so it's still a mystery.

If you do

In [1]: import setupext

In [2]: setupext.get_base_dirs()
Out[2]: ['/usr/local', '/usr']

In [4]: ext.include_dirs
Out[4]: ['/usr/include', '.']

What do you get?

@dmcdougall
Collaborator
In [1]: import setupext

In [2]: setupext.get_base_dirs()
Out[2]: ['/usr/local', '/usr']

In [3]: ext.include_dirs
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-3-1bc9f4a4fcf1> in <module>()
----> 1 ext.include_dirs

NameError: name 'ext' is not defined

In [4]: setupext.include_dirs
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-fe5fcdb253e1> in <module>()
----> 1 setupext.include_dirs

AttributeError: 'module' object has no attribute 'include_dirs'
@mdboom
Owner

Sorry... copy-and-paste error. I meant to say:

In [1]: import setupext

In [3]: ext = setupext.make_extension("foo", [])                                                                                                                                                       

In [4]: ext.include_dirs
Out[4]: ['/usr/include', '.']
@pwuertz
Collaborator

I think I tracked it down to options['basedirlist'] which is read from setup.cfg. If the config file says

# Uncomment to override the default basedir in setupext.py.
# This can be a single directory or a space-delimited list of directories.
basedirlist = /usr

the options['basedirlist'] is just the string '/usr' instead of ['/usr'] and get_base_dirs() returns that. make_extension() then tries to iterate over the characters of the string and thus doesn't find the /usr/include dir, so no default includes are appended. Uncommenting this line in setup.cfg makes the whole thing work again.

@dmcdougall
Collaborator
In [1]: import setupext

In [2]: ext = setupext.make_extension("foo", [])

In [3]: ext.include_dirs
Out[3]: ['/usr/local/include', '/usr/include', '.']
@mdboom
Owner

Ah -- I see. At the very least, the example given in setup.cfg.template needs to be updated. For extra credit, we should probably make basedirlist convert a single string to a list with one entry (providing a string is just too easy).

@dmcdougall
Collaborator

Huzzah! Thanks @mdboom.

@mdboom mdboom was assigned
@mdboom
Owner

Self assigning. I hope to get a fix in later today.

setupext.py
@@ -77,7 +77,7 @@ def check_output(*popenargs, **kwargs):
pass
try:
- options['basedirlist'] = config.get("directories", "basedirlist")
+ options['basedirlist'] = config.get("directories", "basedirlist").split(',')
@pelson Collaborator
pelson added a note

Do we need to strip it? I'd have put something like /usr, /usr/local (notice the space after the comma)

@mdboom Owner
mdboom added a note

Indeed. Stripping would be a good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@pelson
Collaborator

:+1: LGTM

@mdboom
Owner

@pwuertz: Once you've confirmed this fixes things for you, why don't you take the honors of pushing the big green button...

@pwuertz pwuertz merged commit 11e7ed9 into matplotlib:master
@pwuertz
Collaborator

Confirmed, thanks @mdboom !

@mdboom mdboom deleted the mdboom:basedirlist-bug branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 1, 2013
  1. @mdboom
Commits on Apr 2, 2013
  1. @mdboom
This page is out of date. Refresh to see the latest.
Showing with 4 additions and 2 deletions.
  1. +1 −1  setup.cfg.template
  2. +3 −1 setupext.py
View
2  setup.cfg.template
@@ -5,7 +5,7 @@
[directories]
# Uncomment to override the default basedir in setupext.py.
-# This can be a single directory or a space-delimited list of directories.
+# This can be a single directory or a comma-delimited list of directories.
#basedirlist = /usr
[status]
View
4 setupext.py
@@ -77,7 +77,9 @@ def check_output(*popenargs, **kwargs):
pass
try:
- options['basedirlist'] = config.get("directories", "basedirlist")
+ options['basedirlist'] = [
+ x.strip() for x in
+ config.get("directories", "basedirlist").split(',')]
except:
pass
else:
Something went wrong with that request. Please try again.