Building Matplotlib on Ubuntu #1874

Merged
merged 2 commits into from Apr 5, 2013

Projects

None yet

4 participants

@pwuertz
Contributor
pwuertz commented Mar 31, 2013

Hi,
I experienced a problem when compiling matplotlib on Ubuntu 13.04. The setup determined that the freetype2 headers are not installed (although they are). The reason for this is a function in setupext.py script that searches for "ft2build.h" in the freetype2 include directories ("/usr/include/freetype2" and below), whereas the file resides in "/usr/include/ft2build.h". As a quick workaround, I linked /usr/include/ft2build.h to /usr/include/freetype2/ft2build.h and matplotlib compiled just fine.

I don't know it this has been caused by a change in the ubuntu package or by a modification in setupext.py though..

@pwuertz
Contributor
pwuertz commented Mar 31, 2013

Although #1632 reported a problem with the freetype headers as well, I don't think these problems are related to this.

@dmcdougall
Member

The real question is why pkg-config isn't finding them. What does the output of pkg-config freetype2 --cflags say?

@pwuertz
Contributor
pwuertz commented Mar 31, 2013

Nono, pkg-config IS finding freetype2, but matplotlib restricts its search for ft2build.h only to the directories reported by pkg-config, whereas ft2build.h is in the standard /usr/include folder.

@dmcdougall
Member

Hmm, my setup is exactly the same and it works fine. Both on OS X and on Ubuntu.

$ pkg-config freetype2 --cflags
-I/opt/local/include/freetype2 -I/opt/local/include

All my freetype stuff is in /opt/local/include/freetype2 except ft2build.h, which is in /opt/local/include. My point is that pkg-config should still be picking up the ft2build.h file in the parent directory, so that matplotlib restricting to the paths returned by pkg-config is exactly what it should be doing.

@pwuertz
Contributor
pwuertz commented Apr 1, 2013

Your setup seems to be far from being exactly the same. It appears that you manually installed freetype2, which ended up in /opt/local/include. Your pkg-config reflects that. The default package managed setup is to have the headers installed in /usr/include. The headers installed in the subdirectory /usr/include/freetype2 require an include directive as reported by pkg-config

$ pkg-config freetype2 --cflags
-I/usr/include/freetype2

The ft2build.h is in /usr/include which doesn't need to be added to the cflags as it is the default header search path. The way setupext.py is trying to verify the freetype2 setup by looking at the additional include directories only doesn't work in this case.

One could either keep the system as it is and look for another freetype header that is located in /usr/include/freetype2, or try to compile a small code snippet that includes freetype (like the autoconf tools do).

@pwuertz
Contributor
pwuertz commented Apr 1, 2013

Ah, the reason why this is showing up now is because I switched from matplotlib 1.2 to master, to which @mdboom applied some major restructuring in 6aa7b29.

@mdboom
Member
mdboom commented Apr 1, 2013

@pwuertz: What's the exact error message you're getting?

What the new setup code (in master) does is look in both the standard directories (as reported by Python) and what pkg-config returns. The old version only looked in pkg-config, and displayed a warning (but continued) if a header file wasn't to be found there. So I'd think the new version to be more robust to this kind of issue. But perhaps on Ubuntu, Python doesn't explicitly provide the standard directories? I agree that the only way to really do this right is to probably do a test compile, but I was trying to avoid that due to how complex distutils makes that (since we'd have to do the test build with the compiler distutils chose and the same environment etc...)

@dmcdougall
Member

@pwuertz Aha, it only adds the extra flag for nondefault locations. I see now. Then yes, this is a problem. I see this behaviour on a separate machine where the package was installed into the default location.

@dmcdougall
Member

@mdboom How would one check the standard directories as reported by Python? For me, the 'obvious' places don't have the default /usr/local listed:

In [2]: print sys.prefix
/opt/apps/ossw/applications/python/python-2.7.3/sl6

In [3]: print sys.path
['', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/bin', '/h1/damon/python/lib/docopt-0.5.0-py2.7.egg', '/h1/damon', '/h1/damon/python/lib', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python27.zip', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/plat-linux2', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/lib-tk', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/lib-old', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/lib-dynload', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/site-packages', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/site-packages/gtk-2.0', '/opt/apps/ossw/applications/python/python-2.7.3/sl6/lib/python2.7/site-packages/IPython/extensions']
@mdboom
Member
mdboom commented Apr 1, 2013

Sorry -- I was wrong -- it's not actually extracting the directories that distutils sends to the compiler by the default -- we're using a set of hardcoded paths in addition to what pkg-config returns -- but /usr/include is among those, so it's still a mystery.

If you do

In [1]: import setupext

In [2]: setupext.get_base_dirs()
Out[2]: ['/usr/local', '/usr']

In [4]: ext.include_dirs
Out[4]: ['/usr/include', '.']

What do you get?

@dmcdougall
Member
In [1]: import setupext

In [2]: setupext.get_base_dirs()
Out[2]: ['/usr/local', '/usr']

In [3]: ext.include_dirs
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-3-1bc9f4a4fcf1> in <module>()
----> 1 ext.include_dirs

NameError: name 'ext' is not defined

In [4]: setupext.include_dirs
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-fe5fcdb253e1> in <module>()
----> 1 setupext.include_dirs

AttributeError: 'module' object has no attribute 'include_dirs'
@mdboom
Member
mdboom commented Apr 1, 2013

Sorry... copy-and-paste error. I meant to say:

In [1]: import setupext

In [3]: ext = setupext.make_extension("foo", [])                                                                                                                                                       

In [4]: ext.include_dirs
Out[4]: ['/usr/include', '.']
@pwuertz
Contributor
pwuertz commented Apr 1, 2013

I think I tracked it down to options['basedirlist'] which is read from setup.cfg. If the config file says

# Uncomment to override the default basedir in setupext.py.
# This can be a single directory or a space-delimited list of directories.
basedirlist = /usr

the options['basedirlist'] is just the string '/usr' instead of ['/usr'] and get_base_dirs() returns that. make_extension() then tries to iterate over the characters of the string and thus doesn't find the /usr/include dir, so no default includes are appended. Uncommenting this line in setup.cfg makes the whole thing work again.

@dmcdougall
Member
In [1]: import setupext

In [2]: ext = setupext.make_extension("foo", [])

In [3]: ext.include_dirs
Out[3]: ['/usr/local/include', '/usr/include', '.']
@mdboom
Member
mdboom commented Apr 1, 2013

Ah -- I see. At the very least, the example given in setup.cfg.template needs to be updated. For extra credit, we should probably make basedirlist convert a single string to a list with one entry (providing a string is just too easy).

@dmcdougall
Member

Huzzah! Thanks @mdboom.

@mdboom mdboom was assigned Apr 1, 2013
@mdboom
Member
mdboom commented Apr 1, 2013

Self assigning. I hope to get a fix in later today.

@pelson pelson and 1 other commented on an outdated diff Apr 2, 2013
@@ -77,7 +77,7 @@ def check_output(*popenargs, **kwargs):
pass
try:
- options['basedirlist'] = config.get("directories", "basedirlist")
+ options['basedirlist'] = config.get("directories", "basedirlist").split(',')
@pelson
pelson Apr 2, 2013 Member

Do we need to strip it? I'd have put something like /usr, /usr/local (notice the space after the comma)

@mdboom
mdboom Apr 2, 2013 Member

Indeed. Stripping would be a good idea.

@pelson
Member
pelson commented Apr 4, 2013

👍 LGTM

@mdboom
Member
mdboom commented Apr 4, 2013

@pwuertz: Once you've confirmed this fixes things for you, why don't you take the honors of pushing the big green button...

@pwuertz pwuertz merged commit 11e7ed9 into matplotlib:master Apr 5, 2013

1 check passed

default The Travis build passed
Details
@pwuertz
Contributor
pwuertz commented Apr 5, 2013

Confirmed, thanks @mdboom !

@mdboom mdboom deleted the mdboom:basedirlist-bug branch Aug 7, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment