CXX/Python2/cxx_extensions.cxx:1320: Assertion `ob_refcnt == 0' #1532

Merged
merged 1 commit into from Dec 17, 2012

4 participants

@idella

In your recent version matplotlib-1.2.0, you appear to have an issue with the file CXX/Python2/cxx_extensions.cxx. The testsuite is massive and it's currently stopped in its tracks by the above. Comment out the line make_set('mathfont', 'cm', font_tests, ['png']) in matplotlib/tests/test_mathtext.py and the massive testsuite completes fine.

python2.7: CXX/Python2/cxx_extensions.cxx:1320: virtual Py::PythonExtensionBase::~PythonExtensionBase(): Assertion `ob_refcnt == 0' failed.

Building HTML failed.

See here it's called in the building of the docs and it also fails. Why you have an in source version of CXX for pycxx rather than use a system installed version I don't know,. but the one you have has this deficit.

CXX/Python2/cxx_extensions.cxx:1320:

You even have the line number

libpng version 1.2.50
stix-fonts version 1.0.0
ipython version 0.13
Python version 2.7.3, 3.2.3

@idella

Reinstate the manual reference counting as discovered in #1054
mdboom authored 4 months ago 1 parent cbb11da commit ca678a4

This is starting to look like a merry go round.

lssue # /1054 submitter by a fellow gentooer. Now you have in source versions of both pycxx and pyparsing.
the above issue was 'fixed' by a patch to the file src/ft2font.cpp. Well it suggests to me it has more to do with
CXX/Python2/cxx_extensions.cxx given that it's cited by the error msg. I can't say that the fix to #1054 broke 'something else somewhere else', at least not in prior version 1.1.1 which completes the tests and a build of the html docs cleanly

@pelson
Matplotlib Developers member

Thanks @idella. I think this is definitely one for you @mdboom.

@mdboom
Matplotlib Developers member

The reason we need a patched version of PyCXX is because the mainline PyCXX simply doesn't compile on Python 2.7 due to the removal of PyCObject (in favor of PyCapsule). Please ping PyCXX for this -- I have submitted a patch, but it doesn't seem to have been incorporated. I can ping Barry Scott again about this.

I've never understood why the --ob_refcnt lines are in there -- indeed, it intuitively makes sense to not have them, and I'd like to have them removed. I think the reinstating them in #1054 was due to not being able to reproduce the problem and not fully understanding it.

On further investigation, there are two things required to see this bug, which is why it rarely pops up onto the radar of the developers: The binaries must be built in debug mode (by undef'ing NDEBUG), and then the opening of the font file must fail, either because it's an unsupported format or some kind of IO error. Indeed, with NDEBUG defined, the --ob_refcnt lines are not ideal, but harmless, so that's one workaround for Gentoo (it should produce faster code anyway). As for the invalid font file, is Gentoo using something other than the Bakoma fonts that ship with matplotlib? You can determine that using:

In [1]: from matplotlib import font_manager
In [4]: font_manager.findfont("cmr10")

Does the file exist -- is it the same as the one that ships with matplotlib? If not, can you send me a link to it so I can investigate further?

Mike

@mdboom
Matplotlib Developers member

BTW: We need to fix this bug on 1.2.x and in master -- but I should note also to address the issue of using a system pycxx, see #1454. Note that I was wrong about the PyCapsule support -- it's a non-issue. The remaining issue is the lack of buffer support on Python 3.x. I'll submit a patch upstream for that, and once that's in a PyCXX and we merge the setup retooling in #1454, we should be able to not need the local copy of CXX (though we may keep it around in the source tree for a while to support to keep the source install as easy as possible).

@idella

hey Mike;

good to see you're onto this. This is all about the release 1.2.0
archtester matplotlib # eix matplotlib
[D] dev-python/matplotlib
Available versions: 1.0.1-r1 1.1.0 (~)1.1.1 {{cairo doc examples excel fltk gtk latex qt4 test tk traits wxwidgets}}
Installed versions: 1.2.0^m(04:34:02 25/11/12)(cairo gtk qt4 wxwidgets -doc -examples -excel -fltk -latex -test -tk)

A user has at this point gone to quite a bit of effort to prepare an ebuild to bump matplotlib to 1.2.0. So far we have the hindrance of this bug and a tardy pyparsing upstream seemingly sitting on their hands re releasing a next version with a 1 line patch to fix an issue in python3. I've entered the patch and bumped it anyway, the preferred option of course being to have the next release release requiring no patching.

with python2.7 eselected,

matplotlib # bpython

from matplotlib import font_manager
font_manager.findfont("cmr10")
'/usr/lib64/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/cmr10.ttf'
quit()

Looks fine to me. "BTW: We need to fix this bug on 1.2.x and in master "
Yes. This appears to be the final hurdle holding back bumping matplotlib for gentoo
Is that all you need from this end?

@mdboom
Matplotlib Developers member

Ok -- on our end, I'll remove the --ob_refcnt lines. Gentoo can either patch to do the same or not build in DEBUG mode (which seems an odd thing to do in the first place). I will probably not be making a 1.2.1 release for this fix alone, but obviously if we have a 1.2.1 release this fix will be in there.

It remains a puzzle as to why loading that font is failing. If you patch out the --ob_refcnt lines, you should get an exception about why the font loading is failing that may provide more clues.

Ideally, this is the sort of thing that would get sorted out during the release candidate cycle. Do you have contact information for the maintainer of the matplotlib Gentoo package? I can add that person to my list of people that I notify about release candidates etc., though better yet would be for that person to monitor the github download page for new releases.

@idella

hey Mike, good work. Thanks so much

"Do you have contact information for the maintainer of the matplotlib Gentoo package"
You have been talking to him already, me and my ilk in the python herd. You have the luxury of choice here;

Don't forget the 4 so I can distinguish from idellas 5 & 6. They're quite temperamental you know.

" though better yet would be for that person to monitor the github download page for new releases."
We even have someone else in our irc neighbourhood with not enough to do who is good at such ventures. It's 'all good'. The patch I shall clone and enter and subsequently bump matplotlib-1.2.0 to its rightful place, in working order, in portage.

A pleasure doing business with you. So many bugs I file draw no reply. Well done.

@idella

I spoke a fraction too soon. The only test it bails out on now is the building of docs under python3.
I get

x86_64-pc-linux-gnu-g++ -pthread -march=athlon64 -pipe -fomit-frame-pointer -O2 -fno-strict-aliasing -fPIC -DPY_ARRAY_UNIQUE_SYMBOL=MPL_ARRAY_API -DPYCXX_ISO_CPP_LIB=1 -DPYCXX_PYTHON_2TO3=1 -I/usr/local/include -I/usr/include -I/usr/include/libpng15 -I/usr/include -I. -I/usr/lib64/python3.2/site-packages/numpy/core/include -I. -I/usr/include/python3.2 -c CXX/cxx_extensions.cxx -o build-3.2/temp.linux-x86_64-3.2/CXX/cxx_extensions.o
x86_64-pc-linux-gnu-gcc -pthread -march=athlon64 -pipe -fomit-frame-pointer -O2 -fno-strict-aliasing -fPIC -DPY_ARRAY_UNIQUE_SYMBOL=MPL_ARRAY_API -DPYCXX_ISO_CPP_LIB=1 -DPYCXX_PYTHON_2TO3=1 -I/usr/local/include -I/usr/include -I/usr/include/libpng15 -I/usr/include -I. -I/usr/lib64/python3.2/site-packages/numpy/core/include -I. -I/usr/include/python3.2 -c CXX/cxxextensions.c -o build-3.2/temp.linux-x86_64-3.2/CXX/cxxextensions.o
x86_64-pc-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -march=athlon64 -pipe -fomit-frame-pointer -O2 -fno-strict-aliasing build-3.2/temp.linux-x86_64-3.2/src/_png.o build-3.2/temp.linux-x86_64-3.2/src/mplutils.o build-3.2/temp.linux-x86_64-3.2/CXX/IndirectPythonInterface.o build-3.2/temp.linux-x86_64-3.2/CXX/cxxsupport.o build-3.2/temp.linux-x86_64-3.2/CXX/cxx_extensions.o build-3.2/temp.linux-x86_64-3.2/CXX/cxxextensions.o -L/usr/local/lib -L/usr/lib -L/usr/local/lib64 -L/usr/lib64 -L/usr/lib64 -lpng15 -lz -lstdc++ -lm -lpython3.2 -o build-3.2/lib.linux-x86_64-3.2/matplotlib/_png.cpython-32.so
File "./make.py", line 92
except Error, err:
^
SyntaxError: invalid syntax

  • ERROR: dev-python/matplotlib-1.2.0 failed (compile phase):

    matplotlib # find /mnt/gen2/TmpDir/portage/dev-python/matplotlib-1.2.0/work/matplotlib-1.2.0 -name make.py
    /mnt/gen2/TmpDir/portage/dev-python/matplotlib-1.2.0/work/matplotlib-1.2.0/doc/make.py
    /mnt/gen2/TmpDir/portage/dev-python/matplotlib-1.2.0/work/matplotlib-1.2.0/doc/pyplots/make.py

ok, so it appears that make.py is written to python2 format and setup.py doesn't include it for conversion for python3. Is this by intent, ie. python2 is sufficient for building docs? Otherwise a separate issue?

@mdboom
Matplotlib Developers member

Sorry this response fell through the cracks. We don't currently support building the docs on Python 3.x, since there are still issues with Sphinx in that context. See #1383.

I think for that reason, we should probably merge this if there are no objections.

@dmcdougall
Matplotlib Developers member

I'd like to see this rebased so Travis can do the tests for Python 3 before it gets merged.

@mdboom
Matplotlib Developers member

Good idea. Rebased.

@mdboom mdboom merged commit e64fb8c into matplotlib:v1.2.x Dec 17, 2012

1 check passed

Details default The Travis build passed
@mdboom mdboom deleted the mdboom:ob_refcnt_fix branch Aug 7, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment