Build doesn't detect/support libopenjpeg v.2.1.3

[ccazabon]

The docs for Pillow v.2.4.0 say "Pillow 2.4 requires openjpeg 2.0.0", but the build process says OPENJPEG/JPEG2000 support is not available when libopenjpeg v.2.1.3 is installed. I'm not sure if it's supposed to support newer 2.x versions or not, but it appears not to.

This happens on the new Ubuntu "Trusty Tahr" 2014.04, for which v.2.1.3 is the only version of libopenjpeg shipped. I have the main package, the -dev package, and the openjpeg-tools package installed.

P.S. Thanks for maintaining Pillow!

[wiredfool]

There is a report from @cgohlke about it only working on windows from exactly the 2.0.0.tar.gz release. So it's a known issue -- not that I can actually find the reference right now.

If it's not detecting the libraries, they may have renamed their headers or reorganized something. That may lead to needing patches for correctness as well.

[aclark4life]

c.f. 12398f6

[wiredfool]

ah, that's it. Thanks.

[manisandro]

openjpeg-2.1.3 does not exist. But openjpeg-2.1.0 and openjpeg-2.0.1 do, and there are some issue with those versions:

• For both openjpeg-2.1.0 and openjpeg-2.0.1, the tests fail. The upstream commit which introduced the failure is https://code.google.com/p/openjpeg/source/detail?r=2317 . I'm not sure whether the commit is bad or the test images. Would be nice if the author of the jpeg2k plugin, @al45tair, could give it a look.
• For openjpeg-2.1.0, they moved the headers to /usr/include/openjpeg-2.1, and introduced some API changes. setup.py needs to be modified for 2.1.0, and the changes for the new API are easy (opj_stream_set_user_data(stream, decoder) -> opj_stream_set_user_data(stream, decoder, NULL)).

[wiredfool]

See also #584. And potentially #547 (comment), as there are some PPC issues as well.

[al45tair]

I'll try to take a look at this over the next few days (the 2.x.x compatibility situation, I mean, not the PPC issue — I don’t have a PPC box to test on, and the problems there are in the underlying OpenJPEG library).

[manisandro]

Yes on PPC we also see issues (test failures) in Fedora, see https://bugzilla.redhat.com/show_bug.cgi?id=1100762 . I would like to update to 2.1.0 to see whether we are lucky and it fixes the issues, otherwise I'll hunt a bit around.

[al45tair]

The pull request I just sent should fix this problem. It is worth noting that the changes to 2.1 create an additional limitation on the Python side — namely that if you try to decode a non-seekable file-like object, it cannot exceed 4GB in total size. The reason for this is the code around line 460 of OpenJPEG�’s jp2.c; if it were not for this code, we could pass (OPJ_UINT64)-1 for the (badly named) user_data_length, effectively disabling some of the length checks in OpenJPEG. As it is, the largest value we can safely pass is 0xffffffff.

[wiredfool]

Sigh. I feel like I've already gone in loops about this for earlier Ubuntu versions, and didn't remember till now.

Libopenjpeg-2.1.3.so is actually openjpeg 1.3, with a copyright date of 2007 in the openjpeg.h file. I believe that the 2 in the openjpeg2 name refers to the second incompatible version that's been packaged, rather than the actual version 2 that is the current major version.

So, for all current versions of Ubuntu, openjpeg is going to be a download and install dependency.

[wiredfool]

I'll note that in the Pillow/depends folder, there's a script for installing openjpeg on the travis system for testing. If you have cmake installed on your ubuntu system, it will install a working version of openjpeg.

(note that it puts it in /usr, not /usr/local. Requires sudo power)

[aclark4life]

"Meh" what's going on here exactly? Seems like some packaging confusion on Ubuntu… is there anything we should be doing to address this issue? More documentation?

[wiredfool]

What's going on here is that Ubuntu is not packaging a useful version of openjpeg, unless we start supporting a 7 year old version. I'm going to call it a wont-fix, at least on our end.

I'll send Debian a new upstream release bug report, perhaps in a release or two, it will percolate to Ubuntu Current. (and hopefully, Trusty-backports).

[wiredfool]

The 2.1 support is failing on PPC in a similar manner to 2.0.

[calvinbutcher]

We're trying to build a Loris image server docker image running with Grok 1.0, an optimised version of OpenJPEG 2.1.

I've come across the issues mentioned by @manisandro.

Currently I'm having to template the Jpeg2KDecode.c and Jpeg2KEncode.c files with the correct 3 parameter call to OpenJPEG 2.1/Grok.

However, the docker image build then fails on the tests.