Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIL pixel data handler failing for compressed images #769

Closed
fakyras opened this issue Nov 1, 2018 · 18 comments
Closed

PIL pixel data handler failing for compressed images #769

fakyras opened this issue Nov 1, 2018 · 18 comments
Labels

Comments

@fakyras
Copy link

@fakyras fakyras commented Nov 1, 2018

I have been working with CXR dicoms from https://data.tbportals.niaid.nih.gov/

All CXR dicoms are compressed images. The problem is that pydicom (1.2.0) fails to read pixel_array data from the dicoms on roughly 5% of the cases.

import pydicom
pydicom.read_file('test.dcm').pixel_array
Traceback (most recent call last):
  File "/home/db/venv/lib/python3.5/site-packages/pydicom/pixel_data_handlers/pillow_handler.py", line 197, in get_pixeldata
    decompressed_image = Image.open(fio)
  File "/home/db/venv/lib/python3.5/site-packages/PIL/Image.py", line 2585, in open
    % (filename if filename else fp))
OSError: cannot identify image file <_io.BytesIO object at 0x7f48710270a0>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/db/venv/lib/python3.5/site-packages/pydicom/dataset.py", line 901, in pixel_array
    self.convert_pixel_data()
  File "/home/db/venv/lib/python3.5/site-packages/pydicom/dataset.py", line 847, in convert_pixel_data
    raise last_exception
  File "/home/db/venv/lib/python3.5/site-packages/pydicom/dataset.py", line 815, in convert_pixel_data
    arr = handler.get_pixeldata(self)
  File "/home/db/venv/lib/python3.5/site-packages/pydicom/pixel_data_handlers/pillow_handler.py", line 199, in get_pixeldata
    raise NotImplementedError(e.strerror)
NotImplementedError: None

The test.dcm file can be found here: https://files.fm/f/gg266dea
There is no problem to open the dicom with weasis.

Any help on how to debug this would be highly appreciated.

@scaramallion

This comment has been minimized.

Copy link
Member

@scaramallion scaramallion commented Nov 1, 2018

Transfer Syntax is JPEG Lossless, Process 14, SV 1, 16/10 bits allocated/stored, 2140x1760, 1 sample/pixel, 1 frame. Issue reproducible with PIL 5.1.0 and current master.

I don't think PIL can handle JPEG files with 10 bit original sample data, but it works OK with the GDCM handler so maybe try that instead?

We should also fix up that buggy exception...

@mrbean-bremen

This comment has been minimized.

Copy link
Member

@mrbean-bremen mrbean-bremen commented Nov 2, 2018

We should also update the supported transfer syntax table - currently this is shown as supported.

@aditaker

This comment has been minimized.

Copy link

@aditaker aditaker commented Nov 13, 2018

I am also facing the same issue. Can someone please suggest a solution for this issue?

@mrbean-bremen

This comment has been minimized.

Copy link
Member

@mrbean-bremen mrbean-bremen commented Nov 13, 2018

Have you tried using GDCM? Seems like Pillow doesn't handle this transfer syntax.

@LarsAC

This comment has been minimized.

Copy link

@LarsAC LarsAC commented Nov 23, 2018

How would I enforce use of GDCM ?

@mrbean-bremen

This comment has been minimized.

Copy link
Member

@mrbean-bremen mrbean-bremen commented Nov 23, 2018

Currently, the order of pixel handlers is fixed, particularly the GDCM handler is tried before the Pillow handler - so with GDCM installed (and detected by pydicom), it shall always be preferred over Pillow.

@fakyras

This comment has been minimized.

Copy link
Author

@fakyras fakyras commented Nov 23, 2018

Back to original question.
GDCM does not solve the issue:

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/db/venv/local/lib/python3.6/site-packages/pydicom/dataset.py", line 891, in pixel_array return self._get_pixel_array() File "/home/db/venv/local/lib/python3.6/site-packages/pydicom/dataset.py", line 782, in _get_pixel_array self.convert_pixel_data() File "/home/db/venv/local/lib/python3.6/site-packages/pydicom/dataset.py", line 835, in convert_pixel_data raise NotImplementedError(msg) NotImplementedError: No available image handler could decode this transfer syntax JPEG Lossless, Non-Hierarchical, First-Order Prediction (Process 14 [Selection Value 1])

python-gdcm (2.8.4-1build2) is isntalled in Ubuntu 18.04

So having GDCM handler does not solve the problem

@scaramallion

This comment has been minimized.

Copy link
Member

@scaramallion scaramallion commented Nov 24, 2018

You can import gdcm OK? Because I did test it with GDCM and it did work.

@jamesjjcondon

This comment has been minimized.

Copy link

@jamesjjcondon jamesjjcondon commented Nov 29, 2018

I had a similar problem with 16/16 bits allocated/stored with jpeg lossless compression. Was able to decompress with http://gdcm.sourceforge.net/html/gdcmconv.html although now my dicoms (and pixel arrays) are inverted. And I have no idea why or how to fix that...

@scaramallion

This comment has been minimized.

Copy link
Member

@scaramallion scaramallion commented Nov 30, 2018

If gdcmconv is producing the wrong output that sounds more like a GDCM issue then a pydicom issue.

@fakyras

This comment has been minimized.

Copy link
Author

@fakyras fakyras commented Dec 13, 2018

Installing gdcm solved the issue completely.

However, the only thing that worked for me was installing gdcm from source.

python-gdcm it seems is not an option for python3 users.

@fakyras fakyras closed this Dec 13, 2018
@mrbean-bremen

This comment has been minimized.

Copy link
Member

@mrbean-bremen mrbean-bremen commented Dec 13, 2018

I think installing using conda should also work.

@fakyras

This comment has been minimized.

Copy link
Author

@fakyras fakyras commented Dec 14, 2018

I have been avoiding conda on purpose. Just had lots of bad experience with it in the past and I have been doing just fine without it.

@mrbean-bremen

This comment has been minimized.

Copy link
Member

@mrbean-bremen mrbean-bremen commented Dec 14, 2018

Ah well... there seems to be no easy way to install gdcm...

@danielsnider

This comment has been minimized.

Copy link

@danielsnider danielsnider commented Jan 2, 2019

@mrbean-bremen I found a way to install gdcm on Ubuntu 16.04:

sudo apt-get install python3.7-dev libpython3.7-dev virtualenv
virtualenv -p python3.7 python3.7venv
cd python3.7venv/
. bin/activate
pip3.7 install numpy pydicom
git clone --branch master https://github.com/HealthplusAI/python3-gdcm.git && cd python3-gdcm && dpkg -i build_1-1_amd64.deb && apt-get install -f
cp /usr/local/lib/gdcm.py ./lib/python3.7/site-packages/
cp /usr/local/lib/gdcmswig.py ./lib/python3.7/site-packages/
cp /usr/local/lib/_gdcmswig.so ./lib/python3.7/site-packages/
cp /usr/local/lib/libgdcm* ./lib/python3.7/site-packages/
@mrbean-bremen

This comment has been minimized.

Copy link
Member

@mrbean-bremen mrbean-bremen commented Jan 2, 2019

Thanks - good to know for others! Not exactly straightforward to find out...

@danielsnider

This comment has been minimized.

Copy link

@danielsnider danielsnider commented Jan 2, 2019

I petition you recommend users to use GDCM over Pillow since multiple people have run into this issue with Pillow. Suggest using GDCM in the readme, the error message for this issue, and in the error message if the user doesn't have any pixel_handler available.

@mrbean-bremen

This comment has been minimized.

Copy link
Member

@mrbean-bremen mrbean-bremen commented Jan 2, 2019

Agreed - this should be made more clear in the documentation. The usual problem is the same you had - installation of GDCM is often quite cumbersome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.