Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unlisted Dicom tags cause KeyError, and failure for all other purposes #92

Closed
darcymason opened this issue Nov 27, 2014 · 11 comments

Comments

Projects
None yet
4 participants
@darcymason
Copy link
Member

commented Nov 27, 2014

From hans.j.j...@gmail.com on September 18, 2010 09:27:51

What steps will reproduce the problem? 1. For dicom files with tags that are not listed in the _dicom_dict.py file, the files can not be read, even if the dicom tags are not of any use to the application.
2. Try to read a dicom file that has a tag (FFFF,FFFF) or (3F03,1001) (I don't know or care what those tags hold) What is the expected output? What do you see instead? I would expect the program to ignore these tags, or to fill them out with some clearly bogus placeholders to indicate that they are going to silently be ignored.

What I see instead is that the dicom.file_read() throws a KeyError stating that the key is not in the pre-defined dictionary, and the parts of the file that are extractable are not available for querying. What version of the product are you using? I tried both 0.9.4 and 0.9.5rc1. Please provide any additional information below. A hack that works for one of the data sets that was causing problems was to add the missing values to the dictionary. This is not very satisfying because I can not anticipate all the other instances of not reported tags that will encountered in the future.
[johnsonhj@neuron mpy-svn-stats]$ tail -4 /Library/Python/2.6/site-packages/dicom/_dicom_dict.py
'7Fxx0040': ('OW', '1', "Variable Coefficients SDDN", 'Retired'),
'3F031001': ('NONE', '1', "Unknown Item", ''),
'FFFFFFFF': ('NONE', '1', "Unknown Item", ''),
}

Proposal:

In dicom/datadict.py

replace:
41 def get_entry(tag):
42 """Return the tuple (VR, VM, name, is_retired) from the DICOM dictionary
43 ····
44 If the entry is not in the main dictionary, check the masked ones,
45 e.g. repeating groups like 50xx, etc.
46 """
47 tag = Tag(tag)
48 try:
49 return DicomDictionary[tag]
50 except KeyError:
51 mask_x = mask_match(tag)
52 if mask_x:
53 return RepeatersDictionary[mask_x]
54 else:
55 raise KeyError, "Tag %s not found in DICOM dictionary" % Tag(tag)
56

============= with =======================================
41 def get_entry(tag):
42 """Return the tuple (VR, VM, name, is_retired) from the DICOM dictionary
43 ····
44 If the entry is not in the main dictionary, check the masked ones,
45 e.g. repeating groups like 50xx, etc.
46 """
47 tag = Tag(tag)
48 try:
49 return DicomDictionary[tag]
50 except KeyError:
51 mask_x = mask_match(tag)
52 if mask_x:
53 return RepeatersDictionary[mask_x]
54 else:
55 return ('NONE', '1', "Unknown Item : "+str(tag), '')
56

Original issue: http://code.google.com/p/pydicom/issues/detail?id=91

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2014

From darcymason@gmail.com on September 18, 2010 09:06:21

This is strange. The two tags mention that are giving problems are private tags (group number is odd) which pydicom has always been able to ignore, since many would not be in the private dictionary anyway.

In pydicom's testfiles directory, I tried the following:

ds=dicom.read_file("CT_small.dcm")
ds.AddNew((0x3F03,0x1001),"UN", "hi")
ds.save_as("3f03")
ds2=dicom.read_file("3f03")
ds2
and it reads okay, and displays the new tag fine. Same for the (FFFF,FFFF) tag, but group FFFF is reserved by the DICOM standard (but not used as far as I can recall) so that is a strange one.

Maybe there is something different about your file. Is it implicit or explicit VR? Could you provide an anonymized example file for this?

Status: Accepted

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2014

From OScu...@gmail.com on October 01, 2010 14:16:51

Here is a dicom file that should allow you to reproduce the error.

Attachment: IM000001

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2014

From darcymason@gmail.com on October 03, 2010 11:23:27

I was able to reproduce the error using the supplied file. I looked at the hex dump -- the data element is unusual. The file is implicit VR, and the troublesome element has undefined length (FFFFFFFF) followed by a sequence item marker, so it appears to be a SQ. Tried 'SQ' instead of 'NONE' in your _dicom_dict line, and it read and displayed okay, but other than the first item, the sequence looks a bit strange.

I'll need to do some more hex dump inspection to really understand what the original file is trying to do. But I'll hold on that until after release of pydicom 0.9.5 as I don't want to introduce and new bugs with the fix for this.

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2014

From stefan.r...@gmail.com on February 17, 2011 16:45:00

Has this issue been resolved in 0.9.5? I'm encountering something that's probably related in one of my Philips MR DICOM files. Could provide file if needed.

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2014

From darcymason@gmail.com on February 17, 2011 17:17:25

No, it is still present in 0.9.5. I believe these issues ( issue 91 , issue 97 , and this one) are probably related.

Another file would be great though; it can help to see several files to see the commonalities

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2014

From stefan.r...@gmail.com on February 17, 2011 22:26:21

see attached. following the fix from issue 97 (thanks for pointing me there - should have looked more carefully) I can read the file in all its beauty. But why are private tags (x2001XXXX and x2005XXXX) looked for in the dictionary which has little chance of knowing about them?

Attachment: IMAGE8

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2014

From darcymason@gmail.com on February 18, 2011 04:08:28

Got the file (thanks) and confirmed the error. I've also tried this in pydicom 0.9.3, and the error does not occur there. So I think it is related to the switch to "raw data elements" done as part of the faster reading efforts. As part of that, SQ items were not parsed at first unless they were undefined length. Will look into it further.

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2014

From darcymason@gmail.com on February 21, 2011 09:48:02

This issue was closed by revision 861d859b4f .

Status: Fixed

@darcymason darcymason closed this Nov 27, 2014

@RemiTorracinta

This comment has been minimized.

Copy link

commented Jan 20, 2018

Not sure how related it is, but currently dealing with a bug where a dicom with private tag ffff,ffff can't be written to file. This happens when when filewriter.py (line 836) tries to slice the dataset by instantiating a Tag object (dataset.py, line 1025) with value ffff,ffff+1 (dataset.py, line 1023).

@darcymason

This comment has been minimized.

Copy link
Member Author

commented Jan 21, 2018

Yeah, that makes sense it can't add one. I've got some code in another branch that avoided the slicing. I'll try to merge that in.

@darcymason darcymason reopened this Jan 21, 2018

@mrbean-bremen mrbean-bremen added this to the v1.1 milestone Feb 18, 2018

@scaramallion

This comment has been minimized.

Copy link
Member

commented Mar 3, 2018

I can take a look at this one, I've been missing working on pydicom.

@scaramallion scaramallion self-assigned this Mar 3, 2018

scaramallion added a commit to scaramallion/pydicom that referenced this issue Mar 3, 2018

scaramallion added a commit to scaramallion/pydicom that referenced this issue Mar 3, 2018

scaramallion added a commit to scaramallion/pydicom that referenced this issue Mar 3, 2018

darcymason added a commit that referenced this issue Mar 7, 2018

Fix for Dataset slicing not working correctly for (0xFFFF, 0xFFFF) (#586
)

* Add failing unit test for #92 for dataset slicing

* Minor refactor of slicing

* Update docstring for _slice_dataset, add test to check that (0xFFFF, 0xFFFF) tuple slicing works, expand explanatory comment

@scaramallion scaramallion referenced this issue May 16, 2018

Closed

Release 1.1 #644

7 of 7 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.