Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incorrect pixdim3 in nifti header for philips dwi data #546

Closed
psadil opened this issue Sep 29, 2021 · 9 comments
Closed

incorrect pixdim3 in nifti header for philips dwi data #546

psadil opened this issue Sep 29, 2021 · 9 comments

Comments

@psadil
Copy link

psadil commented Sep 29, 2021

Describe the issue

The release candidate constructs Philips DWI images strangely, but not previous stable version. It seems to be estimating pixdim3 incorrectly.

To reproduce

  1. build from master (e973e0f) and development (001336f) with cmake
mkdir build && cd build
cmake ..
make
  1. run dcm2niix v1.0.20210317 and v1.0.20210928
  2. check output with fsleyes and fslinfo

v1.0.20210317

data_type	INT16
dim1		144
dim2		144
dim3		81
dim4		102
datatype	4
pixdim1		1.666667
pixdim2		1.666667
pixdim3		1.700000
pixdim4		8.093593
cal_max		0.000000
cal_min		0.000000
file_type	NIFTI-1+

v1.0.20210928

data_type	INT16
dim1		144
dim2		144
dim3		81
dim4		102
datatype	4
pixdim1		1.666667
pixdim2		1.666667
pixdim3		28.900000
pixdim4		8.093593
cal_max		0.000000
cal_min		0.000000
file_type	NIFTI-1+

Expected behavior

In both cases, pixdim3 == 1.7

Output log

In the console output, I'm seeing a bunch of errors like

Error: Check sorted order: 4D dataset has XXX volumes, but volume index ranges from XX..XX

but from what I see only the DWI images end up with incorrect pixdim3

Output from previous stable version

$ dcm2niix -o stable -z o -b y -i y DICOM
Chris Rorden's dcm2niiX version v1.0.20210317  GCC9.3.0 x86-64 (64-bit Linux)
Found 105973 DICOM file(s)
Skipping non-image DICOM: DICOM/00000051/XX_1524
Skipping non-image DICOM: DICOM/00000051/XX_1525
Skipping non-image DICOM: DICOM/XX_0392
Skipping non-image DICOM: DICOM/XX_0321
Skipping non-image DICOM: DICOM/XX_0238
Skipping non-image DICOM: DICOM/00000004/XX_0520
Skipping non-image DICOM: DICOM/00000004/XX_0464
Skipping non-image DICOM: DICOM/00000004/XX_0576
Skipping non-image DICOM: DICOM/00000028/XX_0026
Skipping non-image DICOM: DICOM/00000016/XX_0301
Skipping non-image DICOM: DICOM/00000039/XX_1799
Skipping non-image DICOM: DICOM/XX_0011
Philips Scaling Values RS:RI:SS = 0.862271:0:0.0541967 (see PMC3998685)
Convert 24300 DICOM as stable/DICOM_CUFF1_20201002090808_901 (96x96x54x450)
Philips Scaling Values RS:RI:SS = 0.830769:0:0.0553062 (see PMC3998685)
Convert 24300 DICOM as stable/DICOM_CUFF2_20201002090808_1001 (96x96x54x450)
Philips Scaling Values RS:RI:SS = 2.99267:0:0.0156413 (see PMC3998685)
Convert 69 DICOM as stable/DICOM_SE_EPI_B0_PA_20201002090808_401 (144x144x69x1)
Slices not stacked: orientation varies (vNav or localizer?) [1 0 0 0 1 0] != [0 1 0 0 0 -1]
DICOM images may be missing, expected 9 spatial locations per volume, but found 3 slices.
Philips Scaling Values RS:RI:SS = 1.4779:0:0.12479 (see PMC3998685)
Convert 3 DICOM as stable/DICOM_Localizer_20201002090808_101_i00007 (320x320x3x1)
DICOM images may be missing, expected 9 spatial locations per volume, but found 3 slices.
Philips Scaling Values RS:RI:SS = 1.4779:0:0.12479 (see PMC3998685)
Convert 3 DICOM as stable/DICOM_Localizer_20201002090808_101_i00001 (320x320x3x1)
DICOM images may be missing, expected 9 spatial locations per volume, but found 3 slices.
Philips Scaling Values RS:RI:SS = 1.4779:0:0.12479 (see PMC3998685)
Convert 3 DICOM as stable/DICOM_Localizer_20201002090808_101_i00004 (320x320x3x1)
Philips Scaling Values RS:RI:SS = 0.93431:0:0.0499011 (see PMC3998685)
Convert 24300 DICOM as stable/DICOM_REST1_20201002090808_801 (96x96x54x450)
Philips Scaling Values RS:RI:SS = 3.72259:0:0.00423683 (see PMC3998685)
Convert 225 DICOM as stable/DICOM_WIP_T1_MPRAGE_20201002090808_201 (256x256x225x1)
Philips Scaling Values RS:RI:SS = 2.4381:0:0.0120176 (see PMC3998685)
Convert 54 DICOM as stable/DICOM_GE_EPI_B0_AP_20201002090808_601 (96x96x54x1)
Philips Scaling Values RS:RI:SS = 0.900611:0:0.0481327 (see PMC3998685)
Convert 24300 DICOM as stable/DICOM_REST2_20201002090808_1101 (96x96x54x450)
Philips Scaling Values RS:RI:SS = 1.46813:0:0.0221851 (see PMC3998685)
Convert 8262 DICOM as stable/DICOM_DWI_20201002090808_501 (144x144x81x102)
Philips Scaling Values RS:RI:SS = 3.14066:0:0.0153661 (see PMC3998685)
Convert 81 DICOM as stable/DICOM_SE_EPI_B0_AP_20201002090808_301 (144x144x81x1)
Philips Scaling Values RS:RI:SS = 3.21734:0:0.01198 (see PMC3998685)
Convert 54 DICOM as stable/DICOM_GE_EPI_B0_PA_20201002090808_701 (96x96x54x1)
Conversion required 121.811780 seconds (70.128006 for core code).

And for release candidate:

$ dcm2niix -o out_new -z o -b y -i y DICOM
Chris Rorden's dcm2niiX version v1.0.20210928  GCC9.3.0 x86-64 (64-bit Linux)
Found 105973 DICOM file(s)
Skipping non-image DICOM: DICOM/00000051/XX_1524
Skipping non-image DICOM: DICOM/00000051/XX_1525
Skipping non-image DICOM: DICOM/XX_0392
Skipping non-image DICOM: DICOM/XX_0321
Skipping non-image DICOM: DICOM/XX_0238
Skipping non-image DICOM: DICOM/00000004/XX_0520
Skipping non-image DICOM: DICOM/00000004/XX_0464
Skipping non-image DICOM: DICOM/00000004/XX_0576
Skipping non-image DICOM: DICOM/00000028/XX_0026
Skipping non-image DICOM: DICOM/00000016/XX_0301
Skipping non-image DICOM: DICOM/00000039/XX_1799
Skipping non-image DICOM: DICOM/XX_0011
Error: Check sorted order: 4D dataset has 450 volumes, but volume index ranges from 1..2
Philips Scaling Values RS:RI:SS = 0.862271:0:0.0541967 (see PMC3998685)
Convert 24300 DICOM as out_new/DICOM_CUFF1_20201002090808_901 (96x96x54x450)
Error: Check sorted order: 4D dataset has 450 volumes, but volume index ranges from 1..2
Philips Scaling Values RS:RI:SS = 0.830769:0:0.0553062 (see PMC3998685)
Convert 24300 DICOM as out_new/DICOM_CUFF2_20201002090808_1001 (96x96x54x450)
Philips Scaling Values RS:RI:SS = 2.99267:0:0.0156413 (see PMC3998685)
Convert 69 DICOM as out_new/DICOM_SE_EPI_B0_PA_20201002090808_401 (144x144x69x1)
Slices not stacked: orientation varies (vNav or localizer?) [1 0 0 0 1 0] != [0 1 0 0 0 -1]
DICOM images may be missing, expected 9 spatial locations per volume, but found 3 slices.
Philips Scaling Values RS:RI:SS = 1.4779:0:0.12479 (see PMC3998685)
Convert 3 DICOM as out_new/DICOM_Localizer_20201002090808_101_i00007 (320x320x3x1)
DICOM images may be missing, expected 9 spatial locations per volume, but found 3 slices.
Philips Scaling Values RS:RI:SS = 1.4779:0:0.12479 (see PMC3998685)
Convert 3 DICOM as out_new/DICOM_Localizer_20201002090808_101_i00001 (320x320x3x1)
DICOM images may be missing, expected 9 spatial locations per volume, but found 3 slices.
Philips Scaling Values RS:RI:SS = 1.4779:0:0.12479 (see PMC3998685)
Convert 3 DICOM as out_new/DICOM_Localizer_20201002090808_101_i00004 (320x320x3x1)
Error: Check sorted order: 4D dataset has 450 volumes, but volume index ranges from 1..2
Philips Scaling Values RS:RI:SS = 0.93431:0:0.0499011 (see PMC3998685)
Convert 24300 DICOM as out_new/DICOM_REST1_20201002090808_801 (96x96x54x450)
Philips Scaling Values RS:RI:SS = 3.72259:0:0.00423683 (see PMC3998685)
Convert 225 DICOM as out_new/DICOM_WIP_T1_MPRAGE_20201002090808_201 (256x256x225x1)
Philips Scaling Values RS:RI:SS = 2.4381:0:0.0120176 (see PMC3998685)
Convert 54 DICOM as out_new/DICOM_GE_EPI_B0_AP_20201002090808_601 (96x96x54x1)
Error: Check sorted order: 4D dataset has 450 volumes, but volume index ranges from 1..2
Philips Scaling Values RS:RI:SS = 0.900611:0:0.0481327 (see PMC3998685)
Convert 24300 DICOM as out_new/DICOM_REST2_20201002090808_1101 (96x96x54x450)
Error: Check sorted order: 4D dataset has 102 volumes, but volume index ranges from 1..88
Philips Scaling Values RS:RI:SS = 1.46813:0:0.0221851 (see PMC3998685)
Convert 8262 DICOM as out_new/DICOM_DWI_20201002090808_501 (144x144x81x102)
Philips Scaling Values RS:RI:SS = 3.14066:0:0.0153661 (see PMC3998685)
Convert 81 DICOM as out_new/DICOM_SE_EPI_B0_AP_20201002090808_301 (144x144x81x1)
Philips Scaling Values RS:RI:SS = 3.21734:0:0.01198 (see PMC3998685)
Convert 54 DICOM as out_new/DICOM_GE_EPI_B0_PA_20201002090808_701 (96x96x54x1)
Conversion required 85.490102 seconds (48.459141 for core code).

Version

okay dwi: v1.0.20210317 GCC9.3.0 x86-64 (64-bit Linux) (e973e0f)
poor dwi: v1.0.20210928 GCC9.3.0 x86-64 (64-bit Linux) (001336f),

This was first observed because I was looking at using v1.0.202100811 for it's handling of GE fieldmap data. That version also produced niftis with incorrect pixdim3 on the Philips data.

Troubleshooting

If it would help, an example dataset may be shareable.

@neurolabusc
Copy link
Collaborator

Please provide exemplar
data, e.g. link to zip archive of DICOMs sent to email address in my avatar.

@psadil
Copy link
Author

psadil commented Sep 29, 2021

Great, thanks. I sent an email linking to the data.

@neurolabusc
Copy link
Collaborator

@psadil I have never seen a Philips classic DICOM like this.

  1. The stable release of dcm2niix v1.0.20210317 assumes that instance number is meaningful, storing slices as XYZT or XYTZ. However, the DICOM standard does not require instance numbers to be sequential, meaningful or even unique. In practice, some Philips systems assign instance numbers in an order that does not correspond to slice order which will confound simple algorithms.
  2. In your example, instance number does predict slice order (XYTZ).
  3. Unlike previous examples of Philips DWI data I have seen, GradientOrientationNumber (2005,1413) does not uniquely designate a volume, but is shared by different volumes. We must consider both GradientOrientationNumber as well as BValueNumber (2005,1412). Consider your data where instance 1 and 34 share GradientOrientationNumber ( = 1, the gradient vector [1 0 0]) but are distnguished by BValueNumber ( 1 vs 6, for 0 and 0.01 s/mm^2). I assume this is a new method devised by Philips to allow custom gradient tables - your example there are multiple volumes that are ~b = 0, and the same vectors are used for b=1000 and b=2000.

Your present data is easy to compensate for, but we should make sure that our decisions do not have unintended consequences. The issues are:

  1. We need to sort on two variables: 2005,1412 and 2005,1413. We should consider order of this sort. Ideally, this sort would allow us to extract the temporal order.
  2. We should make sure the order we sort classic data is the same order we use for enhanced order.
  3. We should make sure that the new Philips format is never able to generate DICOMs where both BValueNumber and GradientOrientationNumber are identical for different volumes. In other words, you can never acquire two b=2000 volumes with identical b-vectors in the same series. If this is possible, our algorithm will fail to reliably sort slices between volumes acquired at different times, leading to artificial motion artifacts. I suspect this explains where your sequence acquires six volumes that are effectively b=0 but are stored with tiny variations (0.0, 0.01, 0.02, 0.03, 0.04, 0.05).

I think @sandeepganji is uniquely placed to resolve this. I actually think the data he already provided allows us to validate this. His series 23 is sort of the inverse of yours: it acquires multiple volumes with identical near zero b-values (b=2), but uses different gradient directions for each. This further supports the notion that Philips does not allow you to acquire two volumes with identical b-value and direction in the same series (which is good news for us, as it provides a method to extract them).

I am committing a patch that illustrates the fix, but please consider it tentative until Sandeep confirms our questions and we decide on the order of sorting. I think that my current kludge will restore the diffusion volume order seen for classic DICOMs in prior releases, but has the consequence that it save volume order differently for classic and enhanced DICOMs. If we can confirm one of these orders is the correct temporal one, we can make both formats consistent.

Worth notifying @captainnova as he is a world leader on these issues, and may have opinions.

Below are the relevant DICOM tags for the first 36 slices (ordered by instance number) to provide a concrete example of the issue:

0020,0013 2001,1003 2005,1412 2005,1413
1 0 1 1
2 3000 2 2
3 3000 2 3
4 2000 3 4
5 3000 2 5
6 1000 4 4
7 3000 2 6
8 3000 2 7
9 2000 3 8
10 3000 2 9
11 1000 4 8
12 3000 2 10
13 3000 2 11
14 500 5 12
15 3000 2 13
16 2000 3 14
17 3000 2 15
18 3000 2 16
19 1000 4 17
20 3000 2 18
21 500 5 19
22 3000 2 20
23 3000 2 21
24 2000 3 22
25 3000 2 23
26 1000 4 24
27 3000 2 25
28 3000 2 26
29 2000 3 27
30 3000 2 28
31 1000 4 27
32 3000 2 29
33 3000 2 30
34 0.01 6 1
35 3000 2 31
36 2000 3 32

@captainnova
Copy link
Collaborator

Ha, I would say I am more of a world follower. But this did pique my interest and I do have lots of opinions and questions.

Since gradient cooling is apparently a big problem for wide-bore dMRI, I would not be surprised if Philips added an option to switch the b vectors between successive slices, as GE has already done. In that case each volume would have several GradientOrientationNumbers, and each GradientOrientationNumber would be scattered across several volumes. (Since that is controversial, it should be an option.) Or as @neurolabusc suggested, it could simply be a way to allow multiple b=0 volumes, which the one-to-one GradientOrientationNumber to volume scheme does not support (without using the small b trick). Either way, it seems like the days of the one-to-one GradientOrientationNumber to volume scheme are probably numbered.

@psadil, do you have access to the enhanced DICOM version of this scan? Would that avoid the problem?

If Philips is planning to add a scheme other than the one-to-one GradientOrientationNumber to volume scheme, they could help by

  • add a (probably private) tag(s) saying what the b (vector) to slice mapping, and/or slice to volume mappings are, and/or
  • if appropriate, warning on the scanner against saving affected scans in classic DICOM.

@psadil
Copy link
Author

psadil commented Oct 1, 2021

@captainnova Thanks for the insight! For this participant, it may be possible to get the enhanced DICOM version, though it isn't guaranteed. Unfortunately, for the rest of the data enhanced versions will not be available. But please let me know if the enhanced version of this example would help, e.g., validate a patch.

@neurolabusc
Copy link
Collaborator

@psadil seeing a single example of a scan with this setup in both classic and enhanced would help.

@psadil
Copy link
Author

psadil commented Oct 1, 2021

@neurolabusc, okay. I'll check and will report back when I know whether a scan in both classic and enhanced versions could be available

@neurolabusc
Copy link
Collaborator

@psadil and @captainnova I am now ready to close this issue. For the Fall 2021 release, Philips enhanced DICOM diffusion volumes will be sorted as instructed by the Dimension Index Values (0020,9157). On the other hand, for classic DICOMs, volumes with identical b-value index (2005,1412) will be stored sequentially, with ties sorted based on gradient direction number (2005,1413). This order should be identical to prior releases of dcm2niix. However, the new release is robust against classic data where the instance number (0020,0013) is jumbled. The DICOM standard does not require meaningful instance numbers, though in my experience all other manufacturers and many tools do provide/expect sequential instance numbers.

Since different algorithms are used, the order of volumes for Philips diffusion may differ depending on whether the data is stored as classic or enhanced DICOM. As @captainnova notes, for many sequences it is probable that neither order matches the true temporal order that data were acquired. Users of Philips data will need to be cautious if the processing tools make assumptions about head motion or T1 effects based on the order that data is stored to disk. This is a limitation of the Philips image data, not dcm2niix. Perhaps future releases of dcm2niix will be able to parse the data Philips stores in non-image DICOM files (though I am aware these often are not retained by PACS systems).

Thanks to James Ford at Dartmouth for supplying an example of a sequence exhibiting these attributes that was saved as both classic and enhanced data.

@neurolabusc
Copy link
Collaborator

@sandeepganji noted that recent Philips software (from R5.6) includes a new tag 2005,1596 for diffusion data that provides the acquired temporal order of volumes.

Thanks to Paul Morgan for providing a public validation dataset. This method is now used for both enhanced and classic DICOMs. The validation dataset includes text to describe how dcm2niix handles older data.

yarikoptic added a commit to neurodebian/dcm2niix that referenced this issue Apr 29, 2024
* tag 'v1.0.20211006':
  Update dcm_qa submodule.
  Unify PASL JSON tags.
  Extract volume order for Philips DWI (from R5.6 onward)
  Version bump, describe issue 546 in documentation
  Remove warning for early GE HyperBand dataset after validation
  Handle enhanced DICOM using syntax 1.2.840.10008.1.2.4.70
  Use both 2005,1412 and 2005,1413 to sort Philips DWI (rordenlab#546)
  GE early HyperBand
  Slice Times for early GE HyperBand fMRI
  Issue 544 (rordenlab#544)
  Yet another attempt to use packed structures for Windows MSVC 19 and UNIX.
  Support packed structs for Windows and Unix (https://newbedev.com/visual-c-equivalent-of-gcc-s-attribute-packed)
  Experimental support for MGH format export
  DWI vector removal detection (rordenlab#542)
  Verbose = 2 will report ProtocolBlockGE (0025,101B) for all GE scans (not only if slice timing is calculated)
  Typos found by codespell
  Never equalize slices if slices have been rotated
  issue 539 (rordenlab#539)
  Set repetitionTimePreparation for known ASL sequences, reduce verbosity of slice re-ordering, improve ECAT success/failure warnings.
  Update issue templates
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants