Filename oddity in 20240202 for Philips DWI

[captainnova]

Describe the bug

dcm2niix 20240202 (stable) worked as expected or better for most of our test suite, but for a Philips b=0 + TRACE series the output filenames have a strange new "r3069.43" suffix:

M_DIANTU_128_P_9975_20130508141740_701_AXIAL_DWI_r3069.43.bval
M_DIANTU_128_P_9975_20130508141740_701_AXIAL_DWI_r3069.43.bvec
M_DIANTU_128_P_9975_20130508141740_701_AXIAL_DWI_r3069.43.json
M_DIANTU_128_P_9975_20130508141740_701_AXIAL_DWI_r3069.43.nii

To reproduce

1. dcm2niix -o . -f %f_%p_%t_%s -b y dicom
2. ls

The data is from a phantom scan, so it is sharable.

Expected behavior

Up to and including dcm2niix 20230411, we got

M_DIANTU_128_P_9975_20130508141740_701_AXIAL_DWI.bval
M_DIANTU_128_P_9975_20130508141740_701_AXIAL_DWI.bvec
M_DIANTU_128_P_9975_20130508141740_701_AXIAL_DWI.json
M_DIANTU_128_P_9975_20130508141740_701_AXIAL_DWI.nii

The _r3069.43 suffix would probably be easier to deal with if I understood it, but it is very mysterious.

Going back through test runs, 20231111 had the problem too, so the change must have happened between 4/11 and 11/11 2023.

Output log

Chris Rorden's dcm2niiX version v1.0.20240202 GCC8.5.0 x86-64 (64-bit Linux)
Found 64 DICOM file(s)
Warning: Instance Number (0020,0013) order is not spatial.
::autoBids:Philips pulseSeq:'DwiSE' scanSeq:'SE' seqVariant:'SK'
Note: this appears to be a b=0+trace DWI; ADC/trace removal has been disabled.
Warning: Volume 1 appears to be derived image ADC/Isotropic (non-zero b-value with zero vector length)
Philips Scaling Values RS:RI:SS = 1.76337:0:0.019983 (see PMC3998685)
Convert 64 DICOM as ./dicom_AXIAL_DWI_SENSE_20130508141740_701_r3069.43 (448x448x32x2)
Conversion required 3.169364 seconds (0.170000 for core code).

Version

Chris Rorden's dcm2niiX version v1.0.20240202 GCC8.5.0 x86-64 (64-bit Linux)

[neurolabusc]

@captainnova this is the intended behavior, for details see issue 499. For Philips MR I do not know how a one-pass converter can determine whether including trigger times is required to avoid file naming clashes. Therefore, in ambiguous situations dcm2niix includes the trigger time in the filename. Happy to update this if someone wants to develop logic to handle all Philips files. I assume that most dcm2niix wrappers that create BIDS filenames use a second level examination of the NIfTI files to provide correct naming, so this is probably not an issue for most users. If you do want to prevent this (at the risk of naming clashes), you can provide the --ignore_trigger_times argument:

dcm2niix -f %s --ignore_trigger_times ~/path/to/dicoms

[captainnova]

OK, I probably will use --ignore_trigger_times on these DWIs since we do not do cardiac gating. But I do not see (0018,1060) or (0020,9153) in this DICOM, although there is

(0020, 9255) Nominal Respiratory Trigger Delay T FD: 0.0

[neurolabusc]

@captainnova this is a limitation with your images, not dcm2niix. The Philips DICOM conformance statement defines the private tag 2005,1030 as MRSeriesRepetitionTime with the FL (float) VR. However, you images have include this tag with the CS (code string) VR:

(2005,1030) [Repetition Time]  CS: [PLUS_A_PLUS_B\PLUS_A_PLUS_B\PLUS_A_PLUS_B\PLUS_A_PLUS_B]

If this tag, is included, it should look like this:

(2005, 1030) [Repetition Time] FL: [30.0, 35.2]

I would suggest that you work with the Philips Clinical Scientist associated with your site and check the provenance of these images. We need to understand why this tag got inserted and why it does not use the expected VR. I would not consider these images to be of archival quality.

Your images will fail with other tools including:

• gdcm and therefore pydicom
• dicm2nii
• CornerStoneJS, OHIF viewer

You could modify dcm2niix to require that the tag uses the FL VR by changing nii_dicom.cpp:

case kTRArray: {
    if (d.manufacturer != kMANUFACTURER_PHILIPS)
        break;
    if (d.isExplicitVR && ((vr[0] != 'F') || (vr[1] == 'L')))
        break;

However, that trick will only work if your data keeps the explicit VR, and will fail if any tool that touches the data converts the DICOMs to implicit VR.

[captainnova]

I agree. The discrepancy between the DICOM conformance statement for Philips 5.x and this data can be attributed to this data coming from Philips 2.6.3.6 and having sat in a test suite for several years.

Philips no longer supports 2 (or 3), but I do have to maintain compatibility with sites that have not updated their software, particularly for DWIs which are used in monitoring exams for acute drug reactions. Since those exams are frequent they tend to be from small clinics close to where study participants live, and updating software licenses is unwelcome overhead to them. However, 2.6.3.6 has probably aged out from what 3T sites are likely to be using. I found a 3.2.3 DWI from January 2023, and its DICOM resembles a 5.x DWI much more than it does the 2.6 one. (The repetition times matched 5.)

In other words, I don't want to bug Philips over a problem they apparently fixed years ago, and if any scans trickle in from ancient scanners I can handle them manually. But I'll be much better equipped to do that now, thanks. This has been very illuminating.

[neurolabusc]

@captainnova I did not realize Philips previously used this tag for string values. That is particularly dangerous if data does become implicit VR. However, given that this data exists in the real world, perhaps we should have a check that makes sure that only uses these values if they are numeric. Your access to a vast amount of historical data is tremendously useful for the community, as it identifies edge cases like this.

[captainnova]

I did think it was odd to put so much weight on (2005, 1030), but I am not using cardiac gating, and in the Philips DWIs I see (2005, 1030) is usually the same as (0018, 0080) Repetition Time, except for the VR (DS for 0018,0080, FL for 2005,1030 when it is a TR).

More digging turned up:

Several Philips 2.1.5\2.1.5.5\Gyroscan PMS/DICOM 4.0 MR datadefs $Revision: 6.30 $ DWIs from as late as 20201214, where (2005, 1030) looks the same as it does in Philips 3 and 5, i.e. it is TR, meaning that the original 2.6.3.6 example is anomalous.
2.1.5 did have the PLUS_A_PLUS_B\PLUS_A_PLUS_B\PLUS_A_PLUS_B\PLUS_A_PLUS_B CS a bit farther down, though, at (2005, 1330).

A few NT 10.4.1\PIIM V2.1.4.1 MIMIT MCS Philips DWIs from 2019 (the site is apparently no longer operating). It had a different PACS, but otherwise looked like 2.1.5.

It is not feasible to share these examples; the study is working on a public release of its data, but it will be heavily anonymized and I would not expect the DICOM to be pristine.

Since Philips software versions both before and after 2.6 were apparently benign with respect to dcm2niix, maybe the 2.6 example is a misleading blip? The old DWIs having PLUS_A_... at (2005, 1330) makes me wonder if a PACS or somesuch rearranged things.

[neurolabusc]

@captainnova well spotted. Due to

(2001,0010) LO [Philips Imaging DD 001]                 #  22, 1 PrivateCreator

dcm2niix is correctly remapping

(2005,1230) FL 3069.42725

to be

(2005,1030) FL 3069.42725

which is appropriate. The DICOM specification does allow private tag renumbering to avoid naming conflicts. The issue is that dcm2niix is not recognizing

(2005,0014) LO [Philips MR Imaging DD 002]

which presumably should make it renumber 2005,1030 to a different tag.

[neurolabusc]

gdcmdump gets this right

(2005,0010) LO [Philips MR Imaging DD 004 ]  # 26,1 Private Creator

remaps (2005,1030) -> (2005,1330) CS MRInPlaneTransforms

[neurolabusc]

@captainnova please carefully test the latest commit to the development branch that attempts to support more Philips Private Data Element Tags.

While the remapping in your files is legal, it will confuse simple tools that do not know the Philips recoding. Note that the remapping is not required, as this does not prevent tag clashes, rather the order is just scrambling the remapping:

2005,1030 -> 2005,1330 CS MRInPlaneTransforms
2005,1103 -> 2005,1403 MRNumberOfSPSCodes
2005,1286 -> 2005,1086 NumberOfGeometry
2005,1300 -> 2005,1200 NumberOfSOPCommon
2005,1499 -> 2005,1199 NumberOfRequestExcerpts

DICOM is complicated enough without obfuscation. The tool that implemented this should be admonished.

[captainnova]

@captainnova please carefully test the latest commit to the development branch

It works, thanks. It is like the ~1 year ago behavior of not adding _ to filenames. I tested with Philips DWIs that both did and did not have the MRInPlaneTransforms tag, including at least one that had it but was not the example I sent you.

While the remapping in your files is legal, it will confuse simple tools that do not know the Philips recoding. Note that the remapping is not required, as this does not prevent tag clashes, rather the order is just scrambling the remapping:

It seems that for a while the different Philips MRI Imaging DD blocks were all scrambling to get (2005, 0010), and then taking (2005, 0011) etc. in a race.

DICOM is complicated enough without obfuscation. The tool that implemented this should be admonished.

I agree, but it does seem to have been only in Philips 3.x, which they replaced years ago. Most software, including pydicom, dcmdump, and dicombrowser, is unable to follow this chain of relocations. In the new example I am looking at now, MRInPlaneTransforms was at top level (2005, 1030), but there is no top level Philips MR Imaging DD 004 to go with it, so even gdcmdump would miss it. There is a top level Philips MR Imaging DD 005, and a nested 004 in a different sequence, but I do not see how somebody would connect the definition without already having a lot of background knowledge. The PLUS_A_PLUS_B value is very distinctive; who knows how many 0s and 1s are mixed up.