Siemens TWIX header variations and possible TWIX reader improvement

[darrencl]

Hi, I'm using suspect for our MRS processing tool (largely for the IO) and some of our TWIX data varies in the regex with unknown reason. These 2 data were scanned at the same session with the same scanner, etc. but has different header for some reason.

In [135]: twix_df.loc[822].ref_epoch                                                                                                                                                                               
Out[135]: '2019-09-25 12:34:15'

In [136]: twix_df.loc[824].ref_epoch                                                                                                                                                                               
Out[136]: '2019-09-25 12:34:15'

In [137]: twix_df.loc[822].software_version                                                                                                                                                                        
Out[137]: 'N4_VE11C_LATEST_20160120'

In [138]: twix_df.loc[824].software_version                                                                                                                                                                        
Out[138]: 'N4_VE11C_LATEST_20160120'

In [139]: mrs_d = suspect.io.load_twix(twix_df.loc[822].new_location)                                                                                                                                              

In [140]: mrs_d = suspect.io.load_twix(twix_df.loc[824].new_location)                                                                                                                                              
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-140-7b0224c0e2ad> in <module>
----> 1 mrs_d = suspect.io.load_twix(twix_df.loc[824].new_location)

~/miniconda3/lib/python3.7/site-packages/suspect/io/twix.py in load_twix(filename)

~/miniconda3/lib/python3.7/site-packages/suspect/io/twix.py in load_twix_vd(fin, builder)

~/miniconda3/lib/python3.7/site-packages/suspect/io/twix.py in set_header_string(self, header_string)

~/miniconda3/lib/python3.7/site-packages/suspect/io/twix.py in parse_twix_header(header_string)

KeyError: 'Unable to identify Frequency from header'

I found out that the header key for Frequency is lFrequency, and the DwellTime can be extracted from alDwellTime.

However, the voxel information, such as size, position, etc. are not present in the part where suspect expects them to be. Even in the first ParamMap."sVoI". I noticed that we can get this information in the second ParamMap."sVoI" - further down after byte junks, which is not in the header_string variable.

I am trying to fix this (already forked suspect) and stuck due to that voxel information stuff are not found in header_string .

Another note, I am maintaining a Julia library (MagneticResonanceSignals.jl) that reads TWIX too, the TWIX IO in this library is actually inspired by suspect :-). My ex-colleague implements this reader functionality and he seems to find out that extracting frequency and dwell time is much more stable using yaps header (between ASCONV START and END). With this reader, I never got this issue on metadata stuff.

https://github.com/TRIImaging/MagneticResonanceSignals.jl/blob/9affd62cb698bdecbccd81f1a807cebe38f21464/src/io_twix.jl#L239
https://github.com/TRIImaging/MagneticResonanceSignals.jl/blob/9affd62cb698bdecbccd81f1a807cebe38f21464/src/io_twix.jl#L258

The voxel information stuff can also be found in yaps header with key sSpecPara.sVoI.*. Below is example in our MeasYaps test data

https://github.com/TRIImaging/MagneticResonanceSignals.jl/blob/892d3948287babdb1f49529d16615831202fd4d0/test/twix/PRESS_TE135_Breast_Coil_Headers/MeasYaps#L374

Another advantage with this approach is that this will also work with other scanner like VIDA (possibly closing #122 too). With that said, I think this will be useful for people who use suspect in their MRS processing script and if maintainer would like to have this, I can help implementing and submitting a PR for this.

Any thoughts on this (best way to implement this within suspect, etc.)? Thanks!

[bennyrowland]

Hi @darrencl, thanks for bringing this to my attention. When I first wrote the Twix reader I had minimal experience with Twix and no pulse programming experience, just a couple of powerpoint slides from Siemens about the raw data structure organisation. On the header side I just had to guess everything, and it is strange to me how much the header varies, as you found out with these files from the same session with different header values. Every so often these kind of issues crop up and we try and fix things by educated guesses about names to look for etc. Sorry you got caught by this one :-(

It would be great to have a PR from you to fix this if you can, or if you can share your anonymised Twix files then I am happy to take a look as well. I would probably start by simply changing the regex to look for the alternative voxel information entries as you suggest, or in fact probably iterating over known possibilities until a match is found. Let me know which of us you want to have a look at this problem.

It is cool to see a Julia library for MRS, I keep meaning to have a look at Julia but never have time. I see you are doing some COSY work with Felix, I also played around with that a little bit, a long time ago. Do you do any processing of the data in Julia, or is it just for I/O

[darrencl]

@bennyrowland that is true! It's just amazing how inconsistent these files are! :-)

That's awesome, I will submit a PR on this then! I will try to find a phantom scan with this issue, just to be save to not leaking any TWIX files of patient.

Yes, Felix has been used by my research group and they also use 2D COSY scans to get better insights. I would also like to open an issue in regards to supporting 2D COSY on suspect, at least for the reader. In our python processing package, we implement this by having a COSYMRSData which extends base object (COSYBase which extends ImageBase), since we need to record the dwell time of T1. I think it's also great to move this bits to suspect instead to add support for COSY.

Do you do any processing of the data in Julia, or is it just for I/O

There are some signal processing functions implemented in the Julia package such as channel combination, apodization, simple averaging, and zero filling and those also works for 2D data. I'm cc'ing my ex-colleague so he knows what is going on here :-) (cc: @c42f)

[c42f]

Hi guys, thanks for CC'ing me Darren. Yes @bennyrowland there's some signal processing stuff in https://github.com/TRIImaging/MagneticResonanceSignals.jl, though it's pretty light on; nothing sophisticated.