Processing parameters for pipelines

[kaustubhmote]

Maybe you can try with the following script:

import numpy as np
import matplotlib.pyplot as plt
import nmrglue as ng

data = {}
data["ng-amx"] = ng.pipe.read("nmrglue_amx/test0.fid")
data["ng-dmx"] = ng.pipe.read("nmrglue_dmx/test0.fid")

ft = {}
ft["np-amx"] = ng.pipe.read("nmrpipe_amx/test4.ft2")[1][0]
ft["np-dmx"] = ng.pipe.read("nmrpipe_dmx/test4.ft2")[1][0]

# processing
for k, v in data.items():
    a, b = ng.pipe_proc.sp(v[0], v[1][0], off=0.5, end=0.95, pow=2, c=0.5)
    a, b = ng.pipe_proc.zf(a, b, zf=1, auto=True)
    a, b = ng.pipe_proc.ft(a, b)
    if k == "ng-amx":
        a, b = ng.pipe_proc.ps(a, b, p0=0, p1=-360*76,) # this was trial and error
    a, b = ng.pipe_proc.ps(a, b, p0=-41.5, p1=-8.0)
    ft[k] = b


# Plotting
fig, ax = plt.subplots(ncols=3, figsize=(15, 3))

for i, (k, v) in enumerate(ft.items()):
    ax[0].plot(v.real + i * 1e10, label=k)
    ax[1].plot(v.real, label=k,)
    ax[2].plot(v.real + i * 1e9, label=k)
    
ax[0].set_xlim(200, 1000)
ax[0].set_ylim(0, 5e10)

for axis in ax.flat:
    axis.legend(loc=1)

for axis in ax[1], ax[2]:
    axis.set_xlim(0, 100)
    axis.set_ylim(-.1e10, 0.5e10)

fig.savefig("compare.png", dpi=150)

As you can see, there are some differences, but even the position where there is noise, it is very similar in all 4 cases. The minor differences still remain (due to slight differences in the baseline, phases). Since the height is 1e11, these get really magnified, so np.allclose might not be the best way to compare the spectra, at least if you use a low tolerance.

[andrealorenzon]

Is there any procedural way to retrieve that "76" that is now made by trial and error?

Also, is this ratio between AMX and DMX values expected?

[jjhelmus]

FWIW 76.0 is stored as the 41st value in the NMRPipe header in the AMX test0.fid file but not the DMX file:

$ cat diff.py 
import numpy as np

amx_header = np.fromfile("./nmrpipe_amx/test0.fid", "float32", 512)
dmx_header = np.fromfile("./nmrpipe_dmx/test0.fid", "float32", 512)
for i, (amx_value, dmx_value) in enumerate(zip(amx_header, dmx_header)):
    if amx_value != dmx_value:
        print(i, amx_value, dmx_value)

$ python diff.py 
40 76.0 0.0
95 2048.0 1970.0
99 2048.0 1970.0
247 16984832000.0 17959842000.0
248 -11607859000.0 -10719855000.0
251 16984832000.0 17959842000.0
252 -11607859000.0 -10719855000.0
386 2048.0 1970.0

This value is not mapped into the metadata dictionary in nmrglue but likely should be.

Also note that the AMX and DMX dataset have different sizes 1024 x 2048 vs 1024 x 1970. When NMRPipe removes the digital filter it also re-sizes the first dimension.

[jjhelmus]

This parameter is named FDDMXVAL according to the definition in the fdatap.h file included with NMRPipe.

#define FDDMXVAL      40 /* Added for NMRPipe. */
#define FDDMXFLAG     41 /* Added for NMRPipe. */
#define FDDELTATR     42 /* Added for NMRPipe. */

I will update nmrglue to include this and any other new NMRPipe metadata values unless someone beats me to it. Until then there is no easy method for accessing this value.

[andrealorenzon]

Thank you. :)

[andrealorenzon]

A last doubt: must the additional PS operation be performed also on the other dimension, for DMX files?

[kaustubhmote]

No. Only the direct dimension is affected by the digital filter. None of the other dimensions will have this correction.

[jjhelmus]

With jjhelmus/nmrglue#144 the DMX value is available in the metadata dictionary in the FDDMXVAL key.