Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unusual behavior in Dipy IVIM implementation/example #1817

Closed
rutgerfick opened this issue May 2, 2019 · 9 comments · Fixed by #1931

Comments

@rutgerfick
Copy link
Contributor

commented May 2, 2019

Description

Dear Dipy developers,

As always, I admire the beautiful work you do for the neurimaging community.

However, coming across your IVIM example, I noticed some unusual behavior:

  • the values the model finds for D* diffusivity occasionally are over 300x that of free water - while the optimization bounds for D* water diffusivity should be at most 20e-9 m^2/s.
  • Also, the histogram of the perfusion volume fractions have a strange peak at 0.25 (which appears to be some threshold in your IVIM implementation algorithm).

I've added a link to a demonstration of the described behavior below, compared to a reference Dmipy IVIM implementation.

Kind regards,
Rutger Fick

Way to reproduce

[If reporting a bug, please include the following important information:]

  • Code example: your own IVIM example, but I compare implementations at https://nbviewer.jupyter.org/github/AthenaEPI/dmipy/blob/master/examples/example_ivim.ipynb
  • Operating system and version (run python -c "import platform; print(platform.platform())"): Linux-4.15.0-47-generic-x86_64-with-debian-stretch-sid
  • Python version (run python -c "import sys; print("Python", sys.version)"): '2.7.15 |Anaconda, Inc.| (default, Dec 14 2018, 19:04:19) \n[GCC 7.3.0]
  • dipy version (run python -c "import dipy; print(dipy.__version__)"): 0.16.0
  • dependency version (numpy, scipy, nibabel, h5py, cvxpy, fury)
    • import numpy; print("NumPy", numpy.version): 1.16.2
    • import scipy; print("SciPy", scipy.version): 1.1.0
    • import nibabel; print("Nibabel", nibabel.version): 2.3.3
    • import h5py; print("H5py", h5py.version) N/A
    • import cvxpy; print("Cvxpy", cvxpy.version): 1.0.21
    • import fury; print("fury", fury.version): N/A
@arokem

This comment has been minimized.

Copy link
Member

commented May 2, 2019

@ShreyasFadnavis

This comment has been minimized.

Copy link
Member

commented May 2, 2019

@rutgerfick thank you so much for pointing this out. I will look into the resources you have shared and the issues you have pointed out!

@skoudoro skoudoro added this to the 1.0 milestone May 3, 2019

@rutgerfick

This comment has been minimized.

Copy link
Contributor Author

commented May 5, 2019

@arokem You can compare the results for VarPro yourself if you just change the solver option in the notebook reference above.

I'm not familiar with the implementation but the VarPro results seem to look within expected boundaries. My only doubts are why you hard-constraint the D* volume fraction to be below 0.25. The resulting D* volume fraction image mostly reflects the constraint on the optimizer rather than the data. Also VarPro is much slower (20+ min for the slice compare to 1min+ for your other solver) but that's of secondary concern I suppose.

@arokem

This comment has been minimized.

Copy link
Member

commented May 5, 2019

I think that the issue of very high D* values could be fixed through a change to this line:

https://github.com/nipy/dipy/blob/master/dipy/reconst/ivim.py#L289

to something like:

self.bounds = ((0., 0., 0., 0.), (np.inf, .3, 0.2, 1.))

(did I get the scale right?)

Where does that 0.25 come from?

@rutgerfick

This comment has been minimized.

Copy link
Contributor Author

commented May 5, 2019

Seems its hard-coded into BOTH your implementations. No comments indicating as to why.

self.bounds = ((0., 0., 0., 0.), (np.inf, .3, 1., 1.))

f[0] <= 0.29,

I suggest you document it somewhere or put a warning since this is typically one of those impossible-to-know blackbox settings that may mysteriously affect your results (and clearly does if you take a look at the parameter maps).

@ShreyasFadnavis

This comment has been minimized.

Copy link
Member

commented May 6, 2019

@arokem what do you suggest? Should I write the documentation? Or should I write warnings for the bounds...?

@arokem

This comment has been minimized.

Copy link
Member

commented May 6, 2019

I think that we should change the bounds and maybe set them as a global variable, so they apply to both implementations.

Better documentation would certainly also be useful to have.

@rutgerfick : do you have a good reference for the 20e-9 m^2/s number?

@rutgerfick

This comment has been minimized.

Copy link
Contributor Author

commented May 6, 2019

@arokem for example Gurney et al 2018. But essentially they find that simply fixing D* to 7e-9 results in the second-best results for D and f (after fancy bayesian fitting), which is also the Dmipy implementation that I compared with the Dipy one. The question is therefore pertinent what is gained by including fancy MIX fitting for IVIM, but frankly I'm not an IVIM expert :-)

Also keep in mind that these guys also essentially say that D* is an unstable and unreliable biomarker, and should be interpreted with caution.

@arokem

This comment has been minimized.

Copy link
Member

commented May 6, 2019

Thanks! These are useful references.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.