New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP : Ivim linear fitting #1110

Merged
merged 127 commits into from Sep 27, 2016

Conversation

Projects
None yet
5 participants
@sahmed95
Contributor

sahmed95 commented Aug 15, 2016

This is a continuation of #1058. Simple least squares fitting fails for some regions with the fit returning the 'f' values quite high (> 0.3) and converging to D/D_star = 0. Setting bounds for 'f' strangely sets it to the higher bound and again gives an erroneous result.
A linear fitting strategy is used here which initially fits for parameters D and S0, then finds 'f' and finally uses least squares fitting for D_star as follows :

i. Linear fitting for D (bvals > split_b) and store S0_prime
ii. Another linear fit for S0 (bvals < split_b).
iii. Estimate f using 1 - S0_prime/S0.
iv. Use least squares only to fit for D_star. 

This gives a sufficiently good prediction for S0 and D but not so good for 'f' and 'D_star'. The two_stage method can be used for a more accurate fit but that fails for noisy data or when 'f' values are very low. Two proposed solutions for that are :

Fit D and S0 linearly and fit f and D_star using least_squares
Write a customized gradient descent for bi exponenetial fits for this particular case.

Regularization, increasing the error tol, changing the loss function for least_squares (SCIPY 0.17) or the method (trf, dogleg, LM) has been tried but eveytime it seems that the solution with unfeasible 'f' (higher value) gets selected.

@arokem @etpeterson can perhaps shed more light on regularization and why it might or might not work.

@etpeterson

This comment has been minimized.

Contributor

etpeterson commented on dipy/reconst/ivim.py in f8a89a4 Jun 16, 2016

I think you need to have it 1-C/data[...,0] because C (intercept of the diffusion only) is < data[...,0](intercept of the perfusion+diffusion).

Also, does the normal DTI fit not potentially return an intercept? The way you're calculating C seems a little fishy although maybe that's because I haven't digested it fully yet.

This comment has been minimized.

Member

arokem replied Jun 16, 2016

The DTI fit will return an intercept once #1036 gets merged. For the time being, the equation to calculate the intercept from a DTI model is in that PR.

@arokem

This comment has been minimized.

Member

arokem commented Sep 18, 2016

Looks like this is still failing with older versions of Scipy? https://travis-ci.org/nipy/dipy/jobs/159584745

@arokem

This comment has been minimized.

Member

arokem commented Sep 18, 2016

One potential solution is to raise the minimal scipy version required. I am testing that out here:

arokem#28

@arokem

This comment has been minimized.

Member

arokem commented Sep 18, 2016

Yeah - looks like that should do the trick: https://travis-ci.org/arokem/dipy/jobs/160846233

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Sep 19, 2016

Should I change the travis.yml file and commit ?

Also, the test which fails is a test for noisy data. I am assuming that whenever fitting fails while using leastsq, or bounds are violated, the results for a simple linear fit is returned at least. I think that is a sensible thing to do rather than sending nan or clipping all values to 0.

@arokem

This comment has been minimized.

Member

arokem commented Sep 19, 2016

It's worth understanding this failure first. Does it always happen, or is it stochastic? If it is stochastic, under what conditions does it happen?

I agree that it might be reasonable to return the results of the linear fit for really messy data, but then this distinction should somehow be made, and the test should also reflect that, right?

See also discussion about this here: https://mail.python.org/pipermail/neuroimaging/2016-September/001175.html

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Sep 19, 2016

and the test should also reflect that, right?

test_leastsq_failing() reflects this. In recent Scipy versions the parameter values returned for noisy data after a linear fit are :
[4.35626890e+03 -1.97516012e-02 5.47370742e-04 6.93668428e-04]

while in the older Scipy versions, the params are :
[ 4.35626890e+03, 1.35121509e+02, 6.93404658e-04, 6.93668428e-04].

The linear fitting still gives either too high or negative values of f. Right now, I am checking for bounds only once, after the leastsq fitting and if bounds are violated the linear fit result are returned. We could perhaps skip this test for values of 'f' like in test_leastsq_failing(). I want the results of linear fit to be returned rather than setting to 0 or nan. Maybe we can have a default keyword argument fill_na = (False, False, False, False) which can be set to True to fill NaN's for violated parameters and left as False to return parameters as they are. (I had tried this as a separate function using np.where before)

@arokem

This comment has been minimized.

Member

arokem commented Sep 23, 2016

Sorry for being slow here. This all seems fine to me. So - is this ready?

Unless someone raises any issues that I am not considering, I can go ahead and merge this early next week.

@arokem arokem merged commit 2f333f3 into nipy:master Sep 27, 2016

4 checks passed

codecov/patch 92.00% of diff hit (target 80.99%)
Details
codecov/project 81.15% (+0.15%) compared to d0bee8c
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.1%) to 83.223%
Details
@arokem

This comment has been minimized.

Member

arokem commented Sep 27, 2016

Congratulations @sahmed95 ! You're GSoC 2016 contribution is now in master!

@etpeterson

This comment has been minimized.

Contributor

etpeterson commented Sep 27, 2016

Congrats @sahmed95!

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Sep 28, 2016

Thank you very much. Feels awesome to get my first open source project merged. Hopefully, this is the first of many to come ! :-)

@arokem

This comment has been minimized.

Member

arokem commented Sep 30, 2016

@sahmed95
@etpeterson
@RafaelNH

Are you interested in writing a short paper for JORS (http://openresearchsoftware.metajnl.com/) or JOSS (http://joss.theoj.org/)? JOSS is probably a bit easier (short!), given that we've already done a lot of the documentation here, but they still have to resolve whether they allow papers on PRs (see openjournals/joss#52). I'll ping them there again.

@arokem

This comment has been minimized.

Member

arokem commented Sep 30, 2016

I reread the discussion on the JOSS issue. It looks like they are OK with us submitting a PR as a paper. Should we start with that?

@etpeterson

This comment has been minimized.

Contributor

etpeterson commented Sep 30, 2016

That's a good idea. I'm in.

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Oct 1, 2016

Sounds great. I am in too. 👍 JOSS sounds nice and short.
I would also not mind a more elaborate paper on JORS since we have discussed a lot of issues at length about performance (using optimize), dealing with noise, poor fitting and the two stage approach.

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Oct 6, 2016

@arokem @RafaelNH @etpeterson. Please have a look. I might need to update your affiliations and orcid's. I created a new ID for myself.

@etpeterson

This comment has been minimized.

Contributor

etpeterson commented Oct 12, 2016

I got through the waitlist on the GSoC mentor summit, so I'm planning to go to that in a little over 2 weeks! I've never been, so if anyone (@arokem?) has I'd be interested in hearing about it.

While I'm there is there anything I should mention/suggest/tout? There are lightning talks (~3 mins each) about projects, so @sahmed95 if you want I could probably present that for you if you want to create one. Here are the talks for last year for reference: https://drive.google.com/drive/folders/0B-c5jwGzSQyPOHZyUVdNV2hFRUk

Also, if I'm forgetting anything then let me know.

@arokem

This comment has been minimized.

Member

arokem commented Oct 12, 2016

Cool! I haven't been. Please do report back about your experience!

On Wed, Oct 12, 2016 at 9:32 AM, Eric Peterson notifications@github.com
wrote:

I got through the waitlist on the GSoC mentor summit, so I'm planning to
go to that in a little over 2 weeks! I've never been, so if anyone (
@arokem https://github.com/arokem?) has I'd be interested in hearing
about it.

While I'm there is there anything I should mention/suggest/tout? There are
lightning talks (~3 mins each) about projects, so @sahmed95
https://github.com/sahmed95 if you want I could probably present that
for you if you want to create one. Here are the talks for last year for
reference: https://drive.google.com/drive/folders/0B-
c5jwGzSQyPOHZyUVdNV2hFRUk

Also, if I'm forgetting anything then let me know.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1110 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAHPNhOYulM6yUb-RUZrOQuZAu2FKq93ks5qzQuMgaJpZM4Jkuqk
.

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Oct 12, 2016

@etpeterson That's great ! You could talk about the fitting method we implemented and the profiling of optimize vs leastsq for performance. Do you want me to prepare a small talk, introducing the IVIM model in a simple way and discussing the issues we faced while implementing the fitting ?

@etpeterson

This comment has been minimized.

Contributor

etpeterson commented Oct 13, 2016

I took another look at the lightning talks and it looks like it's typically a group reporting on a few projects as a whole. Maybe that would be something to do for Dipy as a group?

@arokem

This comment has been minimized.

Member

arokem commented Oct 13, 2016

I think that would be great. We had so many nice projects this summer, on a
whole range of stuff, so it would be good to show-case that.

@Garyfallidis: where do the other projects stand? Would it be useful if
Eric reported on all the projects at the GSoC summit? Maybe we can ask each
one of the students to make a single slide summarizing what they did? We
can also share an overview slide with you, so that you can open with that.

On Thu, Oct 13, 2016 at 1:14 PM, Eric Peterson notifications@github.com
wrote:

I took another look at the lightning talks and it looks like it's
typically a group reporting on a few projects as a whole. Maybe that would
be something to do for Dipy as a group?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1110 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAHPNr5GkTQTtqPGfQ5oucPyWEEtbxlRks5qzpE2gaJpZM4Jkuqk
.

@etpeterson

This comment has been minimized.

Contributor

etpeterson commented Oct 19, 2016

Just bringing this back to the top. Any interest in putting something together for me to present? Don't feel obliged, but I'm happy to do it if you want.

@etpeterson

This comment has been minimized.

Contributor

etpeterson commented Oct 31, 2016

The mentor summit was awesome. It was basically all weekend and was all about various aspects of FOSS and basically whatever people wanted to organize. It is all about discussing and talking with people, really a summit not a conference with set proceedings.

Also, if you get a chance, I'd definitely try to do a lightning talk. Just a really quick way to tout our work. Dipy could do a really cool one just by showing images, most of the talks are about adding features or support or something and are good but not showy. And a few talks mentioned projects from previous years so pretty much anything goes as long as you don't go over time :-)

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Jul 18, 2017

Hi, my poster - Make your code count : Writing open source code for science and research(en) got selected for PyCon Japan 2017 and I will be using what I learnt on this project to make a case for more open source contributions in scientific computing.

https://pycon.jp/2017/en/proposals/vote/166/

@etpeterson (responding very late perhaps) lightning talks seem like a good idea. I will try to find some events where I could do so. I was just so caught up with college. Finally I have the time to do all these stuff. :-)

@etpeterson

This comment has been minimized.

Contributor

etpeterson commented Jul 18, 2017

Congrats! That should be fun and interesting. Hopefully it's not all in Japanese :-)

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Jul 18, 2017

Nah, English. I got lucky they have English options. But I think you can't git in Japanese anyway :). Just joined a group here near Tokyo at Riken. Members are European / British / Chinese / Swiss. Japanese is very difficult. Good thing is people here are interested in learning about open source scientific computing and how to contribute.

@arokem

This comment has been minimized.

Member

arokem commented Jul 19, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment