Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed test_centerline crashing due to bump in Python library #2754

Merged
merged 8 commits into from
Jun 25, 2020

Conversation

jcohenadad
Copy link
Member

@jcohenadad jcohenadad commented Jun 24, 2020

Since recently, test_get_centerline is failing on Travis. This is probably related to an upgrade of a Python library and/or platform-dependent (as noted in #2686)

This PR:

  • Replaced numpy.polyfit by numpy.polynomial.Polynomial as per numpy's doc, in centerline.curve_fitting.polyfit_1d()
  • Removed: test_get_centerline_nurbs[img_ctl5-expected5-params5] due to a ReconstructionError caused by the change in the dummy centerline generation.

Fixes #2753

@jcohenadad
Copy link
Member Author

jcohenadad commented Jun 24, 2020

Cannot reproduce failed test on a456fa5 on an Ubuntu 16.04 (bireli). Note: Python's libraries installed are the same (visual comparison from sct_check_dependencies)

Cannot reproduce on Docker 18.04 (all tests pass).
Cannot reproduce on proper 18.04 (rosenberg).

@jcohenadad
Copy link
Member Author

jcohenadad commented Jun 24, 2020

This issue is likely related to #2686.

Instead of relaxing (again) the threshold, maybe the proper approach would be to remove the tests where fitting is badly conditioned (ie very few points). Currently the Travis tests fail for these cases:

test_get_centerline_polyfit[img_ctl3-expected3-params3] → size_arr=(9, 9, 9), subsampling=3
test_get_centerline_polyfit[img_ctl5-expected5-params5] → size_arr=(30, 20, 50), subsampling=1 (surprising! this is not so ill-defined...)
test_get_centerline_bspline[img_ctl2-expected2-params2] → size_arr=(41, 7, 9)
test_get_centerline_nurbs[img_ctl5-expected5-params5] → size_arr=(30, 20, 50), subsampling=1 (not ill-defined either...)

The following warning suggests this is indeed an issue:

unit_testing/test_centerline.py:61
  /home/travis/build/neuropoly/spinalcordtoolbox/unit_testing/test_centerline.py:61: RankWarning: Polyfit may be poorly conditioned
    (dummy_centerline(size_arr=(30, 20, 50), subsampling=5),

@jcohenadad
Copy link
Member Author

jcohenadad commented Jun 24, 2020

Series of investigations on my OSX laptop:

Trying to replace numpy.polyfit with numpy.polynomial.Polynomial as per this suggestion, and numpy's doc:

The Polynomial.fit class method is recommended for new code as it is more stable numerically. See the documentation of the method for more information.

When replacing generation of dummy_centerline

=========================================================== 11 failed, 30 passed, 1257 warnings in 264.53s (0:04:24) ===========================================================

When using Polynomial also in centerline.core():

=========================================================== 11 failed, 30 passed, 1255 warnings in 250.04s (0:04:10) ===========================================================

@jcohenadad
Copy link
Member Author

Looks like polyfit might actually provide more regularized outputs when extrapolating at edges. Example below:

polyfit:
Screen Shot 2020-06-24 at 5 07 27 PM

Polynomial.fit:
Screen Shot 2020-06-24 at 5 07 10 PM

@jcohenadad
Copy link
Member Author

jcohenadad commented Jun 24, 2020

Given the extrapolation issue caused by polynomial fitting, it would probably be wise to replace a poly fit by a bspline fit for the ground-truth centerline.

However, we should keep the suggested Polynomial.fit()

@jcohenadad jcohenadad added sct_get_centerline context: tests context: unit, integration, or functional tests labels Jun 25, 2020
@jcohenadad jcohenadad added this to the 4.3.1 milestone Jun 25, 2020
@charleygros charleygros self-requested a review June 25, 2020 04:03
Copy link
Member

@charleygros charleygros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me and investigations make sense.
Tests all green on my local machine.
Is there anything in particular you would like me to test / look at? Otherwise it looks ready to be merged!

@@ -116,7 +116,7 @@ def get_parser():
metavar=Metavar.float,
type=float,
help='Size of the output FOV in the RL/AP plane, in mm. The resolution of the destination '
'image is the same as that of the source image (-i).',
'image is the same as that of the source image (-i). Default: 35.',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be relevant to display the default value by default in order to reduce maintenance etc?
eg https://stackoverflow.com/a/12151325/13306686
If so, I will open a new issue

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that would be awesome!

Comment on lines -26 to +25
p = poly1d(polyfit(x, y, deg=deg))
p = np.polynomial.Polynomial.fit(x, y, deg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is actually specified in np.polyfit documentation:
The Polynomial.fit class method is recommended for new code as it is more stable numerically.

Makes sense :-)

@jcohenadad jcohenadad merged commit dce5780 into master Jun 25, 2020
@jcohenadad jcohenadad deleted the jca/2753-test-centerline branch June 25, 2020 04:26
Drulex pushed a commit to Drulex/spinalcordtoolbox that referenced this pull request Sep 30, 2020
…ordtoolbox#2754)

* Removed nurbs dummy centerline (was not used)

* Changed output formatting for results

* Changed ().I for np.linalg.pinv to address Singular mat error

* Display default parameters

* Replaced polyfit by Polynomial.fit

* Improved clarity of saved files for debugging

* Excluded irrelevant or buggy tests

* Replaced polyfit by bspline in dummy centerline creation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sct_get_centerline context: tests context: unit, integration, or functional tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failing test_get_centerline
2 participants