Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move the location of fast_exp to help compilation with strange flags #3883

Closed
wants to merge 3 commits into from

Conversation

hmaarrfk
Copy link
Member

@hmaarrfk hmaarrfk commented May 1, 2019

Description

So I rewrote fast_exp as a cython function
#3866

not sure if this is any better or any worse than the previous impelmentation (in terms of speed). This is faster by a good factor of 5. Could be really important as mentioned below for large datasets.

Personally, I think it is some kind of micro-optimization, but I can see where this might be actually helpful consider the restricted range of a uint8 image

Will need to add a benchmark, not too thrilled about doing that at the moment, but I wanted to give somebody a patch to get this working on their "unsupported" architecture.

Checklist

For reviewers

  • Check that the PR title is short, concise, and will make sense 1 year
    later.
  • Check that new functions are imported in corresponding __init__.py.
  • Check that new features, API changes, and deprecations are mentioned in
    doc/release/release_dev.rst.
  • Consider backporting the PR with @meeseeksdev backport to v0.14.x

@stefanv
Copy link
Member

stefanv commented May 1, 2019 via email

@hmaarrfk
Copy link
Member Author

hmaarrfk commented May 1, 2019

The counter argument there is that often innovation comes from those who need it.
numpy is mostly interested in "double precision accuracy" (as things are naturally up-cast to doubles). Image data is more focused on "small numbers from 0-255" and as such, shortcuts like this might be worthwhile, if the assumptions are justified.

often innovation comes from those who need it. I don't want to inhibit innovation. But I also thing that "speedups" should be quantified and understand that they are very compiler dependent.

I don't think it is necessarily wrong to implement such things like this here. I think maybe we can move it to the one file that needs it. I'm getting headaches from pyd and pyx.....

The primary goal of this PR is to ensure we are compatible with as many "compiler options" as possible, as demonstrated by a user running Alpine linux which seems to disable some optimization that would otherwise have respected the inline suggestion.

@hmaarrfk
Copy link
Member Author

hmaarrfk commented May 5, 2019

Ok, the approximation is pretty cool:

image

I think we should keep it, maybe just move it into the file that needs it to simplify the programming model.

@hmaarrfk
Copy link
Member Author

hmaarrfk commented May 5, 2019

Looking into things differently, it seems the error can be as high as 4% in certain cases. That isn't too encouraging.

image

@hmaarrfk hmaarrfk changed the title convert fast_exp to a cython function remove fast_exp in favour of numpy.exp May 5, 2019
@hmaarrfk hmaarrfk changed the title remove fast_exp in favour of numpy.exp WIP: remove fast_exp in favour of numpy.exp May 5, 2019
@hmaarrfk
Copy link
Member Author

hmaarrfk commented May 5, 2019

@stefanv see the commit message
57c042d#diff-917fcc21c44771a2336c8559bacbfa74

Further more the discussion here
#2878

States that we should probably keep the fast_exp function.

I think folding fast_exp into the nl_means pyx is probably the best way to ensure correct compilation while respecting the author's contributions.

I'll work on that later maybe.

@hmaarrfk
Copy link
Member Author

hmaarrfk commented May 5, 2019

@blochl i totally didn't see your great PR a while back.

I think one the scikit-image policies has been not to include the author names in specific files but rather in the CONTRIBUTORS.txt file. I added a little blurb about your contributation.

Mostly I moved this inside the nl_means so that it would work with some strange compilation options.

Need to test this on an alpine linux build. But I mostly wanted to let you know that your work wasn't really getting erased.

@hmaarrfk hmaarrfk changed the title WIP: remove fast_exp in favour of numpy.exp move the location of fast_exp to help compilation with strange flags May 5, 2019
@hmaarrfk hmaarrfk added this to the 0.16 milestone May 6, 2019

- Leonid Bloch
Implement non-local means restoration.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought NL-means was implemented by @emmanuelle? And git blame corroborates my memory...

Copy link
Contributor

@grlee77 grlee77 Sep 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, Emma contributed the non-local means function and myself and others have made some improvements over the years.

I would describe the contribution here as "Improvements to the performance and stability of non-local means restoration."

@jni
Copy link
Member

jni commented Sep 17, 2019

@hmaarrfk @stefanv I'd favour including this, since NL-means is an intrinsically slow technique, so a speedup of this magnitude can have a major impact. Having said this, the contributor notes are now incorrect (afaict, correct me if I'm wrong), so that needs fixing.

@jni
Copy link
Member

jni commented Sep 17, 2019

@hmaarrfk would you be able to add a benchmark for NLmeans and report on the results from this PR?

@stefanv
Copy link
Member

stefanv commented Nov 19, 2019

Apart from the attribution, this looks fine. Another, possibly simpler, solution, would be to leave the .h file alone, and to add a .pxd that defines the inline function. Either way, it is fine with me.

@sciunto sciunto modified the milestones: 0.16, 0.18 May 2, 2020
@jni
Copy link
Member

jni commented Oct 27, 2020

@hmaarrfk I tried to rebase it, but I got confused about whether these changes actually got partly merged somewhere else, or how they would interact with np_floats. Would you like to rescue this PR or should we close it?

@jni
Copy link
Member

jni commented Nov 26, 2020

Ping @hmaarrfk

I tried to rebase it, but I got confused about whether these changes actually got partly merged somewhere else, or how they would interact with np_floats. Would you like to rescue this PR or should we close it?

Base automatically changed from master to main February 18, 2021 18:23
@grlee77
Copy link
Contributor

grlee77 commented Aug 21, 2021

closing as likely resolved by the changes in #4322. Please reopen if this is not the case.

@grlee77 grlee77 closed this Aug 21, 2021
@grlee77 grlee77 added the 🔧 type: Maintenance Refactoring and maintenance of internals label Aug 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🔧 type: Maintenance Refactoring and maintenance of internals
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants