Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generic_filter slow on large images #8916

Open
cathoderaymission opened this issue Jun 9, 2018 · 6 comments
Open

generic_filter slow on large images #8916

cathoderaymission opened this issue Jun 9, 2018 · 6 comments

Comments

@cathoderaymission
Copy link

I am using the generic_filter function to compute an image quality assessment metric, the image contrast, per the following:

def fnc(buffer): np.sum(np.abs(buffer[:]-buffer[4]))

np.mean(generic_filter(img, fnc, 3))

While this works beautifully, and scipy is awesome being the only package to provide the ability to attach a function to a filter, it is incredibly slow on over 2k x 2k images.

As I can't find any documentation about speeding this up, I'm filing this in hopes their is some possible means of performance increase in a future release.

@jni
Copy link
Contributor

jni commented Jun 9, 2018

Cross posting from the scikit-image repo in case others stumble upon this issue. I actually wrote a blog post about exactly this issue a year ago:

https://ilovesymposia.com/2017/03/12/scipys-new-lowlevelcallable-is-a-game-changer/

There is also a follow-up:

https://ilovesymposia.com/2017/03/15/prettier-lowlevelcallables-with-numba-jit-and-decorators/

I hope you find these useful!

The summary: calling a function in Python is expensive but combining LowLevelCallable and Numba (or Cython) you can bypass this expense and get very fast speeds with generic_filter.

@ev-br
Copy link
Member

ev-br commented Jun 17, 2018

Would you be interested in turning your example to e.g. scipy tutorial pages, @jni?

@jni
Copy link
Contributor

jni commented Jun 18, 2018

@ev-br I would but don't have the bandwidth until after SciPy 2018 at the earliest. If anyone else wants to pick it up, I would be grateful! All content on my blog is CC-BY:

https://ilovesymposia.com/about/

@szymonmaszke
Copy link

Just dropping by to say you can use your own functions decorated with numba.njit (no-python mode) and it becomes much, much faster.

There is no need for jit_filter_function described by @jni in his last linked blog post anymore (though more rigorous testing would be needed to be actually sure).

@jni
Copy link
Contributor

jni commented Mar 11, 2019

@szymonmaszke

  1. WHOA, that's awesome! When did that happen?
  2. But, is the required function signature the same as the Python signature, or the same as the LLC signature?

@szymonmaszke
Copy link

@jni

  1. No clue, used this scipy functionality for the first time. You can see an example bersen thresholding function I have implemented with njit and check for yourself gist here.
  2. Required signature is the one compatible with numba (or unspecified as can be seen in gist), no need for the LLC if I understand you correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants