New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate replacing Cython with numba #169

Open
leouieda opened this Issue Feb 25, 2015 · 4 comments

Comments

Projects
None yet
2 participants
@leouieda
Member

leouieda commented Feb 25, 2015

Numba has come a long way since I last played with it. Anaconda has made installing numba a non-issue and we already recommend it as the preferred way of getting dependencies for Fatiando.

I think it's time to investigate replacing some of our Cython code with numba. The usage is so much simpler and would greatly reduce the barrier to contributing high-performance code. It would also remove the compile step from installing Fatiando. This is usually the cause of bug reports for Windows users.

For the change to work, the target code should be covered by tests and benchmarked.

@eusoubrasileiro

This comment has been minimized.

Contributor

eusoubrasileiro commented Feb 27, 2015

@leouieda I saw the problem with cython in windows and I see your concern. But although not perfect cython parallel makes wavefd simulation code viable for more reasonable grid sizes (the very ones we want to play to compare with real seismic). So I want to follow this.

@leouieda

This comment has been minimized.

Member

leouieda commented Feb 27, 2015

@eusoubrasileiro the idea is not to have the code slower, but see if numba
can match the speeds we have. If it does, I think it would be worth the
effort of switch. It's a lot easier to use.
On Feb 27, 2015 6:31 PM, "André" notifications@github.com wrote:

@leouieda https://github.com/leouieda I saw the problem with cython in
windows and I see your concern. But although not perfect cython makes
wavefd simulation code viable for more reasonable grid sizes (the very
ones we want to play to compare with real seismic). So I want to follow
this closely.


Reply to this email directly or view it on GitHub
#169 (comment).

@leouieda leouieda added the discussion label Mar 6, 2015

@leouieda

This comment has been minimized.

Member

leouieda commented Mar 11, 2015

Made a prototype implementation of tesseroid forward modeling using numba and a pure Python version: http://nbviewer.ipython.org/gist/leouieda/f2824b0062a4446a6e24

  • Current Cython version: 0.252985 s
  • Pure Python: 13.533715 s
  • Numba: 0.214360 s
  • Multiprocessing parallel by splitting computation grid (4 jobs): 0.119183 s

Looks promising! And much easier to implement.

A problem I had: can't pass callbacks to numba.jit'ed functions. Graham Markall suggested using closures:
https://groups.google.com/a/continuum.io/forum/#!msg/numba-users/KTZxkOl2mK8/_V3HCj2jH3gJ

@leouieda

This comment has been minimized.

Member

leouieda commented Mar 11, 2015

So far getting slower execution with closures. Maybe numba was inlining the compiled kernels before.

Update: This was probably bad coding from my part. @gmarkall fixed this and now the code runs just as fast with closures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment