Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Reducing memory overhead of image plots #41

Merged
merged 5 commits into from Jun 13, 2012

Conversation

Projects
None yet
4 participants
Member

cfarrow commented Feb 7, 2012

When plotting a 4096x4096 float image, memory usage approaches 2GB. (This can be tested by running examples/demo/basic/cmap_image_select.py with a 4096x4096 array). I've streamlined much of the image rendering code to reuse arrays, and I've cythonized pieces where this is not possible. This change introduces a new cython module for speedups - _cython_speedups.pyx. As a result, memory usage is now capped around 500MB and the cmap_image_select example is much more responsive for large images.

Why not just use isnan() from math.h here? You could call it in the loop (it's not a complicated function) and save a big chunk of memory.

Owner

cfarrow replied Feb 7, 2012

This call isn't using any additional memory due to the slice on line 65, but it is unnecessary, as you point out. I'll make the change.

Collaborator

jwiggins commented Feb 7, 2012

Overall this looks good. I would recommend you NOT add Cython output to the repository. Cython is already a dependency for Enable, so don't be shy about making it a dependency for Chaco as well. Presumably the older _speedups module should be cythonized as well (though not as part of this pull request).

Member

cfarrow commented Feb 7, 2012

I have had conflicting advice about whether the generated c-file should be in the repo. I'm going to leave it as is until I get more feedback. (In other words, a tie-breaking opinion.)

Member

jdmarch commented Feb 21, 2012

+corranwebster emphatically "Include the generated C code, since we can't yet assume that the whole world has cython". Makes sense to me.

nmichaud commented May 7, 2012

What's the status of this?

Member

cfarrow commented May 7, 2012

It's just waiting final approval. Any objections to pushing this through?

nmichaud commented May 7, 2012

Nope, it looks good to me. Anybody else have objections?

On Mon, May 7, 2012 at 1:27 PM, Chris Farrow <
reply@reply.github.com

wrote:

It's just waiting final approval. Any objections to pushing this through?


Reply to this email directly or view it on GitHub:
#41 (comment)

I was wondering how much this speeds up the image processing?

Owner

cfarrow replied May 10, 2012

The entire pull request is more about memory usage than speed. I didn't do any timings. The last change should speed things up, but I didn't time that either. (Someone is free to do that. I won't have time for it until next week.)

@nmichaud nmichaud pushed a commit that referenced this pull request Jun 13, 2012

nmichaud Merge pull request #41 from cfarrow/efficient_image
Reducing memory overhead of image plots
12f7c1c

@nmichaud nmichaud merged commit 12f7c1c into enthought:master Jun 13, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment