Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sep.extract on large images #24

Closed
kboone opened this issue Nov 20, 2015 · 6 comments
Closed

sep.extract on large images #24

kboone opened this issue Nov 20, 2015 · 6 comments

Comments

@kboone
Copy link
Contributor

kboone commented Nov 20, 2015

Currently, there is a limit of 300000 active object pixels in sep.extract. For settings that have more pixels above the threshold than this, the following error is raised:

Exception: internal pixel buffer full: The limit of 300000 active object pixels over
the detection threshold was reached. Check that the image is background subtracted
and the detection threshold is not too low. detection threshold.

I am trying to run on some Subaru Suprime Cam data, where my image is 8864x10899 pixels, and this limit means that I am limited to having <0.3% of the pixels being active, which is way too low. Can this 300000 be adjusted? Why is it there in the first place? 300k is not a big number on a modern computer...

I'll dig into the code and try to figure this out...

@kboone
Copy link
Contributor Author

kboone commented Nov 20, 2015

And I found the answer pretty quick... there is a hidden sep.set_extract_pixstack() function to change this. I can see why you would want a low default limit, but it would be nice to change the error message to tell you about that hidden function.

@kboone
Copy link
Contributor Author

kboone commented Nov 20, 2015

I added a pull request (#25) which would make the exception message more intuitive, without changing the default threshold.

@kbarbary
Copy link
Owner

The sep.set_extract_pixstack() method should now be documented on the API page.

I'm a bit surprised that you're exceeding the limit with an image of that size though. Not all the objects in the image will be "active" at the same time, so saying "<0.3%" of pixels being active is a bit misleading.

@kboone
Copy link
Contributor Author

kboone commented Nov 20, 2015

I think that there are a bunch of bad pixels, ~1% of the image, which have similar values. Those are probably what was hitting the threshold since they are in big blocks. I could probably have masked them properly, but I was just trying to do image alignment where it doesn't matter.

@kbarbary
Copy link
Owner

Ah, yeah. If they were in big blocks, that could do it.

@kbarbary
Copy link
Owner

I assume we can close this now. Comment if there's still some improvement to be made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants