Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object deblending overflow #51

Closed
hackermd opened this issue Feb 24, 2017 · 10 comments
Closed

Object deblending overflow #51

hackermd opened this issue Feb 24, 2017 · 10 comments

Comments

@hackermd
Copy link

For some images, I get the following error message when running sep.extract():

object deblending overflow: limit of 1024 sub-objects reached while deblending. Decrease number of deblending thresholds or increase the detection threshold.
...
  File "sep.pyx", line 718, in sep.extract (sep.c:6652)
  File "sep.pyx", line 282, in sep._assert_ok (sep.c:3522)

I've tried decreasing deblend_nthresh, but keep getting the error until I decrease it to deplend_nthresh=1.

Is there another way to increase the limit without having to increase the detection threshold?

@kboone
Copy link
Contributor

kboone commented Feb 24, 2017

Have you subtracted the background from the image? I've run into this issue before when the background level is higher than the detection threshold.

If that isn't the issue, can you share the image so that we can take a look?

@hackermd
Copy link
Author

hackermd commented Mar 2, 2017

@kboone thank you for your immediate feedback!

Have you subtracted the background from the image?

Yes, but I simply followed the instructions in the tutorial using the default settings for background estimation:

import cv2
import sep

sep.set_extract_pixstack(10**7)

filename = ''
threshold = 3

img = cv2.imread(filename, cv2.IMREAD_UNCHANGED).as_type('float')

bkg = sep.Background(img)

img_sub = img - bkg

out, blobs_img = sep.extract(
    img_sub, threshold, err=bkg.globalrms, segmentation_map=True
)

I've run into this issue before when the background level is higher than the detection threshold.

The error doesn't occur when I use err=bkg.rms() instead of err=bkg.globalrms.

can you share the image so that we can take a look?

Please find a small region of the image attached (500x500, full image: 2160x2560). I've started to play with the size of the background boxes and filter kernels and start to get satisfactory results. Will need to have a closer look at the SEXtractor manual. Suggestions for improving background estimation would also be very welcome.

image

@kbarbary
Copy link
Owner

kbarbary commented Mar 2, 2017

I couldn't reproduce the error:

In [3]: img = cv2.imread("desktop/db9a1a28-ff73-11e6-8bb6-e68874500515.png", cv2.IMREAD_UNCHANGED).astype('float')

In [7]: bkg = sep.Background(img)

In [11]: img_sub = img - bkg

In [12]: out, blobs_img = sep.extract(img_sub, 3, err=bkg.globalrms, segmentation_map=True)

In [14]: len(out)
Out[14]: 498

Can you reproduce the error on this sub-image, or do you need the full image for the error to occur?

@kboone
Copy link
Contributor

kboone commented Mar 2, 2017

I can't reproduce the error either on this sub-image. I do notice that bkg.rms() is negative in the upper left corner though. @kbarbary have you seen that before? I think that effect is a spline interpolation issue but it might be unrelated to what @hackermd is running into.

@kbarbary
Copy link
Owner

kbarbary commented Mar 2, 2017

I think that is a spline interpolation issue.

Seems likely. This would probably cause a very big blobby object when using err=bkg.rms(). Using err=bkg.globalrms should avoid that problem though.

@hackermd
Copy link
Author

hackermd commented Mar 2, 2017

Can you reproduce the error on this sub-image, or do you need the full image for the error to occur?

It's not reproducible on the sub-image. I've attached the full image as well.

fish

@kbarbary
Copy link
Owner

kbarbary commented Mar 2, 2017

What is this image of?? ;)

image

Zoom in:

image

I think the detection threshold is too low. bkg.globalrms is about 3.5, so the threshold is about 10.5, which is very small compared to the features in the image. Therefore, you're getting one (or a few) giant objects, and then the deblending algorithm is trying to separate out all the little blobs. Setting a higher initial threshold should help.

SEP / SExtractor can still work on this image, but the defaults (and assumptions in the code) are set for wide-field optical/IR imaging. Those defaults and assumptions don't make sense for this image, so you'll need to be more careful and tweak a few things. For example, the background estimation algorithm in SEP / SExtractor basically assumes that the majority of pixels in any region are background. That doesn't seem to be the case here, so you may need a custom background estimation (or to be careful about what you define as "background" anyway).

@hackermd
Copy link
Author

What is this image of?? ;)

It's a microscopic image showing gene transcripts in human cells visualized via fluorescent dyes.

@hackermd
Copy link
Author

Using small BACK_SIZE and BACK_FILTERSIZE for background estimation improves results and doesn't cause deblending overflow:

bkg = sep.Background(img, bw=3, bh=3, fw=2, fh=2)
img_sub = img - bkg
out, blobs_img = sep.extract(img_sub, 5, err=bkg.globalrms, minarea=3, segmentation_map=True)

Thanks for your support!

@kbarbary
Copy link
Owner

👍 Glad you got it to work for you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants