Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No bounds checking on fill/trace buf access #1

Open
Alex-Richman opened this issue Apr 27, 2019 · 5 comments · May be fixed by #2
Open

No bounds checking on fill/trace buf access #1

Alex-Richman opened this issue Apr 27, 2019 · 5 comments · May be fixed by #2

Comments

@Alex-Richman
Copy link

Alex-Richman commented Apr 27, 2019

I'm having some trouble processing big 3d volumes using the library. The cause seems to be a lack of bounds checking on buf access in fill() and trace(), buf being statically sized. Due to the numba.jit usage it manifests as a segfault due to the out-of-bounds write.

Here's a PR to add that bounds checking, and allows caller specification of the fixed buffer size to build_sknw():
#2

I guess it could also be dynamically resized, but I'm not familiar with numba.jit flavoured python so I'll leave that.

@Alex-Richman Alex-Richman linked a pull request Apr 27, 2019 that will close this issue
@yxdragon
Copy link
Member

I am sorry, so long time not watch this repo, your image is very large? are you sure the image is a skeleton image? I think 131072 is enough to contain a edge. maybe I should malloc 1/10 lenght of the 3d image. I think 1/10 size is enough for any edge.

numba cound not resize an array dynamically, we can only np.zeros(...) a longer one, and copy data in.

@Alex-Richman
Copy link
Author

Yeah the images we're looking at are very big, up to 1024x1024x1024 with lots of connectivity in the skeleton.

I don't have the maximum edge lengths we acquired to hand, but I imagine some factor of the total image size would work well as an initialiser for the edge buffer.

Main thing I needed to fix locally though was the breaks preventing overflowing of that edge buffer, since the buffer overrun in numba causes a SEGFAULT not a clean python exception.

Thanks!

  • Alex.

@yxdragon
Copy link
Member

yxdragon commented Jun 1, 2019

can you export your data as png sequence and email to me? I think 1024^3 is not large, 131042 = 1024 * 128. may be there is something wrong other where!

@victor000000
Copy link

131042 * 8(int64) = 1M
131042 *1024 * 8 = 1G

@victor000000
Copy link

I found that in build_sknw(ske, multi=False, iso=True, ring=True)
if ske is of type np.uint8, then if node num > 255 - 10, node_num will be 0, that is a bug
image

so if node num > 255, we shoud make sure ske is of type np.int64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants