Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modern plugins (e.g. Fractal Dimension) fail with AIOOB exception on large data #163

Closed
mdoube opened this issue Dec 17, 2018 · 9 comments

Comments

4 participants
@mdoube
Copy link
Contributor

commented Dec 17, 2018

Describe the bug
Fractal Dimension fails with an AIOOB exception when running on large (but not that huge) stacks. Seems to be having a problem converting between data structures.

The original bug was filed against Fractal Dimension but it will affect any other Modern plugin that uses the conversion convenience method.

To Reproduce
Steps to reproduce the behavior:

  1. Make a new image, 1360 x 1360 x 2293 pixels, black, 8-bit
  2. Run Fractal Dimension, Automatic settings

Expected behavior
Fractal dimension finishes and writes results to shared results table

Environment

  • OS: Ubuntu, Mac

Additional context

[Mon Dec 17 13:13:20 HKT 2018] [ERROR] [] Module threw exception
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: -2068372576
	at net.imagej.ops.thread.chunker.DefaultChunker.run(DefaultChunker.java:103)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at net.imagej.ops.OpEnvironment.run(OpEnvironment.java:950)
	at net.imagej.ops.OpEnvironment.run(OpEnvironment.java:157)
	at net.imagej.ops.map.MapUnaryComputers$IIToIIParallel.compute(MapUnaryComputers.java:101)
	at net.imagej.ops.map.MapUnaryComputers$IIToIIParallel.compute(MapUnaryComputers.java:87)
	at net.imagej.ops.special.chain.UHCFViaUC.compute(UHCFViaUC.java:65)
	at net.imagej.ops.special.hybrid.UnaryHybridCF.calculate(UnaryHybridCF.java:61)
	at net.imagej.ops.special.hybrid.UnaryHybridCF.run(UnaryHybridCF.java:71)
	at net.imagej.ops.special.hybrid.UnaryHybridCF.run(UnaryHybridCF.java:97)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at net.imagej.ops.OpEnvironment.run(OpEnvironment.java:950)
	at net.imagej.ops.OpEnvironment.run(OpEnvironment.java:157)
	at net.imagej.ops.convert.ConvertNamespace.bit(ConvertNamespace.java:132)
	at org.bonej.wrapperPlugins.wrapperUtils.Common.toBitTypeImgPlus(Common.java:80)
	at org.bonej.wrapperPlugins.FractalDimensionWrapper.run(FractalDimensionWrapper.java:172)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService$3.call(DefaultThreadService.java:238)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: -2068372576
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at net.imagej.ops.thread.chunker.DefaultChunker.run(DefaultChunker.java:97)
	... 24 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: -2068372576
	at net.imglib2.img.basictypeaccess.array.AbstractByteArray.getValue(AbstractByteArray.java:61)
	at net.imglib2.type.numeric.integer.GenericByteType.getByte(GenericByteType.java:127)
	at net.imglib2.type.numeric.integer.UnsignedByteType.get(UnsignedByteType.java:151)
	at net.imglib2.type.numeric.integer.UnsignedByteType.getIntegerLong(UnsignedByteType.java:168)
	at net.imagej.ops.convert.ConvertTypes$IntegerToBit.compute(ConvertTypes.java:102)
	at net.imagej.ops.convert.ConvertTypes$IntegerToBit.compute(ConvertTypes.java:89)
	at net.imagej.ops.map.Maps.map(Maps.java:297)
	at net.imagej.ops.map.MapUnaryComputers$IIToIIParallel$1.execute(MapUnaryComputers.java:106)
	at net.imagej.ops.thread.chunker.DefaultChunker$1.run(DefaultChunker.java:77)
	at org.scijava.thread.DefaultThreadService$2.run(DefaultThreadService.java:221)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	... 4 more

Reported by @LMAnovitz on the mailing list

@mdoube

This comment has been minimized.

Copy link
Contributor Author

commented Dec 17, 2018

final Img<BitType> convertedImg = ops.convert().bit(imgPlus.getImg());

@mdoube

This comment has been minimized.

Copy link
Contributor Author

commented Dec 18, 2018

Images with dimension 1024×1024×2048 finish properly but 1024×1024×4096 cause the same AIOOB exception

@LMAnovitz

This comment has been minimized.

Copy link

commented Dec 18, 2018

a 1206 x 1206 x 2293 image also fails with the same problem

@LMAnovitz

This comment has been minimized.

Copy link

commented Dec 18, 2018

1024 x 1024 x 2293 also fails (and loses part of the data)

@mdoube

This comment has been minimized.

Copy link
Contributor Author

commented Jan 2, 2019

Thanks @LMAnovitz could you try with fewer slices? It may help us to understand which of x, y, and/or z dimensions are important. Is it an absolute number of pixels in the image, or a single dimension getting too big, or an unhandled combination of values?

@mdoube mdoube changed the title Fractal Dimension fails with AIOOB exception on large data Modern plugins (e.g. Fractal Dimension) fail with AIOOB exception on large data Jan 2, 2019

@LMAnovitz

This comment has been minimized.

Copy link

commented Jan 2, 2019

@ctrueden

This comment has been minimized.

Copy link
Member

commented Jan 3, 2019

I think @bnorthan has a tentative fix for this bug; see imagej/imagej-ops#589. Still in code review.

will try 2048 slices later today

@LMAnovitz I don't think it's necessary; @mdoube has posted a nice recipe to reproduce above.

@rimadoma

This comment has been minimized.

Copy link
Member

commented Jan 3, 2019

I agree with @ctrueden. The fix should also solve issue #106. Even so, it might still be worth using a ConvertService in 'Common. convertToBitImgPlus' instead of explicitly calling a specific conversion op.

@mdoube

This comment has been minimized.

Copy link
Contributor Author

commented Jan 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.