Can't open large volume

[kthorn]

When I try to open a 2560 x 2160 x 197 volume in ClearVolume I get the following error:

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Number of elements in Container too big, use for example CellContainer instead: 2178662400 > 2147483647
at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:277)
at net.imagej.legacy.DefaultLegacyHooks.interceptRunPlugIn(DefaultLegacyHooks.java:163)
at ij.IJ.runPlugIn(IJ.java)
at ij.Executer.runCommand(Executer.java:137)
at ij.Executer.run(Executer.java:66)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Number of elements in Container too big, use for example CellContainer instead: 2178662400 > 2147483647
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:273)
... 5 more
Caused by: java.lang.RuntimeException: Number of elements in Container too big, use for example CellContainer instead: 2178662400 > 2147483647
at net.imglib2.img.array.ArrayImgFactory.numEntitiesRangeCheck(ArrayImgFactory.java:71)
at net.imglib2.img.array.ArrayImgFactory.createByteInstance(ArrayImgFactory.java:79)
at net.imglib2.img.array.ArrayImgFactory.createByteInstance(ArrayImgFactory.java:58)
at de.mpicbg.jug.clearvolume.ClearVolumeUnsignedShortType.createSuitableNativeImg(ClearVolumeUnsignedShortType.java:320)
at net.imglib2.img.array.ArrayImgFactory.create(ArrayImgFactory.java:63)
at de.mpicbg.jug.clearvolume.ImgLib2ClearVolume.makeClearVolumeUnsignedShortTypeCopy(ImgLib2ClearVolume.java:380)
at de.mpicbg.jug.clearvolume.ImgLib2ClearVolume.makeClearVolumeUnsignedShortTypeCopies(ImgLib2ClearVolume.java:354)
at de.mpicbg.jug.clearvolume.ImgLib2ClearVolume.initRealImgs(ImgLib2ClearVolume.java:446)
at de.mpicbg.jug.clearvolume.gui.ClearVolumeManager.run(ClearVolumeManager.java:182)
at de.mpicbg.jug.clearvolume.gui.GenericClearVolumeGui.launchClearVolumeManager(GenericClearVolumeGui.java:303)
at de.mpicbg.jug.clearvolume.gui.GenericClearVolumeGui.(GenericClearVolumeGui.java:160)
at de.mpicbg.jug.plugins.ClearVolumePlugin.run(ClearVolumePlugin.java:86)
at org.scijava.command.CommandModule.run(CommandModule.java:205)
at org.scijava.module.ModuleRunner.run(ModuleRunner.java:167)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:126)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:65)
at org.scijava.thread.DefaultThreadService$2.call(DefaultThreadService.java:191)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

If I crop the image to a smaller size in X and Y it opens fine.

[skalarproduktraum]

Hi @kthorn and thanks for the report!
This issue is already known and we're currently testing a fix.
The problem is that Java uses 32bit integers to index data arrays, so the capacity basically ends at 2 Gigabytes (Java does not have unsigned types, so integers cover -2^31 to 2^31).

The fix should be out by the end of next week. If you'd like to try a preliminary version, please let me know.

[bmoretti]

Hi! I'm having this same problem with a 239x2560x2160 stack that's over 2 GB in size. @skalarproduktraum, you said in March 24 that a fix for this issue was going out around that time, so I'm wondering if that actually happened or if it might have been delayed?

Thanks a lot for the support!

[royerloic]

Hi Bruno,

Unfortunately the problem goes very deep and has multiple aspects:
i) some limitations of Java make it difficult but not impossible to hadle >2G
ii) the GPUs have limited memory and many only support <2G
iii) for sizes of 2k x 2k x 2k the rendering would be quite slow even on recent GPUs

ClearVolume 2 is taking time to developp, it will be built on top of Scenery (@skalarproduktraum https://github.com/skalarproduktraum)
and will not have these limitations. Right now only @skalarproduktraum https://github.com/skalarproduktraum has time to work on this,
I just got a GL offer and my pipeline is full, and so I have to wait until I can get a student to work
on it to help Ulrik. Or spend two weeks hacking madly… but when ? :-(

In the meantime,
Here is what I suggest: downscale your data by a factor 2 in each direction to get
a stack of ~ 1k x 1k x1k.

Think about it, your screen has probably about 1000x2000 pixels, so
you cannot even see 2000 pixels across all directions on your screen
anyway. Moreover, your PSF probably is about 3x3x3 pixels so that also
limits the utility of having so many pixels.

A volume of 1000 x 1000 x 1000 is already a beautiful stack that will render
very well on ClearVolume if you have a very good graphics card (< 2 years)
with at least 4 gigas of GPU RAM. There will be no perceptual loss of quality,
because of the limitations induced by the PSF and screen resolution.
Yet, you will need 8 times less Video RAM to display this and it will be 8 times
faster…

We should probably add this automatic downscaling…

Hope this helps…

Cheers :-)

loic

On 07 Sep 2016, at 22:01, Bruno Moretti notifications@github.com wrote:

Hi! I'm having this same problem with a 239x2560x2160 stack that's over 2 GB in size. @skalarproduktraum https://github.com/skalarproduktraum, you said in March 24 that a fix for this issue was going out around that time, so I'm wondering if that actually happened or if it might have been delayed?

Thanks a lot for the support!

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub #46 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AByMksnjgwB7gHXg7wOVsgmGPr76eFoLks5qnxgqgaJpZM4H3VgV.

Dr. Loïc Alain Royer

Post-Doc - Myers Lab
Max Planck Institute
of Molecular Cell Biology
and Genetics
Pfotenhauerstr. 108
01307 Dresden

Too short? Here's why: http://emailcharter.org/