PrimitiveBlocks copies RandomAccessible blocks to primitive arrays #330
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds functionality to extract blocks from
RandomAccessible
into flat primitive arrays.This will be useful for example to copy data to
Tensor
, to extract blocks for storing into N5, for interfacing CLIJ, for running small algorithm kernels directly on primitive arrays, etc...The only public API introduced is the interface
PrimitiveBlocks<T extends NativeType<T>>
.The static method
PrimitiveBlocks.of(...)
creates aPrimitiveBlocks
accessor for an arbitraryRandomAccessible
source.The method
PrimitiveBlocks.copy(long[] srcPos, Object dest, int[]size)
is then used to copy blocks out of the source into flat primitive arrays. The idea is to provide an interface similar toSystem.arraycopy
.Object dest
is a primitive array of type corresponding toT
.PrimitiveBlocks.of(...)
understands a lot ofView
constructions (that ultimately end in CellImg, ArrayImg, etc) and will try to create an optimized copier. For example, the following will work:The idea of the optimized copier is:
Instead of using
RandomAccess
that checks for every pixel whether it enters a newCell
, whether it is out-of-bounds, etc., all these checks are precomputed and then relevant data from eachCell
is copied in one go.The speedup can be dramatic, in particular if the underlying source data is in a
CellImg
.Some benchmarks included, here is for example results of
CopyBenchmarkViewPrimitiveBlocks
If a source
RandomAccessible
cannot be understood,PrimitiveBlocks.of(...)
will return a fall-back implementation (based onLoopBuilder
). With the optionalOnFallback
argument ofPrimitiveBlocks.of(...)
it can be configured whether fall-back should beACCEPT
),WARN
) -- the default,IllegalArgumentException
thrown (FAIL
).The warning/exception message explains why the source
RandomAccessible
requires fall-back.The only really un-supported case is if the pixel type
T
does not map one-to-one to a primitive type.For example,
ComplexDoubleType
orUnsigned4BitType
are not supported. (at least not yet).PrimitiveBlocks.copy
is single-threaded, the idea being to parallelize over blocks instead of the copying within a block.PrimitiveBlocks
is not thread-safe in general, but has a methodthreadSafe()
to obtain a thread-safe instance (implemented usingThreadLocal
copies). For example,can safely be used multi-threaded, for example in
CellLoader
s.