-
Notifications
You must be signed in to change notification settings - Fork 267
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Zarr reading and writing support #1474
Merged
Merged
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
2fa9b6c
Added possibility to open Zarr images
Rylern a7b184f
Fix accessing z-stack pyramidal images
Rylern d475f92
Update gradle/libs.versions.toml
Rylern 21b43e0
Improved fix accessing z-stack pyramidal images
Rylern d7fc43a
Improved fix accessing z-stack pyramidal images
Rylern 00d05ff
Addressed comments
Rylern 59ac700
Started zarr writer
Rylern c2528f0
Removed QuPath specific code from attributes
Rylern e8af370
Added Javadoc comments
Rylern 604e585
Added empty downsampled tile creator
Rylern 3b177d6
Added function to write entire image
Rylern c60cc86
Wait for writing to finish in close()
Rylern 8425109
Added support for RGB, set channel names/colors
Rylern a1af13c
Handle opening Zarr files by selecting the .zattrs or .zgroup file
Rylern e3718bc
Simplify OMEZarrWriter
Rylern a0ffcf7
Added possibility to define downsamples and max chunk size
Rylern a1e2172
Improved max number of chunks
Rylern 768c05c
Enforce max number of chunks
Rylern abe5c4e
Added tile size
Rylern e8f9666
Typo
Rylern 16ab826
Addressed comments
Rylern File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
236 changes: 236 additions & 0 deletions
236
...bioformats/src/main/java/qupath/lib/images/writers/ome/zarr/OMEZarrAttributesCreator.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,236 @@ | ||
package qupath.lib.images.writers.ome.zarr; | ||
|
||
import qupath.lib.common.ColorTools; | ||
import qupath.lib.images.servers.ImageChannel; | ||
import qupath.lib.images.servers.PixelType; | ||
|
||
import java.util.ArrayList; | ||
import java.util.HashMap; | ||
import java.util.List; | ||
import java.util.Map; | ||
import java.util.concurrent.TimeUnit; | ||
import java.util.stream.IntStream; | ||
|
||
/** | ||
* Create attributes of a OME-Zarr file as described by version 0.4 of the specifications of the | ||
* <a href="https://ngff.openmicroscopy.org/0.4/index.html">Next-generation file formats (NGFF)</a>. | ||
*/ | ||
class OMEZarrAttributesCreator { | ||
|
||
private static final String VERSION = "0.4"; | ||
private final String imageName; | ||
private final int numberOfZSlices; | ||
private final int numberOfTimePoints; | ||
private final int numberOfChannels; | ||
private final boolean pixelSizeInMicrometer; | ||
private final TimeUnit timeUnit; | ||
private final double[] downsamples; | ||
private final List<ImageChannel> channels; | ||
private final boolean isRGB; | ||
private final PixelType pixelType; | ||
private enum Dimension { | ||
X, | ||
Y, | ||
Z, | ||
C, | ||
T | ||
} | ||
|
||
/** | ||
* Create an instance of the attributes' creator. | ||
* | ||
* @param imageName the name of the image | ||
* @param numberOfZSlices the number of z-stacks | ||
* @param numberOfTimePoints the number of time points | ||
* @param numberOfChannels the number of channels | ||
* @param pixelSizeInMicrometer whether pixel sizes are in micrometer | ||
* @param timeUnit the unit of the time dimension of the image | ||
* @param downsamples the downsamples of the image | ||
* @param channels the channels of the image | ||
* @param isRGB whether the image stores pixel values with the RGB format | ||
* @param pixelType the type of the pixel values of the image | ||
*/ | ||
public OMEZarrAttributesCreator( | ||
String imageName, | ||
int numberOfZSlices, | ||
int numberOfTimePoints, | ||
int numberOfChannels, | ||
boolean pixelSizeInMicrometer, | ||
TimeUnit timeUnit, | ||
double[] downsamples, | ||
List<ImageChannel> channels, | ||
boolean isRGB, | ||
PixelType pixelType | ||
) { | ||
this.imageName = imageName; | ||
this.numberOfZSlices = numberOfZSlices; | ||
this.numberOfTimePoints = numberOfTimePoints; | ||
this.numberOfChannels = numberOfChannels; | ||
this.pixelSizeInMicrometer = pixelSizeInMicrometer; | ||
this.timeUnit = timeUnit; | ||
this.downsamples = downsamples; | ||
this.channels = channels; | ||
this.isRGB = isRGB; | ||
this.pixelType = pixelType; | ||
} | ||
|
||
/** | ||
* @return an unmodifiable map of attributes describing the zarr group that should | ||
* be at the root of the image files | ||
*/ | ||
public Map<String, Object> getGroupAttributes() { | ||
return Map.of( | ||
"multiscales", List.of(Map.of( | ||
"axes", getAxes(), | ||
"datasets", getDatasets(), | ||
"name", imageName, | ||
"version", VERSION | ||
)), | ||
"omero", Map.of( | ||
"name", imageName, | ||
"version", VERSION, | ||
"channels", getChannels(), | ||
"rdefs", Map.of( | ||
"defaultT", 0, | ||
"defaultZ", 0, | ||
"model", "color" | ||
) | ||
) | ||
); | ||
} | ||
|
||
/** | ||
* @return an unmodifiable map of attributes describing a zarr array corresponding to | ||
* a level of the image | ||
*/ | ||
public Map<String, Object> getLevelAttributes() { | ||
List<String> arrayDimensions = new ArrayList<>(); | ||
if (numberOfTimePoints > 1) { | ||
arrayDimensions.add("t"); | ||
} | ||
if (numberOfChannels > 1) { | ||
arrayDimensions.add("c"); | ||
} | ||
if (numberOfZSlices > 1) { | ||
arrayDimensions.add("z"); | ||
} | ||
arrayDimensions.add("y"); | ||
arrayDimensions.add("x"); | ||
|
||
return Map.of("_ARRAY_DIMENSIONS", arrayDimensions); | ||
} | ||
|
||
private List<Map<String, Object>> getAxes() { | ||
List<Map<String, Object>> axes = new ArrayList<>(); | ||
|
||
if (numberOfTimePoints > 1) { | ||
axes.add(getAxis(Dimension.T)); | ||
} | ||
if (numberOfChannels > 1) { | ||
axes.add(getAxis(Dimension.C)); | ||
} | ||
if (numberOfZSlices > 1) { | ||
axes.add(getAxis(Dimension.Z)); | ||
} | ||
axes.add(getAxis(Dimension.Y)); | ||
axes.add(getAxis(Dimension.X)); | ||
|
||
return axes; | ||
} | ||
|
||
private List<Map<String, Object>> getDatasets() { | ||
return IntStream.range(0, downsamples.length) | ||
.mapToObj(level -> Map.of( | ||
"path", "s" + level, | ||
"coordinateTransformations", List.of(getCoordinateTransformation((float) downsamples[level])) | ||
)) | ||
.toList(); | ||
} | ||
|
||
private List<Map<String, Object>> getChannels() { | ||
Object maxValue = isRGB ? Integer.MAX_VALUE : switch (pixelType) { | ||
case UINT8, INT8 -> Byte.MAX_VALUE; | ||
case UINT16, INT16 -> Short.MAX_VALUE; | ||
case UINT32, INT32 -> Integer.MAX_VALUE; | ||
case FLOAT32 -> Float.MAX_VALUE; | ||
case FLOAT64 -> Double.MAX_VALUE; | ||
}; | ||
|
||
return channels.stream() | ||
.map(channel -> Map.of( | ||
"active", true, | ||
"coefficient", 1d, | ||
"color", String.format( | ||
"%02X%02X%02X", | ||
ColorTools.unpackRGB(channel.getColor())[0], | ||
ColorTools.unpackRGB(channel.getColor())[1], | ||
ColorTools.unpackRGB(channel.getColor())[2] | ||
), | ||
"family", "linear", | ||
"inverted", false, | ||
"label", channel.getName(), | ||
"window", Map.of( | ||
"start", 0d, | ||
"end", maxValue, | ||
"min", 0d, | ||
"max", maxValue | ||
) | ||
)) | ||
.toList(); | ||
} | ||
|
||
private Map<String, Object> getAxis(Dimension dimension) { | ||
Map<String, Object> axis = new HashMap<>(); | ||
axis.put("name", switch (dimension) { | ||
case X -> "x"; | ||
case Y -> "y"; | ||
case Z -> "z"; | ||
case T -> "t"; | ||
case C -> "c"; | ||
}); | ||
axis.put("type", switch (dimension) { | ||
case X, Y, Z -> "space"; | ||
case T -> "time"; | ||
case C -> "channel"; | ||
}); | ||
|
||
switch (dimension) { | ||
case X, Y, Z -> { | ||
if (pixelSizeInMicrometer) { | ||
axis.put("unit", "micrometer"); | ||
} | ||
} | ||
case T -> axis.put("unit", switch (timeUnit) { | ||
case NANOSECONDS -> "nanosecond"; | ||
case MICROSECONDS -> "microsecond"; | ||
case MILLISECONDS -> "millisecond"; | ||
case SECONDS -> "second"; | ||
case MINUTES -> "minute"; | ||
case HOURS -> "hour"; | ||
case DAYS -> "day"; | ||
}); | ||
} | ||
|
||
return axis; | ||
} | ||
|
||
private Map<String, Object> getCoordinateTransformation(float downsample) { | ||
List<Float> scales = new ArrayList<>(); | ||
if (numberOfTimePoints > 1) { | ||
scales.add(1F); | ||
} | ||
if (numberOfChannels > 1) { | ||
scales.add(1F); | ||
} | ||
if (numberOfZSlices > 1) { | ||
scales.add(1F); | ||
} | ||
scales.add(downsample); | ||
scales.add(downsample); | ||
|
||
return Map.of( | ||
"type", "scale", | ||
"scale", scales | ||
); | ||
} | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be
round
orfloor
? My intuition saysround
but my intuition isn't certain...Also, it would be useful to look at the number of
z
slices for different levels in the sample images you have access to. It might be possible to compute the ratio between thez
slices for a level and the zero level, to determine how much to rescale thez
index (which might be more robust than using thedownsample
value, in case it's possible that the downsampling inz
uses a different factor).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to use
round
but I got an error whenz / tileRequest.getDownsample()
was giving something like 133.5, which turned into 134 after applyinground
, when the max possible value was 133.