Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support to load ilastik h5 flavors #305

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 95 additions & 6 deletions volumina/__main__.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,26 @@
import argparse
import contextlib
import json

Check warning on line 3 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L3

Added line #L3 was not covered by tests
import signal
import sys
from pathlib import Path
from typing import Tuple, Union

Check warning on line 7 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L6-L7

Added lines #L6 - L7 were not covered by tests

import h5py

Check warning on line 9 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L9

Added line #L9 was not covered by tests
import numpy
import xarray
from PyQt5.QtWidgets import QApplication

from volumina import __version__
from volumina.api import Viewer
from volumina.colortables import default16_new
from volumina.pixelpipeline.datasources import ArraySinkSource, ArraySource
from volumina.layer import ColortableLayer, GrayscaleLayer
from volumina.pixelpipeline.datasources import ArraySinkSource, ArraySource

Check warning on line 18 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L18

Added line #L18 was not covered by tests


class _Suffixes:
npy = [".npy"]
h5 = [".h5", ".hdf5", ".hdf", ".ilp"]

Check warning on line 23 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L21-L23

Added lines #L21 - L23 were not covered by tests


@contextlib.contextmanager
Expand Down Expand Up @@ -48,9 +57,12 @@
usage="",
epilog="",
)
p.add_argument("image", help="Path to .npy image")
p.add_argument("image", help="Path to [.npy, .h5] image (in case of h5 with internal path)")

Check warning on line 60 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L60

Added line #L60 was not covered by tests
p.add_argument(
"--axistags", help="Strings describing axes in image. Valid values: 'tzyxc'", type=axiorder_type, required=True
"--axistags",
help="Strings describing axes in image. Valid values: 'tzyxc'. Required for `.npy` files.",
type=axiorder_type,
required=False,
)
p.add_argument("--version", action="version", version=__version__)

Expand All @@ -67,16 +79,93 @@
return tagged_data_5d.data


def is_npy(path: Path) -> bool:
if path.suffix in _Suffixes.npy:
return True

Check warning on line 84 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L82-L84

Added lines #L82 - L84 were not covered by tests

return False

Check warning on line 86 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L86

Added line #L86 was not covered by tests


def _external_internal_h5(path: Path) -> Tuple[Path, Path]:
if path.suffix in _Suffixes.h5:
external_path = path

Check warning on line 91 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L89-L91

Added lines #L89 - L91 were not covered by tests
# no internal path given!
# guess if there is only one ds in file:
with h5py.File(external_path, "r") as f:
if len(f) == 1:
internal_path = next(iter(f.keys()))

Check warning on line 96 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L94-L96

Added lines #L94 - L96 were not covered by tests
else:
raise ValueError("Could not determine internal path.")

Check warning on line 98 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L98

Added line #L98 was not covered by tests

else:
external_path = next(iter(a for a in path.parents if a.suffix in _Suffixes.h5), None)

Check warning on line 101 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L101

Added line #L101 was not covered by tests

if not external_path:
raise ValueError("Could not determine external path.")

Check warning on line 104 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L103-L104

Added lines #L103 - L104 were not covered by tests

internal_path = path.relative_to(external_path)

Check warning on line 106 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L106

Added line #L106 was not covered by tests

return external_path, internal_path

Check warning on line 108 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L108

Added line #L108 was not covered by tests


def is_h5(path: Path) -> bool:
try:
_ = _external_internal_h5(path)
except ValueError:
return False

Check warning on line 115 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L111-L115

Added lines #L111 - L115 were not covered by tests

return True

Check warning on line 117 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L117

Added line #L117 was not covered by tests


def load(data_path: str) -> Tuple[numpy.ndarray, Union[str, None]]:
p = Path(data_path)
if is_npy(p):
return load_npy(p), None
elif is_h5(p):
return load_h5(p)

Check warning on line 125 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L120-L125

Added lines #L120 - L125 were not covered by tests
else:
raise NotImplementedError("Unsupported file format - sorry.")

Check warning on line 127 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L127

Added line #L127 was not covered by tests


def load_npy(data_path: Path) -> numpy.ndarray:
return numpy.load(data_path)

Check warning on line 131 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L130-L131

Added lines #L130 - L131 were not covered by tests


def determine_h5_axistags(ds: h5py.Dataset) -> str:
if ds.dims:
return "".join([dim.label for dim in ds.dims])
elif "axistags" in ds.attrs:
return "".join([ax["key"] for ax in json.loads(ds.attrs["axistags"])["axes"]])

Check warning on line 138 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L134-L138

Added lines #L134 - L138 were not covered by tests

return ""

Check warning on line 140 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L140

Added line #L140 was not covered by tests


def load_h5(data_path: Path) -> Tuple[numpy.ndarray, str]:
external_path, internal_path = _external_internal_h5(data_path)
with h5py.File(external_path, "r") as f:
ds: h5py.Dataset = f[str(internal_path)]
axistags = determine_h5_axistags(ds)
data = ds[()]
return data, axistags

Check warning on line 149 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L143-L149

Added lines #L143 - L149 were not covered by tests


def main():
args = parse_args()
data = numpy.load(args.image)
reordered_data = reorder_to_volumina(data, args.axistags)
data, axistags = load(args.image)

Check warning on line 154 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L154

Added line #L154 was not covered by tests

if not (args.axistags or axistags):
print(f"Axistags required for file {args.image} with shape {data.shape}")
return 1

Check warning on line 158 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L156-L158

Added lines #L156 - L158 were not covered by tests
Comment on lines +157 to +158
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will print a message to stderr and exit with code 1.

sys.exit(f"Axistags required ...")


reordered_data = reorder_to_volumina(data, args.axistags or axistags)

Check warning on line 160 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L160

Added line #L160 was not covered by tests

with volumina_viewer() as v:
v.addGrayscaleLayer(reordered_data, name="raw")
v.setWindowTitle(f"Volumina - {args.image}-{args.axistags}")
v.showMaximized()

return 0

Check warning on line 167 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L167

Added line #L167 was not covered by tests


if __name__ == "__main__":
main()
sys.exit(main())

Check warning on line 171 in volumina/__main__.py

View check run for this annotation

Codecov / codecov/patch

volumina/__main__.py#L171

Added line #L171 was not covered by tests
Loading