Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] Example files for 2D and 3D progressive loading #6043

Open
wants to merge 157 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
157 commits
Select commit Hold shift + click to select a range
cd7b9c4
add Camera.look_at API
alisterburt Dec 18, 2022
44506f3
kinda a thing
alisterburt Jan 11, 2023
d6427b7
push the button
alisterburt Jan 16, 2023
542a1d1
Changes by jni
alisterburt Jan 16, 2023
dc41e0b
Color by position along centre line from camera
jni Jan 18, 2023
a1f7d39
Remove print statement for now
jni Jan 18, 2023
520dcfc
Update priority function; rearrange code
jni Jan 18, 2023
64d411d
Add debouncing
jni Jan 18, 2023
97a750c
minor formatting
jni Jan 18, 2023
f6e5271
Add function for finding chunk centers and chunk slices
jni Jan 18, 2023
68bfeab
Fix chunk slice computation
jni Jan 18, 2023
4e6cc15
Add function to update shown chunk in separate layer
jni Jan 18, 2023
d45fecd
Add layers and hook up high-res chunk loading
jni Jan 18, 2023
3cd77a8
Add another level of downsampling
jni Jan 25, 2023
0e675f9
Add TODOs for merging recursive multiscale chunk selection
kephale Feb 2, 2023
94083f6
Add TODOs for merging recursive multiscale chunk selection
kephale Feb 2, 2023
80137e0
Merge remote-tracking branch 'origin/poor-mans-octree' into poor-mans…
jni Feb 2, 2023
4e172a8
Add docstrings
jni Feb 2, 2023
2186315
Fix bug: incorrect variable name in function
jni Feb 2, 2023
dfb9ea1
Merge branch 'poor-mans-octree' of github.com:jni/napari into poor-ma…
kephale Feb 2, 2023
b3bb169
[wip] Merging poor-mans-octree work with napari-multiscale-prototype
kephale Feb 2, 2023
59b5aef
Use slices instead of tuple intervals, more kwargs for cache manager
kephale Feb 3, 2023
b6f78f2
Use slices for cache keys
kephale Feb 3, 2023
d16d313
Clarify coordinate systems when testing for visibility
kephale Feb 3, 2023
b4ef053
Refreshable with 'k' key, currently buggy after refreshing
kephale Feb 3, 2023
d079000
Fix typo in node_offset, almost working but dimension swap bug
kephale Feb 3, 2023
2416cb1
Refresh works!
kephale Feb 3, 2023
dd76a76
Remove scale factor keyword from chunk_centers
kephale Feb 3, 2023
21814e2
Clean up, logging
kephale Feb 3, 2023
045c7e8
Update textures in place
kephale Feb 3, 2023
005a127
Formatting, add a couple of docstrings
kephale Feb 4, 2023
e7f3483
Fix memory overflow issue for texture assignment
kephale Feb 4, 2023
7e57e38
Overwrite old data when rendering an interval non-multiple chunksize
kephale Feb 6, 2023
ce6b8ae
Add a TODO warning, clean up a little
kephale Feb 6, 2023
1069a0d
Introduce scale_factors, separate dataset loading into functions
kephale Feb 7, 2023
8688af7
Behaves as expected for openorganelle labels and Joel's, bug@idr44
kephale Feb 13, 2023
78e9e0c
Use a different chunking for idr0044a, slow but works
kephale Feb 13, 2023
cc58ea8
Separate data fetching into a generator and chunk renderer
kephale Feb 13, 2023
6f30e60
Threaded generator version
kephale Feb 13, 2023
a43910e
Minor cleaning, starting to work on profiling
kephale Feb 14, 2023
37199c2
Use camera_dict to freeze camera state, camera-triggered update
kephale Feb 14, 2023
7eb5a7f
Add notes from pairing with @jni, chunk fetch retry, fix debounce
kephale Feb 16, 2023
abb94a7
Use pydantic copy of camera instead of camera_dict, some cleaning
kephale Feb 16, 2023
e19d632
Add some performance timing code
kephale Feb 16, 2023
e98bb9b
best_priorities experiment, consider undoing.
kephale Feb 21, 2023
d5bc81a
Add and update docstrings
kephale Feb 21, 2023
26b663a
Fix typo
kephale Feb 21, 2023
c4910d8
Merge remote-tracking branch 'upstream/main' into poor-mans-octree
kephale Feb 21, 2023
35959a0
[wip] this worked on Joel's multichannel, but code got very messy
kephale Feb 24, 2023
c2ecf8d
Event listener for Dims
kephale Feb 24, 2023
18f68ff
Improve update ordering + node translation
kephale Feb 24, 2023
06b7256
Use pooch to get Joel's data, plus some cleaning
kephale Feb 24, 2023
8f568fb
Expose the example as a widget
kephale Feb 24, 2023
cacf48e
Add example rendering parameter to widget, more simplification
kephale Feb 24, 2023
7a530ad
Add colormap as an option to the widget
kephale Feb 24, 2023
f5fa7a9
Switch reader to ome-zarr-py, fibsem_tools changes too fast
kephale Mar 21, 2023
6d0e4a1
Add interpolated tiled rendering example, add private utils package
kephale Mar 28, 2023
211ad87
Unify example data loading for 2D and 3D
kephale Mar 29, 2023
0ce5a15
Update 2D tiled loading to use slices for fetching/caching chunks
kephale Mar 29, 2023
2fc95a1
Cleaning up
kephale Mar 29, 2023
0b4f469
Merge remote-tracking branch 'upstream/main' into poor-mans-octree
kephale Mar 29, 2023
7d3dd01
Merge remote-tracking branch 'upstream/main' into poor-mans-octree
kephale Apr 11, 2023
4f0cde2
wip: eliminate ChunkCacheManager, split out datasets
kephale Apr 28, 2023
ae90add
wip: lots of cleaning, renaming to data_plane_key
kephale Apr 28, 2023
206493a
Finish removal of cache_manager
kephale May 10, 2023
124c321
Note potential bug location
kephale May 10, 2023
fc387ed
wip: changes to support mandelbrot_vizarr on nesoi repo
kephale May 12, 2023
e309f3d
wip: more generalizing of functions to ND testing with nesoi
kephale May 15, 2023
b552043
wip: add a temporary example file from nesoi to simplify solving env
kephale May 15, 2023
7cbe01e
Fix float64->uint64 conversion that @kcpevey found
kephale May 15, 2023
1f6b3e3
Safer type conversion/truncation
kephale May 15, 2023
b091794
first pass
May 16, 2023
f3c6cbf
a few more things
May 16, 2023
4b632c8
Direct texture updates for 2D
kephale May 17, 2023
b2d43ab
nogil for numba
kephale May 17, 2023
064ae05
Compute chunk_slice indices on-demand
kephale May 17, 2023
3699994
Smooth transition between scales a little more
kephale May 17, 2023
cdf4dff
adding more stuff
May 18, 2023
7cf3af2
merge conflicts
May 18, 2023
bc529ea
Fix code swap between xcoords_image and mandelbrot
kephale May 18, 2023
8acd8ca
always render something - min/max scale when zoomed in/out
May 18, 2023
ef75178
Merge pull request #16 from kcpevey/add_comments
kephale May 18, 2023
8acff19
comments and max_levels to dataset
May 18, 2023
b2ebbfc
Fix slowdown with scales
kephale May 18, 2023
0911fda
nothing much
May 18, 2023
a296965
Merge remote-tracking branch 'origin/poor-mans-octree' into kim_stuff
May 18, 2023
88c910b
Working demo on massive image
kephale May 19, 2023
dd6247f
Merge branch 'poor-mans-octree' into kim_stuff
kcpevey May 22, 2023
a3b4ef1
Merge pull request #17 from kcpevey/kim_stuff
kephale May 22, 2023
1e30141
Fix interval usage for zarr arrays in chunk_slices
kephale May 24, 2023
664a330
add_tests
May 24, 2023
3457b17
add more tests
May 24, 2023
a2a937f
remove logger statement
May 24, 2023
5912ada
Merge pull request #18 from kcpevey/add_tests
kephale May 24, 2023
e634c2b
Formatting and adding @rabernat's zarr storage fix
kephale May 25, 2023
568f51e
Code cleaning
kephale May 31, 2023
40b3b31
More code cleaning, rename VirtualData.data_plane -> hyperslice
kephale Jun 1, 2023
e0962a9
Fix rest of hyperslice refactor, fix memory alloc bug from refactor
kephale Jun 1, 2023
43bc723
Merge remote-tracking branch 'upstream/main' into poor-mans-octree
kephale Jun 1, 2023
feca8ce
Fixes to match upstream/main
kephale Jun 1, 2023
5a46202
Use proper scale factors instead of powers of 2
kephale Jun 1, 2023
5479a18
Generalize render_sequence, dims_update_handler, and add_im
kephale Jun 1, 2023
ff22578
Update tests
kephale Jun 2, 2023
947623c
Minor refactoring
kephale Jun 7, 2023
a8463fe
Fixes to help 3D support
kephale Jun 9, 2023
13ed4a2
Mandelbulb demo stuff
kephale Jun 22, 2023
3c57db6
Merge branch 'main' into poor-mans-octree
kephale Jul 5, 2023
5a27941
Begin refactor of generative zarr code
kephale Jul 6, 2023
cc637b9
[wip] refactoring generative zarr code
kephale Jul 6, 2023
6aa6736
Fix dimension ordering in mandelbulb, 2D and 3D both work now
kephale Jul 6, 2023
a8049f1
Clean up examples, remove interpolated demo because only works for Z
kephale Jul 6, 2023
579172b
More docstrings
kephale Jul 6, 2023
d1dfb2f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 6, 2023
b0a904c
adding, organizing tests
kcpevey Jul 7, 2023
111bfb0
cleanup
Jul 7, 2023
73d6613
cleanup
Jul 7, 2023
2305d06
Add rendering as an argument, use contrast_limits. ty @chrishavlin
kephale Jul 20, 2023
2e1d7c1
Merge remote-tracking branch 'upstream/main' into poor-mans-octree
kephale Aug 10, 2023
06a56b5
Add example file for Joel's data, fix for ND images
kephale Aug 10, 2023
6f427e9
Use awful conditional for handling >3D
kephale Aug 10, 2023
7d8be30
Factor out corner variable
kephale Aug 10, 2023
dabe019
More code simplifications from @andy-sweet's feedback
kephale Aug 10, 2023
560d30e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 10, 2023
88704f8
Fix corner_pixels reference, remove magic scale pixel bounds
kephale Aug 11, 2023
b29f3ed
Merge branch 'poor-mans-octree' of github.com:kephale/napari into poo…
kephale Aug 11, 2023
d390b70
add more tests, update dependencies
Aug 21, 2023
14f97b2
Merge branch 'poor-mans-octree' into add_tests
kcpevey Aug 21, 2023
b25d29c
Merge pull request #20 from kcpevey/add_tests
kephale Aug 21, 2023
d3c0926
[wip] exploring alternative 3D chunk priority
kephale Aug 28, 2023
dc157e3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 28, 2023
804b016
Begin adding support for scale
kephale Aug 30, 2023
1fa7624
Merge branch 'poor-mans-octree' of github.com:kephale/napari into poo…
kephale Aug 30, 2023
f7ff0ee
Fix qt_viewer access.
kephale Sep 27, 2023
6d55b8d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 27, 2023
00f1efe
Improved 3D chunk prioritization
kephale Sep 27, 2023
792d965
Make 2D chunk prioritization match 3D
kephale Sep 27, 2023
dde1c75
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 27, 2023
1a95944
Rename dims_update_handler to progressively_update_layer
kephale Sep 27, 2023
1d642c2
[wip] Add zebrahub example, still working on timepoints
kephale Sep 28, 2023
566c75a
Some follow up tweaks for zebrahub demo
kephale Oct 2, 2023
b87f4d3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 2, 2023
3cba91f
Removing texture handling code, which is bad
kephale Oct 2, 2023
e83701e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 3, 2023
445cb50
Factor VirtualData into new module/tests
kephale Oct 4, 2023
4756f88
Merge
kephale Oct 4, 2023
32ab2fd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 4, 2023
3507cea
Add test for VirtualData.__getitem__
kephale Oct 4, 2023
640dad3
Disable debug rendering because it is 3D only
kephale Oct 4, 2023
b121ee4
merge
kephale Oct 4, 2023
667cc3f
Add more tests for VirtualData and MultiScaleVirtualData
kephale Oct 4, 2023
117d4a4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 4, 2023
e4f6123
Add more progressive loading tests
kephale Oct 4, 2023
4e26c45
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 4, 2023
dcff4ef
Merge remote-tracking branch 'upstream/main' into poor-mans-octree
kephale Oct 4, 2023
cc3917c
Update for napari main
kephale Oct 4, 2023
5f1c3ca
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 4, 2023
16bfc63
[wip] add multichannel_multiscale_mandlebulb
kephale Oct 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions examples/_mandelbrot_vizarr.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import logging
import sys
import warnings

import napari
from napari import Viewer
from napari.experimental._progressive_loading import (
add_progressive_loading_image,
)
from napari.experimental._progressive_loading_datasets import (
mandelbrot_dataset,
)

warnings.simplefilter(action='ignore', category=FutureWarning)


LOGGER = logging.getLogger("mandelbrot_vizarr")
LOGGER.setLevel(logging.DEBUG)

streamHandler = logging.StreamHandler(sys.stdout)
formatter = logging.Formatter(
"%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)
streamHandler.setFormatter(formatter)
LOGGER.addHandler(streamHandler)


if __name__ == "__main__":
# This dataset is 2D and visualized in 2D
ndisplay = 2
large_image = mandelbrot_dataset(max_levels=21)

# This dataset is 3D and visualized in 2D
# from napari.experimental._progressive_loading_datasets import openorganelle_mouse_kidney_em
# large_image = openorganelle_mouse_kidney_em()

# This dataset is 3D and visualized in 3D
# ndisplay = 3
# large_image = mandelbulb_dataset(max_levels=3)

viewer = Viewer(ndisplay=ndisplay)

multiscale_img = large_image["arrays"]
viewer._layer_slicer._force_sync = False

rendering_mode = "progressive_loading"

if rendering_mode == "progressive_loading":
# Make an object that creates/manages all scale nodes
add_progressive_loading_image(
multiscale_img,
viewer=viewer,
contrast_limits=[0, 255],
colormap='PiYG',
ndisplay=ndisplay,
)
else:
layer = viewer.add_image(multiscale_img)

napari.run()
Loading