Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Axis sliders with multiple translated arrays #659

Open
d-v-b opened this issue Nov 5, 2019 · 5 comments
Open

Axis sliders with multiple translated arrays #659

d-v-b opened this issue Nov 5, 2019 · 5 comments
Labels
bug
Milestone

Comments

@d-v-b
Copy link
Contributor

@d-v-b d-v-b commented Nov 5, 2019

馃悰 Bug

The 'translatekwarg toview_image` does weird stuff.

offset = 100
t1 = (0,0,0)
# shifted in "Z"
t2 = (offset,0,0)
data = np.random.randn(offset,offset,offset)
# view the unshifted array
v = napari.view_image(data, translate=t1, name='fixed', blending='additive')
# view the same data with a shift
v.add_image(data, translate=t2, name='moving', blending='additive') 

This renders both arrays overlapping in the same region in viewer space, which seems weird, given that I translated the second array in Z expressly so that they would not overlap.

I would expect the "roll dimensions" button to offer me a "side view" of the arrays that represents the Z offset, but this is not the case -- all the views afforded by the "roll dimensions" button show the arrays overlapping in space.

The volume rendering button does show the arrays in the expected shifted positions in space. Interestingly, if I roll the displayed dimensions once, then engage volume rendering, then disable volume rendering, I see the two arrays shifted as expected.

Presumably this can all by avoided by padding the shifted array with 0s, but that's far from ideal....

Expected behavior

When a single array is being rendered, the axis sliders are both incrementing the slice of the data to be displayed (i.e., changing coordinates in the reference frame of the array) AND (implicitly) moving the clipping plane of the data (i.e., changing coordinates in the reference frame of the viewer area). But these are actually two very distinct operations, as I think the two-shifted-arrays example demonstrates. So maybe the job of the axis slider should be formally changed from "change the index of the arrays" to "move the clipping plane through viewer space, which happens to contain arrays". Unfortunately, this would break a lot of stuff! So I'm not really sure what's right here.

Environment

napari, 0.2.4
Qt, 5.9.6
NumPy, 1.17.2
SciPy, 1.3.1
VisPy, 0.6.1
scikit-image, 0.16.2

Platform: Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid
Python: 3.7.4 (default, Aug 13 2019, 20:35:49) [GCC 7.3.0]
Backend: PyQt5
GL version: '4.6.0 NVIDIA 430.26'
MAX_TEXTURE_SIZE: 32768 (is this a lot?)

@sofroniewn

This comment has been minimized.

Copy link
Contributor

@sofroniewn sofroniewn commented Nov 5, 2019

Some of this might be related to #640 - I need to play with it though. Can you include some screenshots / maybe gifs describing this in more detail too.

Also I feel t2 = (offset,offset,0) is needed to offset in both Z and Y

@sofroniewn sofroniewn added the bug label Nov 5, 2019
@sofroniewn sofroniewn added this to the 0.2.0 milestone Nov 5, 2019
@d-v-b

This comment has been minimized.

Copy link
Contributor Author

@d-v-b d-v-b commented Nov 5, 2019

The reference to Y is erroneous, from an earlier version of the MRE, I will edit it out of the issue.

I posted a screencap in zulip for your enjoyment.

@jni

This comment has been minimized.

Copy link
Contributor

@jni jni commented Nov 6, 2019

Yep, I think #640 is essentially the same issue, and one we have avoided confronting for some time: does napari operate on pixel space, or "real" (e.g. 碌m) space? Or some weird hybrid of the two? Right now, it's generally pixel, with a bit of fudging on top. imho we should move to a real space model, but that's going to take a fair bit of engineering... Things like the proportional spacing of the sliders, and what a "step" is in the slider, will have to be re-thought. What happens if you superimpose two images with different scales on top of each other, as in CLEM? Does the slider step through the low-res volume, or the high-res one, or, in general, in the highest-res one? Or do we make it continuous?

All this stuff does my head in a little. =P

@d-v-b

This comment has been minimized.

Copy link
Contributor Author

@d-v-b d-v-b commented Nov 6, 2019

I think napari has to operate on pixel data sampled in real space for stuff like translate and scale to make any sense at all, or for napari to be useful for stuff like CLEM data. So that means that axis sliders should increment in real units, not indices.
Here's what I think this ends up looking like: If there's only 1 array being viewed, nothing obviously changes -- the step size of the axis slider is exactly the interval between samples of the array. When two arrays are being rendered where one has been translated along the axis controlled by the slider, the axis slider range (which is now defined in real units, not indices) should expand to span the bounding box defined by the extreme edges of both arrays. Interpolation along the slider dimension should be used to render the result of rendering a clipping plane that doesn't align with the sampling grid of the arrays (this is kind of nice insofar as it generalizes the currently 2D interpolation options to ND :) ) These at least are my initial thoughts. Maybe getting this stuff sorted out is a necessary task on the road to arbitrary clipping planes of 3D data?
It might be good to look at how other programs handle this. Bigdataviewer, Amira, and Imaris come to mind.

@jni

This comment has been minimized.

Copy link
Contributor

@jni jni commented Nov 6, 2019

I think napari has to operate on pixel data sampled in real space for stuff like translate and scale to make any sense at all, or for napari to be useful for stuff like CLEM data.

I agree with this, of course.

So that means that axis sliders should increment in real units, not indices.

Sure, but I really want to make sure that all the stuff you suggest later does not interfere at all with the simplicity of using napari to just look at arrays. I think that's a big part of why the response to napari has been so good, and there is a real danger of napari becoming a beast that people are reluctant to launch for quick viz tasks.

But, that's just my spiel to check our solutions, not to argue that this is not something we need to solve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can鈥檛 perform that action at this time.