Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose tensor slice selection to blueprint #6590

Merged
merged 24 commits into from
Jun 24, 2024
Merged

Conversation

Wumpf
Copy link
Member

@Wumpf Wumpf commented Jun 18, 2024

What

Updated python example:

blueprint = rrb.Blueprint(
    rrb.TensorView(
        origin="tensor",
        name="Tensor",
        # Explicitly pick which dimensions to show.
        slice_selection=rrb.TensorSliceSelection(
            # Use the first dimension as width.
            width=0,
            # Use the second dimension as height and invert it.
            height=rr.TensorDimensionSelection(dimension=1, invert=True),
            # Set which indices to show for the other dimensions.
            indices=[
                rr.TensorDimensionIndexSelection(dimension=2, index=4),
                rr.TensorDimensionIndexSelection(dimension=3, index=5),
            ],
            # Show a slider for dimension 2 only. If not specified, all dimensions in `indices` will have sliders.
            slider=[2],
        ),
        # Set a scalar mapping with a custom colormap, gamma and magnification filter.
        scalar_mapping=rrb.TensorScalarMapping(colormap="turbo", gamma=1.5, mag_filter="linear"),
        # Change sizing mode to keep aspect ratio.
        view_fit="fill",
    ),
    collapse_panels=True,
)

image

Models out the tensor slice selection in store compatible data structures

Refrained from polishing up the ui all that much, it's pretty much the same but uses the new datastructure under the hood.
Minor differences:

  • No longer show both arrows for 1D tensor slices.
  • allow reset to default and reset to default blueprint
  • change default for fill to keep aspect ratio

Had to do add a bit of codegen for python struct serialization, but ended up doing most of the new datatypes by hand after all because of hard to generalize conversions.

Tested manually against 1D tensors.

Checklist

  • I have read and agree to Contributor Guide and the Code of Conduct
  • I've included a screenshot or gif (if applicable)
  • I have tested the web demo (if applicable):
  • The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
  • If applicable, add a new check to the release checklist!

To run all checks from main, comment on the PR with @rerun-bot full-check.

@Wumpf Wumpf added 🟦 blueprint The data that defines our UI include in changelog labels Jun 18, 2024
Copy link

github-actions bot commented Jun 18, 2024

Deployed docs

Commit Link
523554b https://landing-7zely1ym8-rerun.vercel.app/docs

@Wumpf Wumpf mentioned this pull request Jun 18, 2024
12 tasks
@emilk emilk self-requested a review June 19, 2024 15:30
Copy link
Member

@emilk emilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(half-reviewed - I ran out of day)

Comment on lines 9 to 10
tensor = np.random.randint(0, 256, (32, 20, 12, 14), dtype=np.uint8)
rr.log("tensor", rr.Tensor(tensor, dim_names=("width", "height", "batch", "other")))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we should make our examples as realistic as possible to make it easier for users to copy-paste them

Suggested change
tensor = np.random.randint(0, 256, (32, 20, 12, 14), dtype=np.uint8)
rr.log("tensor", rr.Tensor(tensor, dim_names=("width", "height", "batch", "other")))
tensor = np.random.randint(0, 256, (32, 768, 1024, 3), dtype=np.uint8)
rr.log("tensor", rr.Tensor(tensor, dim_names=("batch", "height", "width", "channel")))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not going with an as-high resolution since it looks too ugly with this noise

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm unless I set the filter to "nearest" but that's already the default so not as interesting

docs/snippets/all/views/tensor.py Outdated Show resolved Hide resolved
docs/snippets/all/views/tensor.py Outdated Show resolved Hide resolved
@Wumpf
Copy link
Member Author

Wumpf commented Jun 19, 2024

thanks! some neat stuff in there. Plz have a quick look at the comments I replied to

@emilk
Copy link
Member

emilk commented Jun 20, 2024

The component defaults menu in the tensor selection is empty, which looks very buggy:

tensor-component-defaults

/// General rules for scrubbing the input data:
/// * out of bounds dimensions and indices are clamped to valid
/// * missing width/height is filled in if there's at least 2 dimensions.
pub fn load_tensor_slice_selection_and_make_valid(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could really use a unit-test or two.

For instance, what happens if your dimensions are named height, width and the input is height=1, width=null? From just reading the code, it looks like maybe the output will be height=1, width=1.

crates/re_space_view_tensor/src/space_view_class.rs Outdated Show resolved Hide resolved
crates/re_space_view_tensor/src/space_view_class.rs Outdated Show resolved Hide resolved
crates/re_space_view_tensor/src/space_view_class.rs Outdated Show resolved Hide resolved
crates/re_space_view_tensor/src/space_view_class.rs Outdated Show resolved Hide resolved
crates/re_space_view_tensor/src/space_view_class.rs Outdated Show resolved Hide resolved
@Wumpf
Copy link
Member Author

Wumpf commented Jun 20, 2024

The component defaults menu in the tensor selection is empty, which looks very buggy:

I guess we should grey it out when there's nothing. But very unrelated to this PR

@emilk
Copy link
Member

emilk commented Jun 20, 2024

The component defaults menu in the tensor selection is empty, which looks very buggy:

I guess we should grey it out when there's nothing. But very unrelated to this PR

I'll make a new issue

@Wumpf
Copy link
Member Author

Wumpf commented Jun 24, 2024

@rerun-bot full-check

Copy link

Started a full build: https://github.com/rerun-io/rerun/actions/runs/9646353988

@Wumpf Wumpf merged commit b5da55a into main Jun 24, 2024
73 of 74 checks passed
@Wumpf Wumpf deleted the andreas/tensor-slice-selector branch June 24, 2024 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🟦 blueprint The data that defines our UI include in changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants