Warping transform is not taken into account in volume rendering

[lassoan]

Summary

It is documented that "The volume must not be under a warping (affine or non-linear) transformation. To render a warped volume, the transform must be hardened on the volume.". However, we cannot expect users to know about this limitation.

Environment

• Slicer version: Slicer-5.1.0-2022-11-06
• Operating system: all

[lassoan]

Potential solutions when the volume is under a warping transform:

• Option A: Disable showing the volume rendering 3D view and show a warning in Volume Rendering module that explains what the problem is and shows a button to harden the transform on the volume.
• Option B: Clone the volume internally, harden the warping transform on it, and volume render that. The warped volume could be stored in the vtkMRMLVolumeRenderingDisplayNode node so that it can be shared between all volume renderers; or in the pipeline in vtkMRMLVolumeRenderingDisplayableManager.

[pieper]

I like option B, but maybe we should be more explicit since it will take resources and add time to computing. That is, how would we know when it's okay to discard the clone? Should the clone be saved with the scene?

How about when the user tries to make a volume under a non-linear transform visible Slicer won't display it, but offers a button to clone and harden the volume and display the clone instead. This would then be explicit in the scene and the user could manage when it's no longer needed. We could also keep a reference from the clone to the original so that it could be updated if the source volume or transform is modified.

[pieper]

Maybe we should also make a Harden dialog that is something like a the geometry selection dialog in the segment editor or the interface of the Crop Volume module so that users can control the resampling parameters.

[lassoan]

That is, how would we know when it's okay to discard the clone

We already have a flag to automatically free up resources on hiding volume. It could be used for this, too.

We do this automatic internal cloning and hardening in the Segment Editor and nobody complained so far. Even for large volumes, the computation is not too long (up to maybe 10-20 seconds).

How about when the user tries to make a volume under a non-linear transform visible Slicer won't display it, but offers a button to clone and harden the volume and display the clone instead

That's about what I meant by Option A. Would you display a popup window? We would need some GUI solution that works reasonably when the user drag-and-drops the image to the 3D view (or enables volume rendering in the right-click menu in the Subject Hierarchy eye icon).

I think Option B would be easier to implement (less GUI development, which is always complex and a lot of effort) and would be simpler for users (it would "just work"). Since this would only be used for volume rendering (users would not access the resampled volume), we would not need to worry about losing some details (we could even limit the output resolution to be within some memory limit - as it used to be in earlier VTK versions).

Actuay, we could first implement Option B and then later we could add Option A (add the GUI for cloning&resampling, optionally with manual adjustment of output geometry).

[pieper]

I see your points. Maybe the easiest to implement would be just to automatically clone and harden when volume rendering but keep it as a visible node in the scene. Then it can be re-used if needed or deleted manually. It's not unusual for operations to result in new nodes being added to the scene. We could always modify this behavior later if we don't like it.

[lassoan]

The volume is no longer displayed in incorrect position, so the issue is not urgent anymore. The full fix will be to internally harden the transform (or to set up a resampling module with auto-run to create a resampled volume).