BUG: use alpha to calculate RGBA lighting #43
Conversation
Implements the feature and fixes discussed in this thread: https://discourse.slicer.org/t/volume-rendering-colorized-with-segmentation/26689/6 Basically allows RGBA volumes to be shaded according to their alpha component so that something like a CT or microCT can define an rgb value at spatial regions, while getting opacity and gradient information from the alpha channel (e.g. the Hounsfield unit). Also always uses the 0 channel for defining the lighting parameters. This part is WIP since it may be more appropriate to set the alpha channel lighting parameters in this scenario. Currently, at least in 3D Slicer, only the 0 compnent's lighting parameters are being set, so they are being used here.
|
See also: lassoan@c372d71 |
In the vtkOpenGLGPUVolumeRayCastMapper when the components are dependent (that is, RGBA), only a single value is set in the lighting arrays (in_ambient, in_diffuse, ...). So in the shader code it must also use the zeroth element of the lighting arrays to calculate the final lit colors.
|
@lassoan I think this looks better than hard coding those 0 values. I tracked down that when it's copying lighting info to the shader code it only copies When the components are dependent (rgba) it only has one sampler. So with this proposed change the component will be zero in the case of dependent components, but will be the component number in the case of independent components. Can you have a look and see if you agree with the logic? If so we make the an upstream MR for VTK and then cherry pick the commit back here. |
|
Regex can be slow and seems that it might be a bit of an overkill. Could we add something like |
|
@pieper Thanks for working on this. |
|
FYI, I've implemented "Colorize volume" module in Slicer that creates really nice RGBA volumes. To make edges smooth, I've developed a masked median filter that can slightly expand the labels and it is very quick, robust, and works well when there are multiple labels in the neighborhood. For example, CTLiver sample data set segmented with TotalSegmentator. Without masking (segmentation is only used for coloring and slightly decreasing opacity outside segments): SlicerCapture.mp4With masking (region outside segments are blanked out): SlicerCapture4.mp4What is remarkable that this allows visualization of the texture from the original CT, see for example the muscle fibers of gluteus maximus. |
| shaderStr += std::string("\ | ||
| \nvec4 computeColor(vec4 scalar, float opacity)\ | ||
| \n {\ | ||
| \n return clamp(computeLighting(vec4(scalar.xyz, opacity), 3, 0.0), 0.0, 1.0);\ |
There was a problem hiding this comment.
Can we consider that using alpha for shading is what makes more sense for every use cases? Then this part should be contributed back to VTK ideally.
There was a problem hiding this comment.
I think yes, if there's an alpha channel you almost always want to use that for lighting, shading, and opacity calculations. I don't know of any use case where you wouldn't want that. And yes, this should go in VTK once we finalize it.
Since the whole shader system in VTK is based on string replacements this seems consistent. And this is not performance critical code since the shaders are cached and meant to be rarely recomputed. But yes, using a variable would be an option. |
|
@pieper please have a look, squash, and mark it ready if you think it is ready to be merged. Thank you! It would be nice if the fix could get into Slicer/VTK today. |
Implements the feature and fixes discussed in this thread:
https://discourse.slicer.org/t/volume-rendering-colorized-with-segmentation/26689/6
Basically allows RGBA volumes to be shaded according to their alpha component so that something like a CT or microCT can define an rgb value at spatial regions, while getting opacity and gradient information from the alpha channel (e.g. the Hounsfield unit).
Also always uses the 0 channel for defining the lighting parameters. This part is WIP since it may be more appropriate to set the alpha channel lighting parameters in this scenario. Currently, at least in 3D Slicer, only the 0 compnent's lighting parameters are being set, so they are being used here.