Add optional view transform to ColorSpaceTransform

[michdolan]

This is a feature request:

There was some discussion around this in Slack recently and I'm observing in some implementation work that this could be useful. When converting between scene and display referred color spaces in a ColorSpaceTransform, currently only the default view transform can be used. The only way to use a specific view transform is by creating a view , even if only for temporary toolbox-like use, and using it with a DisplayViewTransform. I wanted to propose we add a "viewTransform" parameter to ColorSpaceTransform, which is unset by default (maintaining existing behavior), but could be set with a view transform name to be used when a conversion requires it. This would especially be useful for implementation in nodes like OCIOColorSpace in Nuke and the OFX plugins being worked on. It would also aid in creating display-like color spaces when needed.

Perhaps there are some cons I'm not considering which we can discuss here.

[zachlewis]

I think this is a really great idea.

[doug-walker]

Thanks for raising this question Michael. There are two aspects to this that I want to make sure we keep separate: the UX question of what tools should be made available to artists; and second the functionality of the underlying OCIO classes such as ColorSpaceTransform and DisplayViewTransform. (Once we identify the former, hopefully the latter will become clear.)

I would like to propose that there be two separate tools offered to artists. The first would be for converting between color spaces that is essentially a "units conversion" similar to converting inches to centimeters. In other words, it is the same image, just represented in different units. This would be intended primarily for transformations between scene-referred spaces or between display-referred spaces (but not for across the divide). There are no options needed, just source and destination. This is essentially what ColorSpaceTransform (both the artist tool and underlying class) should be intended for.

The second tool would be for converting between scene-referred and display-referred or vice versa and therefore involve a view transform. As you know, these are not simple unit conversions since they typically involve tone-mapping, appearance corrections, and etc. that mean it is effectively not the same image anymore (e.g. the highlights may be compressed or even clamped). An additional option is needed which is the view transform to use. (Also, a forward/inverse switch may be needed depending on how the UI is set up.) This is what the DisplayViewTransform class is intended for, and I'd like to suggest that it should have an accompanying artist tool that would satisfy this feature request.

As you point out, ColorSpaceTransform class can use the default view transform to convert from scene-referred to display-referred, or vice versa, but that was done as a fall-back to avoid breaking things. I worry that it would cause confusion to encourage that the ColorSpaceTransform be used for conversions involving a view transform and providing more flexibility about which view transform is used.

The complicating factor is that in OCIO v1, there were no explicit view transforms and so views were defined via color spaces. Hence, for a config like that, ColorSpaceTransform has always in some cases involved a LUT that has a view transform baked into it and converts from scene-referred to display-referred, or vice versa. So there are arguments pro and con and I can understand the interest in extending ColorSpaceTransform in this way. But I've never liked that usage of ColorSpaceTransform since it requires artists to somehow know the mapping from the Display/View names to the corresponding color space names.

A related practical problem of adding a view transform option to ColorSpaceTransform is what to do if one of the color spaces involved is a v1 type space that already has a view transform baked into it. An artist could obviously get themselves into trouble by doubling up a transform. With the legacy configs, there is no way to know if this is the case.

My argument is that it will be clearer for the artists if there are two separate tools. For example, they are already used to working with Display/View menus in the DCC viewports. So it should make sense to have a separate "Display/View" tool with exactly the same menus for use when they want to bake in (or remove) that type of transformation.

A related benefit is that if the Displays/Views are built using looks, these get applied automatically by DisplayViewTransform without any additional menus. I'm not clear how that would work with adding a viewTransform option to ColorSpaceTransform, is a looks option needed too?

But I think we should discuss it further. OCIO is so flexible that it can be difficult to figure out how to best map it onto artist tools, especially when there are limits as to how much config introspection an app is able to do, particularly with v1 configs.

[michdolan]

Thanks Doug. Your points are certainly valid. I think the challenge is that intuitively artists who are used to OCIO v1 will try to use a ColorSpaceTransform (and the associated tools that use it) to convert between an scene and display referred space (like from ACEScg to Output - sRGB) expecting that "Output - sRGB" has the ACES RRT baked in, but will end up essentially with a ACEScg to canonical sRGB transform, which will look wrong and cause confusion.

I do agree this problem could be solved with UX. If I were to design an ideal color space conversion node (based solely on my opinions and experience in a production environment), I would make that node only show a "src" and "dst" color space selection by default, but then if the user selected color spaces with different reference spaces I would unhide, or enable, an additional combo box with available view transforms, and a tooltip to make it clear what it is. I think the struggle for me here is that we do allow ColorSpaceTransform to convert between scene and display referred spaces, but we don't support the possibility of a non-default view transform, which will be needed to convert between these correctly most of the time (unless the config's default view transform is set to the most common case, which is a possible workaround).

If the right thing to do is always funnel implementations to the DisplayViewTransform, I would almost prefer that ColorSpaceTransform not support crossing the scene/display referred boundary, though I know others will disagree with me there. It just feels like an omission to allow it fundamentally but not in a way that could be used in many cases without hackery (i.e. putting a DisplayViewTransform in a scene referred color space, which will more than likely be done to get around this).

But to your points, perhaps the answer is to have the OCIOColorSpace node only allow conversion between similarly referenced color spaces (have to choose between "scene" and "display"), which would then force users to use OCIODisplayView to cross that boundary. But if that was the intended design, the OCIODisplayView node would have to have an "invert" option, as being able to get from a display to scene referred space with the view transform inverse is critical behavior in a VFX pipeline IMO.

And if we decide not to expose view transform on the ColorSpaceTransform API, I think we should at least put a clear note in the documentation regarding the limited view transform behavior.

[doug-walker]

I think this issue is one of the main color management UX issues, not only for OCIO but for ACES and other projects as well, so this is a great discussion to have. For example, this come up all the time on ACESCentral where users want to convert sRGB to ACES2065-1 and get confused because there are two different ways of doing it depending on the use-case: either as a texture (i.e., just re-encode without any inverse view transform); or as an inverse Output Transform. One could argue that it is conceptually clearer to have the first one be a ColorSpaceTransform tool and the second be an inverse DisplayViewTransform tool to drive home to the user that they are two quite different types of conversions. But as Michael suggested, it could also be done by adding a View Transform menu to a ColorSpaceTransform tool.

As I was saying during the TSC discussion today, OCIO basically has three main types of color spaces:

1. Pure scene-referred color spaces (e.g., ACEScg, ARRI LogC, etc.)
2. Pure display-referred color spaces (e.g. Rec.709 monitor, sRGB monitor, etc.)
3. Display-referred color spaces that had a view transform baked in (e.g. Rec.709 with an ACES RRT+ODT, Rec.709 with an ARRI K1S1 view transform, etc.)

The second type (pure display-referred) was added in OCIO v2, along with the ability to define a View Transform as a distinct object in the config.

The third type are arguably not proper "color spaces" in the traditional color science sense and, with the new features in OCIO v2, these could be replaced with a combination of a distinct View Transform plus a display-referred color space. The new ACES reference config for OCIO v2 demonstrates how to do this.

So this third type of "color space" could be phased out gradually over time as config authors and tool vendors adopt the new features, but there will be some period of time where users will have to be able to work with a mixture of all three types of spaces. One big question is how to give users the best UX during the transition period where both styles of config needs to be supported.

I do see the appeal of adding a View Transform menu to ColorSpaceTransform, since it means it's just a change to an existing tool, it doesn't require a new tool. And as Michael points out, users who are accustomed to always just reaching for the ColorSpaceTransform would need to readjust their behavior. But my preference is still to have the separate DisplayViewTransform tool, for the reasons I wrote above. Also, Nuke (OCIODisplay) and other apps already have a separate tool very similar to this (although it will need an "invert" button added, as Michael wrote, if not already present).

At some point in the future, a ColorSpaceTransform tool could be more restrictive in what transforms it allows. But it seems that in the near term at least it must continue to offer all of its current capabilities.

But we talked during the TSC about potentially having a ColorSpaceTransform tool warn users in some cases. For example, if they select Source and Destination color spaces that are scene/display or display/scene, this could be done. The encoding attribute could also be used to try and detect that situation. I think this is definitely worth doing, but we should keep in mind that this will not always work since there is no foolproof way of determining if a color space is type 1 or type 3 above. (In other words, it requires use of OCIO v2 features which are optional and that the config author may not have set up.)