Anisotropy direction parametrization

[virtualzavie]

The current OpenPBR specification proposes to control the anisotropy with a specular_anisotropy (respectively coat_anisotropy) and the direction of the specular highlight elongation with a specular_rotation (respectively coat_rotation).

In the case those inputs are given with a texture, or more generally if some filtering is involved, the specular_rotation will require special care to avoid artifacts due to the discontinuity at the 360° / 0° angle. At a minimum, this requires the ability to specify a nearest-neighbor filter, but obtaining a level of quality consistent with bilinear, trilinear or anisotropic filtering involves implementing a custom filtering scheme to handle the discontinuity case. Such a custom filtering has to be implemented deep into the shader, requires more texture fetches, and represents a non trivial amount of work for the developer of the renderer.

An alternative approach is to specify the direction with a flow map of the anisotropy direction 2D vectors. This parametrization is more suitable to texture filtering implemented in most renderers and 3D hardware, does not require a special case, and is very similar to normal maps which are widely supported.

Although it may seem expressing an angle (1 parameter) as a vector (2 parameters) might incur an additional bandwidth cost, factoring specular_anisotropy into the vector norm should lead to an equivalent cost (or lesser since no custom filtering with additional fetches is involved).

glTF expresses the anisotropy direction and strength as a 3 component texture though, so pro and cons of the two solutions would have to be weighted.

It may also seem that authoring a flow map directly is more difficult, but authoring a rotation map directly is in fact difficult as well and better performed with a dedicated tool anyway.

[portsmouth]

It would be useful to see some renders. Is it not possible to avoid this discontinuity by computing the tangent vector before the interpolation? (That is, blend vectors, not angles).

[peterkutz]

It would be useful to see some renders. Is it not possible to avoid this discontinuity by computing the tangent vector before the interpolation? (That is, blend vectors, not angles).

@portsmouth Yes, I believe that is the custom filtering that @virtualzavie is referring to:

obtaining a level of quality consistent with bilinear, trilinear or anisotropic filtering involves implementing a custom filtering scheme to handle the discontinuity case. Such a custom filtering has to be implemented deep into the shader, requires more texture fetches, and represents a non trivial amount of work for the developer of the renderer.

[virtualzavie]

It would be useful to see some renders.

As an illustration, here is the same model:

• displayed correctly, thanks to a custom filtering implemented in the fragment shader:

• displayed with a MIP mapping bias such that the custom texturing filtering no longer prevents the anisotropy angle discontinuity from causing a visible seam:

• the anisotropy angle map:

[portsmouth]

As discussed in the meeting, I take it that the proposal is to replace the specular_rotation angle parameter with a vec2 parameter say $(t, b)$, where the resulting rotated tangent vector is defined to be:

$$ \mathbf{T}' = t \mathbf{T} + b \mathbf{B} $$

with $\mathbf{T}$, $\mathbf{B}$ the (assumed orthonormal) reference tangent and bitangent vectors (in world or local space). Then we should require $t^2 + b^2=1$ (we could not require that and instead say the vector needs to be normalized before use, but then adjacent points could use different normalizations and produce filtering artifacts).

From the discussion, it also seemed to be desirable to have $t, b$ be parametrized by values in [0,1], remapped internally to [-1,1] (so that the vec2 parameter can be supplied as the RG components of a (raw) RGB texture? Presumably that texture is the "flow-map"?). So then the default value would be (1, 0.5), meaning the tangent is unrotated.

Or is some other parametrization preferable?

[portsmouth]

Another thought, perhaps it could be convenient to keep the existing rotation angle parameter as well as the vec2 "flow map"? The rule could be that the rotation is applied to the tangent vector obtained from the flow map -- which by default is the original tangent. If we did that, then the flow map would be additive to the existing model. (The flow-map could be seen as an "advanced" parameter for control over the tangent vector direction without introducing filtering artifacts).

This would cover the cases where the rotation angle is the preferred parametrization (for convenience say, despite the potential filtering issues). Obviously, if the flow map is supplied and the rotation left untouched, the filtering artifact would not occur.

[portsmouth]

@elalish How does the proposals/discussion here align with what you've specified in glTF? (You mentioned you came up with a scheme for handling anisotropy interpolation artifacts).

[elalish]

@elalish How does the proposals/discussion here align with what you've specified in glTF? (You mentioned you came up with a scheme for handling anisotropy interpolation artifacts).

Mostly, you can see ours here. We did the vec2 direction plus a 3rd channel for strength to avoid various texture filtering problems like the ones you've described. We also have a scalar rotation angle, but that is not textured, given the filtering issues. I'm not sure if you have the concept of factors that are separate from textures like glTF has.

[portsmouth]

@elalish OK thanks, that all makes sense. Perhaps for consistency with your approach we could define our "flow map" with the same parametrization, i.e. a vec3 with XY encoding the [-1,1] components along T, B, and Z containing an optional [0,1] multiplier of the scalar anisotropy.

We can also keep the anisotropy rotation angle, with the understanding that usage of this instead of a flow map can lead to interpolation artifacts. (We don't have the concept of non-texturable quantities at present).

@virtualzavie thoughts?

[virtualzavie]

As discussed in the meeting, I take it that the proposal is to replace the specular_rotation angle parameter with a vec2 parameter say (t,b), where the resulting rotated tangent vector is defined to be:

T′=tT+bB

with T, B the (assumed orthonormal) reference tangent and bitangent vectors (in world or local space).

Yes, that's exactly how I envision this.

Then we should require t2+b2=1 (we could not require that and instead say the vector needs to be normalized before use, but then adjacent points could use different normalizations and produce filtering artifacts).

This one is a bit tricky. I originally was wondering whether to have a normalised vector and a separate strength, or just having a single 2D vector whose norm would represent the strength, but one aspect makes me lean strongly toward the former. In case of a low resolution texture, the norm of the vector is going to altered by the texture filtering.

For that reason, I think it's safer to assume a normalised 2D vector indicating the direction of the flow, and a 1D anisotropy strength. Thus, the direction vector can be normalised in the shader to reduce artifacts introduced by filtering, just like is typically done for normal maps.

From the discussion, it also seemed to be desirable to have t,b be parametrized by values in [0,1], remapped internally to [-1,1] (so that the vec2 parameter can be supplied as the RG components of a (raw) RGB texture? Presumably that texture is the "flow-map"?). So then the default value would be (1, 0.5), meaning the tangent is unrotated.

Yes, that was my conclusion as well. I'm happy to see we independently came to the same conclusion on those various points.

Another thought, perhaps it could be convenient to keep the existing rotation angle parameter as well as the vec2 "flow map"?
[...]
This would cover the cases where the rotation angle is the preferred parametrization (for convenience say, despite the potential filtering issues). Obviously, if the flow map is supplied and the rotation left untouched, the filtering artifact would not occur.

I'm not sure I see the benefit of allowing two different parametrisations.
I can see why glTF found convenient to expose a uniform rotation parameter, but since we don't want exclusively uniform parameters, I don't think the benefit outweights the added complexity.

I'll draft a PR specifying this.

[elalish]

Agreed - at glTF we originally wanted to use a vec2 where the length was the strength, but upon implementation we found serious interpolation artifacts as you suggest, particularly bad when interpolating across large rotations, which end up with low intermediate strength values. It was more eye-catching than we expected.

[portsmouth]

I'm not sure I see the benefit of allowing two different parametrisations.
I can see why glTF found convenient to expose a uniform rotation parameter, but since we don't want exclusively uniform parameters, I don't think the benefit outweights the added complexity.

Though if we keep the existing specular_rotation angle (normalized to [0,1]) then the flow map would be additive to the existing model (and consistent with what we had e.g. in Standard Surface).

Also it seems potentially like it could be convenient for artists to be able to modify the angle directly (either as a constant/uniform, or texturing it), rather than having to work with a more complex flow-map parametrization. It's quite intuitive to change the angle and see the highlights rotating, and it seems to me a shame to remove that functionality. (We can just warn that if textured the angle may generate interpolation artifacts due to the discontinuity at 360°, and the flow-map is preferable in production to avoid that).

As noted, in the implementation it would trivially just be rotating the T' given by the flow map through an additional angle (defaulting to zero, so does nothing).

Seems worth considering at least.

[virtualzavie]

Proposal PR to address this issue: #170.

I'm also considering the following changes:

Specify anisotropy direction in terms of NDF

The current state of the specification states:

The tangent direction corresponds to the direction in which the highlights are elongated (equivalently, the surface grooves lie along the orthogonal bitangent).
[...]
specular_rotation [...] specify direction of the elongation of the specular highlight.

When possible we should probably specify in terms of properties of the BSDF rather than the result of shading. Moreover, although it is strictly equivalent from a technical point of view, from an intuition point of view it might make more sense to consider that the vector indicate the direction of the grooves.

Combining the two considerations, I think it would be more rigorous and less ambiguous to specify instead:

The tangent direction corresponds to the direction in which the NDF is stretched, meaning the resulting highlights tend to be elongated along the orthogonal bitangent.
[...]
specular_direction [...] specify direction along which the NDF is stretched."

Naming

We have:

• specular_roughness, specular_anisotropy, specular_direction (previously specular_rotation)
• coat_roughness, coat_anisotropy, coat_direction (previously coat_rotation)
• transmission_scatter, transmission_scatter_anisotropy
• subsurface_anisotropy

There seem to be a slight inconsistency here, where on one side anisotropy parameters are named as a property of the slab, and on the other side anisotropy parameters are named as a parameter to a property of a slab. Although more verbose, I would argue the latter is more rigorous.

So I would suggest something like:

• specular_roughness, specular_roughness_anisotropy, specular_roughness_direction
• coat_roughness, coat_roughness_anisotropy, coat_roughness_direction
• transmission_scatter, transmission_scatter_anisotropy (unchanged)
• subsurface_scatter_anisotropy

[virtualzavie]

By the way, it was suggested during the discussion to specify the vector as values in $[0, 1]$ with 0.5 being the origin, to better suit texture workflows. I notice my PR still specifies the vector as values in $[-1, 1]$.

What is the consensus here? I lean slightly toward a texture workflow friendly specification, but I certainly don't feel strongly about it either, as I see pros and cons to both.

[portsmouth]

The [0,1] convention seems best to me, as the flow maps can then be authored simply as the 8-bit RG channels of a PNG say, rather than requiring floating point textures. (Also more easily visualized as colors, than a float texture with negative values in it). Though it could be argued that [-1, 1] channels are more appropriate for this use case, even though it could be more cumbersome to work with.

[virtualzavie]

The lower texture format requirements is a very good point. Thanks.

[virtualzavie]

There's a bit of detail I have trouble wrapping my head around. Maybe you'll have some insight.
In the anisotropy mapping, I am proposing to specify the anisotropy direction with a flow map. I see a lot of similarities with a normal map, and for that reason, I would like as much as possible to align way the flow map is done, with the way the normal map is done.

In the spec, we mostly consider the normal map to be outside of the scope, and assume we are given a geometry_normal and a geometry_coat_normal, which I understand are the mesh normal that may or may not be perturbed by a normal map.
If I mimic this specification, I end up with a geometry_tangent and a geometry_coat_tangent, which are the mesh tangent that may or may not be perturbed by a flow map.

With that, I have everything I need for anisotropy: all that is left are the specular_roughness and specular_roughness_anisotropy.
What bothers me though is that it's a naming that I find a lot less clear than what I would have called specular_roughness_direction and coat_roughness_direction.

Another point that's bothering me, is that by 100% mimicking the normal map and letting the definition outside of the scope, I'm introducing unnecessary choices. It would be simpler to just define the flow map as a UNORM RG texture with values remapped to $[-1, 1]$.

Do you have thoughts on the topic?

[portsmouth]

We said in the "Normal maps" section:

The geometry_tangent may also be specified, which controls the direction of microfacet anisotropy, for effects such as brushed metal. The base and coat are assumed to share the tangent, which is orthonormalized accordingly.

So I think there is no need for a separate geometry_coat_tangent. I can see that in some special cases the coat and specular lobes might need to have different T directions via specular_roughness_direction and coat_roughness_direction (though seems a bit unlikely that a "brushed coat" is needed at all), but it would be total overkill to allow the coat to specify its own independent reference tangent vector, so it's fine to have a single geometry_tangent. (No functionality is lost really by having both spec and coat share the reference tangent).

In contrast, it makes sense to have a separate geometry_coat_normal as it would be fairly reasonable for the coat "scratches" to be independent from the underlying base scratches.

Can you clarify what naming you're unhappy with? I thought in #170 we had settled on the following, which seems fine (the consistency of the specular_roughess_ and coat_roughness_ prefixes are an improvement):

• specular_roughness
• specular_roughness_anisotropy
• specular_roughness_direction
• coat_roughness
• coat_roughness_anisotropy
• coat_roughness_direction

[portsmouth]

Another point that's bothering me, is that by 100% mimicking the normal map and letting the definition outside of the scope, I'm introducing unnecessary choices. It would be simpler to just define the flow map as a UNORM RG texture with values remapped to $[-1,1]$.

What do you mean by "by letting the definition outside of the scope, I'm introducing unnecessary choices"?

Do you want to not state a formula for how the flow map components affect the tangent, and leave that open to interpretation? That would be bad I think, since it's then not clear what to do in the implementation.

These formulas need to be explicitly stated, I think: #170 (comment)

[virtualzavie]

Thank you for your input.

Just to clarify, my thought is twofold.

My first observation is the anisotropy direction is just a modification of the local tangent space. If we have a perturbed geometry_tangent, then the specular_roughness_direction becomes completely redundant when it comes to expressing the anisotropy direction.

My second observation is that we can express that modification of the local tangent space very similarly to how we do with normal maps. Furthermore, I think we should take advantage of that similarity so we have a workflow that require less adaptation. I believe users used to thinking of normal maps as an RGB texture that paints the local normal, would gain from thinking of tangent maps as an RG texture that paints the local tangent direction.

Can you clarify what naming you're unhappy with?

If specular_roughness_direction is considered unnecessary in favour of geometry_tangent, that results in a naming that is less obvious. From an artist's perspective, geometry_tangent and geometry_coat_tangent are less clear than specular_roughness_direction and coat_roughness_direction.

What do you mean by "by letting the definition outside of the scope, I'm introducing unnecessary choices"?

If we consider the anisotropy direction to effectively be the geometry_tangent, then details of how to represent that tangent vector (i.e. anisotropy direction vector) are left completely up to the user. For normal maps we don't really have a choice, as there are already established workflows.

In the end, the choice is between:

• a) Consider the { geometry | geometry_coat }_tangent to be sufficient to represent the {specular | coat} anisotropy direction, and express it very similarly to normal maps. Consequently, consider {specular | coat}_roughness_direction unnecessary.
  This has the advantage of reusing the well known paradigm of normal mapping, but has the drawback of making the control parameter less obvious as its name seems disconnected from the intended effect (i.e. it wouldn't be obvious that the direction of the reflection is controlled by the geometry tangent).
• or b) Having a separate {specular | coat}_roughness_direction as we've discussed so far.
  This has the advantage of leaving fewer data format decisions to the user. It has several drawbacks as well: if the geometry tangent can be perturbed, the two are strictly redundant. Even if the geometry tangent cannot be perturbed, it feels unsatisfying that two similar concepts (maps perturbing a tangent reference space) are specified at very different levels (geometry on one side, material parameter on the other side).

Both approaches have their own tradeoff, and it's not clear to me which is preferable.

[virtualzavie]

Addendum: after discussing the choice with a couple of people, there are some additional arguments that make me lean more towards having a specular_roughness_anisotropy and unperturbed geometry_tangent.

The rationalisation to justify having the two maps described at different levels would be one of scale, as anisotropy is concerned with microgeometry, thus it doesn't belong to the "geometry" section, unlike the normal map.

[portsmouth]

OK I see what you mean. The flow-map parametrization we're discussing is analogous to a normal-mapping convention, but we don't try to pin down any particular such convention in that case (because there is no universal convention it seems). I agree actually, this flow-map idea seems out of scope to me, in a similar sense to normal maps. It seems similarly uncomfortable to be trying to pin down some particular option for the meaning of the components, when we don't even know if there are tools that generate the maps in the form we're assuming.

It would technically be fine to just assume that geometry_tangent specifies the tangent vector used to define the GGX stretch direction, and leave it out of scope as to how that tangent field is authored and modified. This obviously would be simpler for the spec.

Actually, it makes little sense to me to have both the geometry_tangent and the specular_roughness_direction. We say by default the geometry_tangent is the "unperturbed tangent field", which in practice comes from the asset geometry setup (e.g. via UV mapping). But what does it mean to allow for a different "perturbed" geometry_tangent field to be fed in, purely as the reference for the specular_roughness_direction -- in what case does that happen? (Perhaps during normal mapping, but in practice renderers take the perturbed normal and recompute the tangents internally, they don't take a "tangent map").

The rationalisation to justify having the two maps described at different levels would be one of scale, as anisotropy is concerned with microgeometry, thus it doesn't belong to the "geometry" section, unlike the normal map.

I disagree with this, as the tangent vector field is not about macro- or micro-geometry per se, it's just an arbitrary reference direction at every point on the surface, defining a local coordinate system. You can use it for whatever you want, e.g. stretching the NDF, texturing etc. It is more geometric than shading I suppose, since it is just a vector field associated with the geometry.

It seems to me you only need to specify the tangent vector field in one place -- geometry_tangent, and don't specify exactly how it was arrived at, just assume it is given. Either as a standard per-vertex attribute of the geometry, or further modified (e.g. by flow-map tools, which boil to just changing the input tangents).

We also already explicitly said that coat and spec share the tangent, used for the NDF:

The geometry_tangent may also be specified, which controls the direction of microfacet anisotropy, for effects such as brushed metal. The base and coat are assumed to share the tangent, which is orthonormalized accordingly.

So that was inconsistent with having the coat have its own rotation direction. If you think we really need to allow for separate stretch directions for coat and spec, suggest a plausible use case (I can't think of one).

So IMO, we should:

• omit specular_roughness_direction.
• omit coat_roughness_direction -- let's just assume stretch of coat and spec NDFs are in the same direction, though specular_roughness_anisotropy and coat_roughness_anisotropy may differ, obviously.
• geometry_tangent then is taken to mean the stretch direction of the NDF. That is the only reason for telling the shader to pick out some particular direction T in the plane orthogonal to the shading normal. How it picks that out is left to the renderer, e.g. have a node that takes a flow map and generates the world space tangent vector field for that. (But we don't define the parametrization for that, or concern ourselves with defining what a flow-map is). I imagine that usually, assets that need anisotropy for their shading will just bake the required tangent field into the per-vertex geometry, not e.g. ship with some weird flow-map textures.

So in summary, the NDF parametrization then consists entirely of:

• specular_roughness
• specular_roughness_anisotropy
• coat_roughness
• coat_roughness_anisotropy

and we can stop talking about flow maps. I like the sound of that.

What's your view on this:

Do we have a consensus on what to define as the highlight stretch direction, T or B? (Where the groove direction is then the opposite). As noted, I think highlight=B, groove=T is the most sensible. However, glTF I think does the opposite. So what do we do?

[virtualzavie]

It would technically be fine to just assume that geometry_tangent specifies the tangent vector used to define the GGX stretch direction, and leave it out of scope as to how that tangent field is authored and modified. This obviously would be simpler for the spec.

Yes, that seems sufficient, and simplicity is an argument for it. Moreover, nothing would prevent from adding a specular_roughness_direction on top of it if it appears necessary.
My concern however is that people will inevitably ask how to control the anisotropy direction. So the document should be explicit about it.

You can use it for whatever you want, e.g. stretching the NDF, texturing etc.

If the tangent is used for several things, the NDF should have its own tangent, as we may not want to link all of them to the same local tangent space. But in the context of OpenPBR, I can't think of anything but the NDF.

If you think we really need to allow for separate stretch directions for coat and spec, suggest a plausible use case (I can't think of one).

Having two different anisotropic specular lobes but having both of them use the same direction sounds surprising to me. In Adobe Standard Material, only the base layer specular is anisotropic. In Autodesk Standard Surface, it seems both the base layer and the coat have anisotropic reflection, and they have separate rotation parameters. Was this not used?
I can think of examples, like a scratched coating on top of carbon fiber, but I don't know if they're relevant to artists' needs.

Do we have a consensus on what to define as the highlight stretch direction, T or B? (Where the groove direction is then the opposite). As noted, I think highlight=B, groove=T is the most sensible. However, glTF I think does the opposite. So what do we do?

After talking with artists, I think it's irrelevant to the workflow.
Artists are going to pick the path of least resistance, and while in many cases that means painting the creases, in some cases that means painting the highlight elongation. Fortunately, changing from one behaviour to the other is just a matter of applying a 90° rotation, which is trivial with a vector map (in Substance Painter that's as simple as adding a layer).
In fact, sometimes neither will be convenient: for embroidery for example, the creases relevant to the NDF are not aligned with the threads (because of how strands are twisted). In that case, the artist can paint in the direction of the threads, then apply a rotation until the highlight looks coorect.

So we can either pick a side (I agree T for creases seems sensible), or try to align with an existing industry consensus if we identify any.

[portsmouth]

Having two different anisotropic specular lobes but having both of them use the same direction sounds surprising to me. In Adobe Standard Material, only the base layer specular is anisotropic. In Autodesk Standard Surface, it seems both the base layer and the coat have anisotropic reflection, and they have separate rotation parameters. Was this not used?
I can think of examples, like a scratched coating on top of carbon fiber, but I don't know if they're relevant to artists' needs.

If we really want separate anisotropy directions for coat and specular lobes, then obviously it's not sufficient to rely on geometry_tangent alone.

I suppose you could argue that it makes sense for geometry_tangent to specify the reference $\mathbf{T}$, $\mathbf{B}$ and for spec and coat to each have a flow-map vec2 input defining $\mathbf{T}'$, $\mathbf{B}'$ according to the formulas from #170 (comment). By default, the $\mathbf{T}'$ of coat and spec both equal the reference $\mathbf{T}$ (which might be pre-baked to contain the desired anisotropy direction for this asset). But they can be decoupled using the flow-map(s).

Just it's more elaborate and I'm not sure it's necessary.

[jstone-lucasfilm]

I don't have a strong preference either way, but I'll note that the open implementation of Autodesk Standard Surface supports independent rotations for specular and coat layers:

https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/libraries/bxdf/standard_surface.mtlx#L151
https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/libraries/bxdf/standard_surface.mtlx#L171

Our reference implementation of OpenPBR inherits support for these independent controls as well:

https://github.com/AcademySoftwareFoundation/OpenPBR/blob/main/reference/open_pbr_surface.mtlx#L133
https://github.com/AcademySoftwareFoundation/OpenPBR/blob/main/reference/open_pbr_surface.mtlx#L153

[portsmouth]

If we need to retain the independent base/coat lobe rotation, I'd like to suggest a third option:

• Have the existing geometry_tangent input mean the "reference" tangent field $T$, used by default for both coat and base. If not connected to anything, this reference tangent field is assumed to be the one given by the per-vertex geometry.

• Have a flow-map input for the coat rotation only, which defines the coat $T'$ relative to $T$. By default, $T' = T$, but coat can be decoupled if necessary via the flow-map.

The flow map components for the coat then have the easy to understand meaning as giving the relative rotation of base and coat anisotropy.

[virtualzavie]

Do we have a consensus on what to define as the highlight stretch direction, T or B? (Where the groove direction is then the opposite). As noted, I think highlight=B, groove=T is the most sensible. However, glTF I think does the opposite. So what do we do?

So we can either pick a side (I agree T for creases seems sensible), or try to align with an existing industry consensus if we identify any.

I went through the documentation of various tools, to try to see if there was a consensus.
I've identified several trends:

• Models that consider the NDF is stretched along T (glTF, BabylonJS, ASM).
• Models that consider the NDF is stretched along T if the anisotropy level is positive, along B if negative (Filament, Blender Principled BSDF, UE5 Substrate, Unity HDRP).
• Models that consider the NDF is stretched along B if the anisotropy level is positive, along T if negative (Blender, Cinema4D, UE4).
• Models that consider the NDF is stretched along B (UE3).

It looks like there's somewhat of a preference towards stretching the NDF along T, so I'm inclined to follow that trend, although I wouldn't object aligning the micro-grooves with T instead.

Hopefully I didn't confuse things. Another takeaway is that models often lack clarity on that topic; some don't even say what the reference is.