Skip to content

Lit meshes: specular highlights are wrong under a scaled ancestor #1636

Description

@obiot

What breaks

MeshBatcher.setPlacementUniforms computes uEyePosition — the camera's world position, which the lit shader needs for the specular half-vector — by extracting it from the view matrix as -Rᵀ·t:

const v = this.viewMatrix.val;
const ex = -(v[0] * v[12] + v[1] * v[13] + v[2] * v[14]);
// ...

That identity holds only when the upper 3×3 is orthonormal. But viewMatrix is renderer.currentTransform, which is not the camera's view alone — every ancestor's transform is folded into it on the way down the tree. One scaled ancestor and the extracted eye is simply a different point in space than the camera.

The game case

A lit 3D model displayed inside a container that has been scaled — the common ones being:

  • a model shown in an inventory or character panel, scaled to fit the frame
  • a diorama or miniature view scaled down as a whole
  • a grow/shrink powerup or squash-and-stretch animating scale on a group containing lit meshes
  • a zoom effect implemented by scaling a world container

The mesh renders in the right place at the right size — position comes from the model matrix, which is correct. What goes wrong is the shading: specular highlights sit where they would be if the camera were somewhere else entirely, so a glossy surface catches its highlight on the wrong part of the model, and it does not move correctly as the camera orbits. Diffuse lighting is unaffected; this is specular only. An unlit mesh is unaffected entirely.

Measured

The camera 600 back and 300 up, looking at the origin:

view extracted eye true eye
rigid 0, 300, 600 0, 300, 600
non-uniform scale on an axis with no translation 0, 300, 600 0, 300, 600
uniform scale 0.5 0, 150, 300 0, 600, 1200

Uniform scale is the worst case rather than the benign one: the extraction yields s · t where the truth is t / s, so the error is a factor of — at half scale the eye lands four times too close.

Measured against Matrix3d.invert() on the formula directly. I have not filmed the on-screen result; the visual description above follows from what uEyePosition feeds.

Fix

Invert the view properly rather than transposing it — Matrix3d already has invert(). It runs once per lit mesh, and the existing early-out (skip the GL call when the eye has not moved) already absorbs most of the cost, so the extra work lands on a path that mostly no-ops.

Related

The transparent pass had the same assumption in its sort key and shed it in #1635 (24b7c9ded) by pushing the position through the view instead of extracting the eye. That trick does not transfer here — the shader genuinely needs the eye's position, not a distance — so this one wants a real inverse. Raised in review of that PR and deliberately left out of it: it predates the PR, affects highlight direction rather than draw order, and needs a different fix.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions