Shaders somehow behave differently in current master than march master, stereoscopy breaks

[mcclure]

Here is a test file I have:

local shader = lovr.graphics.newShader([[
    out vec3 lightDirection;
    out vec3 normalDirection;

    vec3 lightPosition = vec3(0, 3, 3);

    vec4 position(mat4 projection, mat4 transform, vec4 vertex) {
      vec4 vVertex = transform * vec4(lovrPosition, 1.);
      vec4 vLight = lovrView * vec4(lightPosition, 1.);

      lightDirection = normalize(vec3(vLight - vVertex));
      normalDirection = normalize(lovrNormalMatrix * lovrNormal);

      return projection * transform * vertex;
    }
  ]], [[
    in vec3 lightDirection;
    in vec3 normalDirection;

    vec3 cAmbient = vec3(.25);
    vec3 cDiffuse = vec3(.75);
    vec3 cSpecular = vec3(.35);

    vec4 color(vec4 graphicsColor, sampler2D image, vec2 uv) {
      float diffuse = max(dot(normalDirection, lightDirection), 0.);
      float specular = 0.;

      if (diffuse > 0.) {
        vec3 r = reflect(lightDirection, normalDirection);
        vec3 viewDirection = normalize(-vec3(gl_FragCoord));

        float specularAngle = max(dot(r, viewDirection), 0.);
        specular = pow(specularAngle, 5.);
      }

      vec3 cFinal = vec3(diffuse) * cDiffuse + vec3(specular) * cSpecular;
      cFinal = clamp(cFinal, cAmbient, vec3(1.));
      return vec4(cFinal, 1.) * graphicsColor * texture(image, uv);
    }
  ]])

local tim = 0

function lovr.load()
	lovr.graphics.setBackgroundColor(.8, .8, .8)
	lovr.headset.setClipDistance(0.1, 3000)
	--error("ok")
end

function lovr.update(dt)
	--print(dt)
	tim = tim + dt
  	for i, controller in ipairs(lovr.headset.getControllers()) do
  		if controller:isDown("touchpad") then
  			tim = 0
  		end
  	end
end

function lovr.draw()
	local gs = 30
	local far = 1*gs
	local grid = 2*gs
	for y=-grid,grid,gs do for z=-grid,grid,gs do
		lovr.graphics.line(-far, y, z, far, y, z)
		lovr.graphics.line(y, -far, z, y, -far, z)
		lovr.graphics.line(y, z, -far, y, z, far)
	end end

	lovr.graphics.setShader(shader)
	local count = 30
	for i=1,count do
		local stim = (tim * i) / count
		local stim1 = stim + 1
		lovr.graphics.translate(0, stim, -stim)
		lovr.graphics.rotate(stim, 0, 1, 0.1)
		lovr.graphics.scale(stim1, stim1, stim1)
		lovr.graphics.cube('fill', 0, 0, 0, .25)
	end
end

The "local shader =" value is taken exactly from a file shader.lua that i got off the lovr.org docs at some point. I use this file in multiple of my programs.

I build two copies of lovr. lovr-march is built from commit 58e59d9 Mar 3 2018. lovr-sept is built from 48dcb50 Sep 11 2018. In the test program many cubes spiral around the space. You can click the touchpad to restart the animation. I run my test program. I turn around 180 degrees and look "up and to the left" (so the upper southeast corner). Using lovr-march, this looks fine. Using lovr-sept, the cube shading looks "wrong", the stereoscopy is broken.

Expected behavior: Because the shader is not dependent on stereo instancing, lovr-march and lovr-sept should look the same. The fact they are different suggests some sort of bug.

[bjornbytes]

The problem is the gl_FragCoord. It's the window coordinates of the current pixel and doesn't make sense to use in the lighting equation.

There was a change in the rendering method that amplified the problems caused by this: old lovr rendered each eye to an single-eye-sized texture so vertices got roughly the same gl_FragCoord, but in new lovr the eyes are rendered side-by-side so the gl_FragCoord is very different between eyes.

To fix the shader, you can set viewDirection to be the real vector from the fragment to the camera. The calculations are done in view-space, so the camera is (0, 0, 0), and the vertex position can be sent in from the vertex shader. Here's an updated shader:

local shader = lovr.graphics.newShader([[
    out vec3 lightDirection;
    out vec3 normalDirection;
    out vec3 vertexPosition;

    vec3 lightPosition = vec3(0, 3, 3);

    vec4 position(mat4 projection, mat4 transform, vec4 vertex) {
      vec4 vVertex = transform * vec4(lovrPosition, 1.);
      vec4 vLight = lovrView * vec4(lightPosition, 1.);

      lightDirection = normalize(vec3(vLight - vVertex));
      normalDirection = normalize(lovrNormalMatrix * lovrNormal);
      vertexPosition = vVertex.xyz;

      return projection * transform * vertex;
    }
  ]], [[
    in vec3 lightDirection;
    in vec3 normalDirection;
    in vec3 vertexPosition;

    vec3 cAmbient = vec3(.25);
    vec3 cDiffuse = vec3(.75);
    vec3 cSpecular = vec3(.35);

    vec4 color(vec4 graphicsColor, sampler2D image, vec2 uv) {
      float diffuse = max(dot(normalDirection, lightDirection), 0.);
      float specular = 0.;

      if (diffuse > 0.) {
        vec3 r = reflect(lightDirection, normalDirection);
        vec3 viewDirection = normalize(-vertexPosition);

        float specularAngle = max(dot(r, viewDirection), 0.);
        specular = pow(specularAngle, 5.);
      }

      vec3 cFinal = vec3(diffuse) * cDiffuse + vec3(specular) * cSpecular;
      cFinal = clamp(cFinal, cAmbient, vec3(1.));
      return vec4(cFinal, 1.) * graphicsColor * texture(image, uv);
    }
  ]])

Gonna go backfill this fix to all the docs/examples that have the broken version...