Use of undeclared identifier 'va0' in fragment shader

[jasonsturges]

When I add a depth of field filter, c++, neko, and html5 targets throw an error compiling the fragment shader. Only Flash target works.

There's an example project to replicate this error at: openfl-away3d-filter3d

In that project, a depth of field filter is added:

import away3d.filters.DepthOfFieldFilter3D;

dofFilter = new DepthOfFieldFilter3D(32, 32);

Mac

When the fragment shader is compiled, it throws error:

Program3D.hx:332: fragment: ERROR: 0:18: Incompatible types (float and vec4) in assignment (and no available implicit conversion)
ERROR: 0:18: Use of undeclared identifier 'va0'

Error : Error compiling fragment shader: ERROR: 0:18: Incompatible types (float and vec4) in assignment (and no available implicit conversion)
ERROR: 0:18: Use of undeclared identifier 'va0'

In openfl.display3D.Program3D:

[openfl.display3D.Program3D] // AGAL fragment shader
#version 120
uniform vec4 fc0;
uniform vec4 fc1;
uniform vec4 fc2;
uniform vec4 fc3;
varying vec4 v0;
uniform sampler2D sampler0;
uniform sampler2D sampler1;
void main() {
    vec4 ft0;
    vec4 ft1;
    vec4 ft2;
    vec4 ft6;
    ft0 = texture2D(sampler1, v0.xy); // tex
    ft1.z = vec4(dot(vec4(ft0), vec4(fc3))).z; // dp4
    ft1.z = ft1.z - fc1.z; // sub
    ft1.z = vec4(1) / ft1.z, va0.x; // rcp (untested)

Call stack:

Called from openfl.display.Stage::__broadcastEvent openfl/display/Stage.hx line 1068
Called from openfl.display.DisplayObject::__dispatch openfl/display/DisplayObject.hx line 337
Called from openfl.events.EventDispatcher::__dispatchEvent openfl/events/EventDispatcher.hx line 218
Called from Away3dViewport::render Away3dViewport.hx line 124
Called from away3d.containers.View3D::render away3d/containers/View3D.hx line 771
Called from away3d.core.render.Filter3DRenderer::render away3d/core/render/Filter3DRenderer.hx line 150
Called from away3d.filters.tasks.Filter3DTaskBase::getProgram3D away3d/filters/tasks/Filter3DTaskBase.hx line 172
Called from away3d.filters.tasks.Filter3DTaskBase::updateProgram3D away3d/filters/tasks/Filter3DTaskBase.hx line 138
Called from openfl.display3D.Program3D::upload openfl/display3D/Program3D.hx line 80
Called from openfl.display3D.Program3D::__uploadFromGLSL openfl/display3D/Program3D.hx line 336

Neko

Uncaught exception - Error compiling fragment shader: ERROR: 0:18: Incompatible types (float and vec4) in assignment (and no available implicit conversion)
ERROR: 0:18: Use of undeclared identifier 'va0'

html5

Program3D.hx:365: program: WARNING: Could not find vertex shader attribute 'va0' to match BindAttributeLocation request.

OpenFlAway3dFilter3d.js:28250
Program3D.hx:365: program: WARNING: Could not find vertex shader attribute 'va0' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'va1' to match BindAttributeLocation request.

OpenFlAway3dFilter3d.js:28250
Program3D.hx:365: program: WARNING: Could not find vertex shader attribute 'va0' to match BindAttributeLocation request.

OpenFlAway3dFilter3d.js:28250
Program3D.hx:332: fragment: ERROR: 0:19: 'assign' : cannot convert from 'highp 4-component vector of float' to 'highp float'
ERROR: 0:19: 'va0' : undeclared identifier
ERROR: 0:19: 'x' : field selection requires structure or vector on left hand side

OpenFlAway3dFilter3d.js:70318
Uncaught openfl_errors_Error
errorID: 0
message: "Error compiling fragment shader: ERROR: 0:19: 'assign' : cannot convert from 'highp 4-component vector of float' to 'highp float'↵ERROR: 0:19: 'va0' : undeclared identifier ↵ERROR: 0:19: 'x' : field selection requires structure or vector on left hand side

[jasonsturges]

Josh indicated this may be related to: openfl/openfl@03e9c77

[mightymarcus]

Ah ok. But how to fix it?

[chfoo]

I ran into this problem too. It doesn't seem possible for the filter to have ever worked before outside of Flash. As of OpenFL 6.0.1, the line

"rcp ft1.z, ft1.z			\n" + // screenZ = -n*f/(d-f)

inFilter3DVDepthOfFFieldTask.hx and Filter3DHDepthOfFFieldTask.hx is translated to

ft1.z = vec4(1) / ft1.z; // rcp

There appears to be an attempt at getting it fixed on openfl/openfl#1613 but it still doesn't work since vect4 is there. I'm not familiar with shaders but based on the translated AGAL rcp instruction, we can just change it to a div instruction to workaround the problem. Since we just want to divide 1/z and luckily it passes 1.0 as a constant in the program data array (in the 12 index as fc3.x), we can substitute the rcp line in the Task files as:

div ft1.z, fc3.x, ft1.z

Doing that change seems to compile without error. One more thing, there is a carryover bug away3d/away3d-core-fp11#566 in the ActionScript source where you must set range in order for it to work. Otherwise it seems to be functional.

[mightymarcus]

Wow ...

[jgranick]

@chfoo Is there a sensible default we could use in the depth filter, to avoid away3d/away3d-core-fp11#566, do you think?

[chfoo]

The default value is fine, it's just that the original coder forgot to update the constructor in this commit: away3d/away3d-core-fp11@3474bab so _data[8] is incorrect until you do something like filter.range = filter.range.

[jgranick]

Thank you so much for your help!

I believe I have resolved the problem in our AGAL converter, as well as in Away3D (for the default depth value), but if you guys any further feedback or improvements, please send them our way 😀

5d3c3da