HLSL losing vertex elements?

[ed022]

I posted this to the community forums after staring at it for several days... I think I've ruled out the possibility of it being something foolish on my end.

http://community.monogame.net/t/hlsl-losing-vertex-elements/7724/

In summary, one call to DrawUserIndexedPrimitives seems to be affecting another several calls later, despite setting a different render target and clearing the graphics device several times in between, etc. For what it's worth, they're both acting upon different techniques in the same effect, although it's not EffectPass.Apply that's causing the issue. Oddly, the vertex shader of the later call in question seems to have all of its vertex elements - except for Position - set to zero, despite the incoming vertex data.

I tried downloading the Monogame source to debug locally, but I had issues compiling it, probably related to reference/external dependencies.

[ed022]

In case it helps track the bug, everything works properly if I compile the techniques to target different versions of the vertex shader. I'd been using vs_2_0 heretofore; if I change one of them to vs_3_0, it works, but if I change two to vs_3_0, then the second of which experiences the problem described above still.

[tomspilman]

I'll try to give this a look this week and see what I find.

[ed022]

Any news on this? I just ran into it again in a fresh new project. It's quite frustrating. =(
It seems to be related to changing techniques; if I just use one technique ever, it's just fine, but if I switch, then it seems to zero out these incoming vertex elements in the subsequent techniques.

[ed022]

I just tried making clones of the effect, each of which to use just one technique each, but that had the same results; the vertex elements other than Position are all zero when they reach the vertex shader.

[tomspilman]

Sorry for the slow progress on this.

If you can pull together a simple test project that exhibits the issue it would make it much easier for one of us to investigate it closely.

I've been swamped with work for the last few months otherwise I would have already investigated it.

[ed022]

Here you go. I draw your attention to the Game1.Draw method; I'm drawing a cube that should be rendered with four different techniques - ambient light, directional light, point light, and one that just shows the normals. They should be blended additively, but as one can see by commenting out the individual DrawUserIndexedPrimitives lines, only certain combinations appear; if the ambient technique has its DrawUserIndexedPrimitives called, all the others receive <0, 0, 0> as their vertex normals and therefore are rendered as black. (You can take out the vertex shader's normal transformation, leaving it the same as it came in, and the same thing happens, so that's not the issue.)
ShaderDemo.zip

[ed022]

Is it possible that the normals are being optimized out? The ambient light technique does not use them, but the others do. Yet, as stated above, if the ambient light is used, then none of the others work as they don't receive the normals data.

[ed022]

Here's some more evidence to further the mystery... As mentioned above, if the ambient light is used, then the other lights don't work... but only if the ambient light is first. If one of the other two (which use the normals data) is first, then all of the light techniques - including the ambient one - work just fine. Furthermore - and this is the odd bit - what seems to matter is the order of the techniques as of their first use; if I reorder them during the course of the program, it won't change whether or not the others work.

[ed022]

I hate to pester, but, any updates on this? I'm reluctant to continue working on this project as long as this error persists.

[ed022]

Ok, I think I found the issue, after stepping through the Monogame code via debugging.

Much as the symptoms above illustrate, basically, the VertexDeclaration is caching its attributes upon its first call of GraphicsDevice.ApplyState, which it stores in a dictionary under the vertex shader's hash code. In caching the attributes, any that are not used in that particular shader program (vertex and pixel shader combination) are marked as location -1, which then get dropped upon subsequent calls to VertexDeclaration.Apply. The problem, of course, is that this vertex declaration cache assumes it's still that same combination of vertex/pixel shaders, which may well not be the case, as it only looks up the cache by vertex shader hash code.

On my local copy of Monogame, I changed the following three (disjointed) lines of VertexDeclaration.Apply as follows:
internal void Apply(Shader vshader, Shader pshader, IntPtr offset)
int shaderHash = vshader.GetHashCode() ^ pshader.GetHashCode();
var attributeLocation = vshader.GetAttribLocation(ve.VertexElementUsage, ve.UsageIndex);
Note that this only requires the pshader parameter for hashing purposes.

An alternate approach would be to take the shader program instead of the two shader individually, but I don't know offhand how to access the vertex shader specifically from within.

I do hope that this investigation helps with the development and debugging of this project.

[Jjagg]

Great job figuring this out @ed022! Sorry for the lack of response you got here :/
I have just set up a PR with the fix. Thanks for the help :)