Debug normal mapping

[smcameron]

I have made a stab at doing normal mapping of planets. However, it's got some problems.

You can see the current state by checking out the "debug-normal-mapping" branch.

git clone git@github.com:smcameron/space-nerds-in-space.git
cd space-nerds-in-space
git checkout debug-normal-mapping

I am computing the per-vertex tangents and bitangents according to the method described here:
http://www.iquilezles.org/www/articles/patchedsphere/patchedsphere.htm (or trying to, anyway.)
Another one I've been looking at: http://www.gamasutra.com/blogs/RobertBasler/20131122/205462/Three_Normal_Mapping_Techniques_Explained_For_the_Mathematically_Uninclined.php?print=1

That only shows how to calculate them for one face of a cube. I have tried to extrapolate this for the remaining five faces. The code for the calculation shown on the iquilezles.org site is in quat.c, near the bottom in https://github.com/smcameron/space-nerds-in-space/blob/debug-normal-mapping/quat.c#L938 cubemapped_sphere_tangent_and_bitangent(). The code that calls this for each vertex of a subdivided icosohedron is in mesh.c, in https://github.com/smcameron/space-nerds-in-space/blob/debug-normal-mapping/mesh.c#L439 mesh_set_spherical_cubemap_tangent_and_bitangent().
(I will have to change from using a subdivided icosohedron to six subdivided normalized quads with duplicated vertices at the edges to get the normal map interpolation to work correctly at the boundaries at some point, but, one thing at a time.)

If you want to try this yourself:

1. Download this image as a normal map (from here, which is an interesting blog post, btw http://dansinteractive.blogspot.com/2012/02/normal-maps-of-coins-part-2.html)
   
   (or use some other normal map.) Scale the image to 1024x1024 and save it as a png file. (has to be square, and has to be a png file.)
   Then copy the image six times to:

normal0.png
normal1.png
normal2.png
normal3.png
normal4.png
normal5.png

1. Then, build mesh_viewer:

   make mesh_viewer

2. Then invoke mesh_viewer like so:

   mesh_viewer -p share/snis/textures/test -n normal

The mouse-wheel will zoom in and out, and dragging with the right mouse button clicked will rotate the sphere. ('s' key will toggle free spinning off and on). dragging with right mouse button clicked and CTRL key held down with move the light.

If you want to see how the six images map to the sphere, check out "master" branch, and run
mesh_viewer -p share/snis/textures/test

This will show a sphere with six images plainly numbers "0", "1", "2", "3", "4", "5" and "left", "right", "top" and "bottom" on each one so you can see how they are arranged, like so:

You can verify the orientation of axes and so on by:
make share/snis/models/axis.stl
mesh_viewer share/snis/models/axis.stl

The shaders (both vertex and fragment) is here:
https://github.com/smcameron/space-nerds-in-space/blob/debug-normal-mapping/share/snis/shader/textured-cubemap-and-lit-per-pixel.shader

If you run it, you'll see a sphere, something like this:

Press the 'a' key to toggle the atmosphere off.

So far so good. but then rotate the sphere a bit and...

Note the "red" on the forehead in the 2nd image. That' should not be red.

Initially the camera position is such that:
+x is straight out the screen
+z is to the left
+y is up.
Rotating the image makes things a bit more traditional.

So I have banged my head against this for a while and I am not really getting anywhere.

[smcameron]

Maybe my tangent and bitangent calculations are wrong? In this picture, normal, tangent and bitangent are red, green and blue, respectively. Should the tangent and bitangents be, well, tangent to the surface of the sphere? Clearly, they aren't. But, then they are supposedly in "tangent space", so I am not quite sure what they should look like. At least the normals look right.

To see an image like this, run mesh_viewer:

mesh_viewer -p share/snis/textures/test -n normal -i 3

(The "-i 3" makes it subdivide the icosohedron only 3 times so there aren't quite so many vertices.) Press 'a' to turn off the atmosphere. Press 'r' to change the rendering to black triangles outlined with lines. Press "pause/break" to toggle on the debug menu, and select the first item to enable drawing normals/tangents/bitangents as r/g/b lines.

http://i.imgur.com/OPJkAdP.jpg

[smcameron]

Note for anyone trying this:

The code to tint things red green and blue is in the shader, in share/snis/shader/textured-cubemap-and-lit-per-pixel.shader. It's for trying to debug... the per-pixel normal's x,y,z is mapped to r,g,b (at 85%, 15% is diffuse lighting). The following change will make the lighting more "ordinary". You can right-drag the mouse while holding down the CTRL key to move the light around.

diff --git a/share/snis/shader/textured-cubemap-and-lit-per-pixel.shader b/share/snis/shader/textured-cubemap-and-lit-per-pixel.shader
index 407d496..9344fc9 100644
--- a/share/snis/shader/textured-cubemap-and-lit-per-pixel.shader
+++ b/share/snis/shader/textured-cubemap-and-lit-per-pixel.shader
@@ -108,8 +108,8 @@ varying mat3 tbn;
                gl_FragColor = textureCube(u_AlbedoTex, v_TexCoord);

                gl_FragColor.rgb *= diffuse;
-               gl_FragColor.rgb *= diffuse * 0.15;
-               gl_FragColor.rgb += 0.85 * pixel_normal;
+               //gl_FragColor.rgb *= diffuse * 0.15;
+               //gl_FragColor.rgb += 0.85 * pixel_normal;
                //gl_FragColor.rgb = pixel_normal;

                /* tint with alpha pre multiply */

With that change, you'll get images like this (which appears to be more or less correct)

However, if you rotate the model 180 degrees about the axis coming out of the screen, the lighting is now incorrect as the light moves on the horizontal axis:

[smcameron]

Ok, I have implemented a "spherified cube" mesh rather than the subdivided icosohedron, and I have come up with an empirical way to compute the tangents, bitangents and normals, which seems correct, (however I think I have run headlong into the "Hairy Ball Theorem") and things still don't act right in the shader... but progress, of a sort.

[smcameron]

Ok, tentatively, as of commit e05c4ff I think maybe it's working:

https://www.youtube.com/watch?v=GbBlPy2lpcA

[smcameron]

Alright, I found a bug in the code that generates the normal map images in earthlike.c, and after tinkering with it a bit I am pretty sure it is all working very nicely now on the debug-normal-mapping branch.

[smcameron]

Fixed as of commit de28079