Incompatible tangents are generated in models using a single UV "shell"

[CaffeineViking]

Hi!

We've recently started transitioning our engine (Northlight) to support per-pixel MikkTSpace. It has been mostly smooth sailing! We did run into a small issue when using the library in this repository though (which I think is the same one as http://mikktspace.com/).

We have the option to either use DCC exported tangents (e.g. from Blender) or generate them in our conversion pipeline. The ones coming from Blender are perfect, but the ones going through our conversion pipeline (which use this code) exhibit a warping effect. The reason why they are warped is because the vertex tangents that are being produced by our conversion pipeline don't match the ones coming out of Blender. See the screenshots below. On the left, you have the tangents from Blender, and on the right the ones generated from our conversion pipeline. Note that a few of them are fine, but some of them actually have two tangents produced per corner, when they should only have a single one. This is of course because of welding, but it's a bit surprising that these are not the same between Blender and our conversion pipeline. Our welding is just a hash of the vertex attributes, so nothing super fancy.

 

Both of these use the same normal map, baked from Substance (if I'm not wrong). We did a bit of investigation, and we came to the conclusion that it's somehow related to UV shells. If you use multiple UV shells (like the one on the left) our converter and Blender's agree completely, and we don't get any warping (we do of course get multiple tangents per corner in this case). However, if we use a single UV shell (like the one on the right) that's when we get incompatible tangents generated from this library and Blender. We've also tried exporting from Modo, and it gives us the same results as Blender. I'm not sure if this type of setup is an edge-case or not?

 

Since Blender uses the "same" mikktspace.c implementation, I downloaded the source code and stepped through the code to see where things were different. This isn't the root of the problem, but it's the first location where I could see that something seemed fishy. In BuildNeighborsFast the pEdges which are produced are completely different between this implementation (on the right) and Blender (on the left). Actually, the ones coming out from there don't seem to make much sense, they are edges connected via the same vertex! So something with building the neighborhood connectivity doesn't seem quite right, which would explain why it can't weld things properly, since it doesn't know which vertices are the actual neighbors! When looking closer at the Blender code, some functions have been re-written. My hunch is that the welding code in GenerateSharedVerticesIndexList was fixed in Blender.

 

This might seem a bit anticlimactic, but we simply switched over to Blender's mikktspace.c and that seemed to fix our problems. I would've liked to figure out what was the actual problem and submit a patch, but I couldn't find anything obviously out of place in here. So my question is if there could be some sort of edge-case in this implementation? Or rather, if this type of weird "single UV shell setup" was even supposed to work properly in MikkTSpace? This seems to be a pretty extreme test, so I'm not sure if any real content would ever be built like this. Below is the screenshot of our test scene, where we've swapped to Blender's implementation.

Here are the test assets we've used: mikkt_tests.zip. We assign a perfect mirror material and add a environment probe to the scene.

I think it would be worth fixing this edge-case, or integrating the changes from Blender. I will try to take another stab at this at some point, but in the meantime, is anybody else able to reproduce these issues, or is it only us? Are there any hunches what it could be?

Cheers!

[mmikk]

Thanks for reaching out. Before I dig in just a quick thing to confirm. If you triangulate the mesh in blender first (you can use the modifier stack so it’s not destructive), then do the bake, then export this version of the mesh so you bring the same triangulation of the mesh into your engine. Does the problem go away? Similar to Gouraud shading how you tringulate your polygons has an impact on your interpolated vertex attributes.

Morten

[mmikk]

The two copies of mikktspace should definitely generate identical results. Ill take a look at that

[CaffeineViking]

Thanks for the lighting fast reply! I'm pretty sure that they were triangulated before baking/export (I'll double-check with our artist).

[CaffeineViking]

Actually, the warping is a bit hard to see in my earlier screenshots, here's a short video we did at some point to compare the results:

mikkt_tests_f_game_tangent_export_2021_03_24.mp4

The face without any kind of warping corresponds to the case where tangents between Blender and our converter were matching:

 

• No Tangents Exported: tangent vertex attributes were generated through our converter (which was using this implementation).
• Tangents Exported: we use the tangent data directly from the FBX we get from Blender (i.e. using Tangent Space export option).

[CaffeineViking]

Yeah, they should've been per-triangulated in Blender according to our artist. I opened them in Blender and they do indeed seem to be proper. The normal map was baked in Substance. Contained within mikkt_tests.zip are our test assets. Here's a short description:

• mikkt_tests_01_blender_high.fbx: high-polygon ground truth made in Blender (pretty much a cube with some rounded edges).
• mikkt_tests_01_blender_n.tga: baked per-pixel MikkTSpace normal map for a single UV shell (i.e. for the two FBX files below).
• mikkt_tests_01_blender_tangents_exported.fbx: low-poly single UV shell version of the asset, tangents exported from Blender.
• mikkt_tests_01_blender_no_tangents_exported.fbx: low-poly single UV shell, tangents need to be generated through our code.
• mikkt_tests_01_blender_separate_shells_n.tga: baked per-pixel MikkTSpace normal map with multiple UV islands (see below).
• mikkt_tests_01_blender_tangents_exported_separate_shells.fbx: low-poly multiple UV islands, tangents exported from Blender.
• mikkt_tests_01_blender_no_tangents_exported_separate_shells.fbx: low-poly multi-UV islands, tangents need to be generated.

Artists have been trying to follow this guide: Generating Perfect Normal Maps for Unity, but maybe we're still missing something. I'll also try to do a bit more digging later this week and see if I can pin down what's wrong, since I still have both versions of the code.

[mmikk]

bakes.zip

Could you try these two? It's a bake from xnormal and one from knald. Neither one uses Blender's version of the .c file but both are working as intended. What happens when you use these in your engine?

Btw this was baked to mikkt_tests_01_blender_no_tangents_exported.fbx

[CaffeineViking]

Hey Morten, thanks for checking that out for us! I downloaded those bakes and proceeded to try them in our engine with that asset:

 

I think the left one was the one from Knald and the one on the right is the one from xNormal, notice the warping in that closest one.

Since it sounded like it worked on your side that got me thinking what could be different between our implementation and the one in this repository (and why it worked when we pulled the one from Blender). Well, it turns out that it was something embarrassing... Our compiler is setup to treat warnings as errors, even in middleware code (which is kind of stupid to begin with), so when I ported over mikktspace.c we got quite a few warnings that I needed to fix. I quickly hacked that together and didn't think too much of it.

Well, it turns out when doing vimdiff between those changes and the code here reveals a pretty stupid change I didn't notice I did:

I didn't actually notice that was a union before double checking now, so that explains the weird edge connections I was getting 😅. Instead of fixing those compilation warnings, I just pulled this repository again and disabled warnings with a few pragma directives. And yeah, after that, the results are completely perfect xD (not so surprisingly). So there is absolutely nothing wrong with the code here, it was just me being too carefree when patching out the warnings and not thinking too much about it afterwards... **sigh**

Thank you very much for your patience and help Morten! I think the mystery has been solved on our side at least 😅. Closing issue!

[mmikk]

No problem. I'm glad it was an easy fix for you :)

[perkele1989]

After reading the tech deep dive of Northlight / Alan Wake 2 today, I've been feeling that impostor syndrome. You've really done an amazing job on that engine, twisting the GPU to do your bidding for you in ways I couldn't have imagined. Very impressive.

Then I found this issue while integrating mikktspace in my engine, and realize it's one of you guys. Instantly felt much better lol. What are the odds? Great work on Northlight, you are an inspiration.

And sorry for off-topic.