VideoBackends: Use proper floating point depth precision. #2369

CrossVR · 2015-05-05T21:36:58Z

Still doing some accuracy tests, but it seems promising that this will fix our off-by-one inaccuracies.

comex · 2015-05-05T21:38:21Z

Can you explain in more detail what this does?

CrossVR · 2015-05-05T21:41:54Z

Hardware tests showed that our depth values are off-by-one in the hardware video backends: https://gist.github.com/Armada651/bab3e023b5b813e18cb7#comment-1446986

Using a graphics analyzer I could see that we are accurate to the point where we actually store the depth value. According to Fiora this inaccuracy was introduced by the divisor so I changed that after which the D3D backend passes the hardware test. The OGL backend still fails the test though, but now in the other direction with a value of 13884591.

but you can choose "correct range" or "correct precision", but not both

FioraAeterna · 2015-05-05T21:43:27Z

It's a problem of "correct range, correct precision, but not both".

The only way to losslessly store a 24-bit integer into a 32-bit float is to multiply it by a power of 2 before conversion (or no multiply at all). 0xFFFFFF isn't a power of 2, so error will occur.

This unfortunately also means "max depth" (0xFFFFFF) won't actually map to 1.0. I don't know enough about the graphics stack to know if this matters.

CrossVR · 2015-05-05T21:51:02Z

Confirmed to fix pokemon snap in D3D, this PR currently breaks OGL because it's now off-by-one in the other direction.

CrossVR · 2015-05-05T22:48:09Z

@degasus Since the OGL backend reads the depth buffer as integers this is a bit difficult to fix. It may be fixed if we just use glClipControl(), but then we'd still need to implement a hack for older versions of OpenGL.

I could also switch the OGL backend to use floating points, but that's a bit of a shame since integers more accurately reflect the console.

Another option is to switch back to the old divisor and add or subtract one where needed like we do with the epsilon hack, but I don't really like adding more epsilons to our code.

Source/Core/VideoBackends/D3D/PixelShaderCache.cpp

@@ -146,7 +146,7 @@ const char depth_matrix_program[] = {
 	" in float4 pos : SV_Position,\n"
 	" in float3 uv0 : TEXCOORD0){\n"
 	"	float4 texcol = Tex0.Sample(samp0,uv0);\n"
-	"	int depth = int(round(texcol.x * float(0xFFFFFF)));\n"
+	"	int depth = int(round(texcol.x * 16777216.0));\n"


Source/Core/VideoBackends/OGL/FramebufferManager.cpp

@@ -89,7 +89,7 @@ FramebufferManager::FramebufferManager(int targetWidth, int targetHeight, int ms
 		glTexParameteri(m_textureType, GL_TEXTURE_MAX_LEVEL, 0);
 		glTexParameteri(m_textureType, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
 		glTexParameteri(m_textureType, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
-		glTexImage3D(m_textureType, 0, GL_DEPTH_COMPONENT24, m_targetWidth, m_targetHeight, m_EFBLayers, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_INT, nullptr);
+		glTexImage3D(m_textureType, 0, GL_DEPTH_COMPONENT24, m_targetWidth, m_targetHeight, m_EFBLayers, 0, GL_DEPTH_COMPONENT, GL_FLOAT, nullptr);


CrossVR · 2015-05-06T22:44:37Z

@degasus Ready for review.

Do you still want to switch the D3D depth buffer from DXGI_FORMAT_R24G8_TYPELESS to DXGI_FORMAT_R32_FLOAT?

Source/Core/VideoCommon/PixelShaderGen.cpp

@@ -1224,11 +1224,11 @@ static inline void WritePerPixelDepth(T& out, pixel_shader_uid_data* uid_data, A
 		if (ApiType == API_OPENGL)
 			out.Write("\tscreenpos.y = %i - screenpos.y;\n", EFB_HEIGHT);

-		out.Write("\tdepth = float(" I_ZSLOPE".z + " I_ZSLOPE".x * screenpos.x + " I_ZSLOPE".y * screenpos.y) / float(0xFFFFFF);\n");
+		out.Write("\tdepth = float(" I_ZSLOPE".z + " I_ZSLOPE".x * screenpos.x + " I_ZSLOPE".y * screenpos.y) / 16777216.0;\n");


degasus · 2015-05-07T06:17:10Z

@Armada651 If there are no issues, we should use the same format on both backends.

PatrickFerry · 2015-05-08T01:52:00Z

The sky is black in the simpsons-tev above.

CrossVR · 2015-05-08T08:00:57Z

@ZephyrSurfer We're aware of the regression, but we're probably going to merge it without fixing that bug. Since the sky in simpsons-tev is also black on the Software backend and that game has never worked correctly in Dolphin anyway we don't see a need to fix it right now.

…t instead or rounding.

degasus · 2015-05-08T12:35:01Z

LGTM, but please wait for fifoci ;)

degasus · 2015-05-08T13:01:22Z

@Armada651: This comment grants you the permission to merge this pull request whenever you think it is ready. After addressing the remaining comments, click this link to merge.

@dolphin-emu-bot allowmerge

VideoBackends: Use proper floating point depth precision.

dolphin-emu-bot · 2015-05-09T19:58:01Z

FifoCI detected that this change impacts graphical rendering. Here are the behavior differences detected by the system:

ed-lighting on ogl-lin-mesa: diff
fog-adj on ogl-lin-mesa: diff
fortune-street on ogl-lin-mesa: diff
fortune-street-fog on ogl-lin-mesa: diff
fortune-street-white-box on ogl-lin-mesa: diff
kirby-shadows on ogl-lin-mesa: diff
mario-tennis-menu on ogl-lin-mesa: diff
megaman-heat on ogl-lin-mesa: diff
melee-lighting on ogl-lin-mesa: diff
mini-ninjas on ogl-lin-mesa: diff
mkdd-efb on ogl-lin-mesa: diff
mkwii-bluebox on ogl-lin-mesa: diff
nfsu-reflections on ogl-lin-mesa: diff
nsmbw-intro on ogl-lin-mesa: diff
rs2-zfreeze on ogl-lin-mesa: diff
sadx-ui on ogl-lin-mesa: diff
sf-assault-flashing on ogl-lin-mesa: diff
simpsons-tev on ogl-lin-mesa: diff
soa-black on ogl-lin-mesa: diff
ss-timestone on ogl-lin-mesa: diff
tsp3-pinkgrass on ogl-lin-mesa: diff
xenoblade-menu on ogl-lin-mesa: diff
ztp-grass on ogl-lin-mesa: diff
zww-armos on ogl-lin-mesa: diff
ed-lighting on ogl-lin-nv: diff
fifa-street on ogl-lin-nv: diff
fog-adj on ogl-lin-nv: diff
fortune-street on ogl-lin-nv: diff
fortune-street-fog on ogl-lin-nv: diff
fortune-street-white-box on ogl-lin-nv: diff
kirby-shadows on ogl-lin-nv: diff
mario-tennis-menu on ogl-lin-nv: diff
megaman-heat on ogl-lin-nv: diff
melee-lighting on ogl-lin-nv: diff
mini-ninjas on ogl-lin-nv: diff
mkdd-efb on ogl-lin-nv: diff
mkwii-bluebox on ogl-lin-nv: diff
nfsu-reflections on ogl-lin-nv: diff
nsmbw-intro on ogl-lin-nv: diff
rs2-zfreeze on ogl-lin-nv: diff
sf-assault-flashing on ogl-lin-nv: diff
simpsons-tev on ogl-lin-nv: diff
smg2-fog on ogl-lin-nv: diff
soa-black on ogl-lin-nv: diff
ss-timestone on ogl-lin-nv: diff
tsp3-pinkgrass on ogl-lin-nv: diff
xenoblade-menu on ogl-lin-nv: diff
ztp-grass on ogl-lin-nv: diff
zww-armos on ogl-lin-nv: diff
zww-water on ogl-lin-nv: diff
zww-waves on ogl-lin-nv: diff

_{^{automated-fifoci-reporter}}

thegrb93 · 2015-05-10T00:38:00Z

Some good and some bad.

This looks nice.
https://fifoci.dolphin-emu.org/compare/286655-281290/

Some broken ones.
https://fifoci.dolphin-emu.org/compare/277017-276312/
https://fifoci.dolphin-emu.org/compare/286660-281278/

BhaaLseN · 2015-05-10T15:37:48Z

Too bad Windows/DX builder was stuck, so we didn't get this one earlier. Simpsons however was more or less accepted as broken, since it never worked correctly. Def Jam however is odd.

CrossVR changed the title ~~VideoBackends: Use proper floating point depth percision.~~ VideoBackends: Use proper floating point depth precision. May 5, 2015

CrossVR force-pushed the depth-percision branch from c8a2f6b to 9fb0027 Compare May 5, 2015 22:53

degasus reviewed May 6, 2015
View reviewed changes

CrossVR force-pushed the depth-percision branch from 9d4401c to 418b62c Compare May 6, 2015 19:34

degasus reviewed May 6, 2015
View reviewed changes

degasus reviewed May 7, 2015
View reviewed changes

CrossVR force-pushed the depth-percision branch 3 times, most recently from 96a466b to de265c7 Compare May 7, 2015 22:28

CrossVR added 10 commits May 8, 2015 14:29

VideoBackends: Use proper floating point depth precision.

c4f85a3

OGL: Change the depth buffer type to GL_FLOAT.

4b2e04b

OGL: Switch depth buffers to GL_DEPTH_COMPONENT32F format.

be810eb

VideoBackends: Floor depth values in depth copy shaders.

b0770e2

VideoCommon: Clamp integer conversions.

0f2c72f

VideoBackends: Clamp Z peek values.

1a409a2

VideoBackends: Use the new divisor when clearing the depth buffer.

84a5f4a

D3D: Use a 32-bit floating point depth buffer.

a224c60

PixelShaderGen: Use new multiplier everywhere and directly cast to in…

5dbb43a

…t instead or rounding.

TextureConversionShader: Use floating point values in clamp().

d04af15

CrossVR force-pushed the depth-percision branch from b3b4188 to 4e98cd3 Compare May 8, 2015 12:32

PixelShaderGen: Clamp zCoord to the depth range.

1d745d6

CrossVR force-pushed the depth-percision branch from 4e98cd3 to 1d745d6 Compare May 8, 2015 12:47

dolphin-emu-bot added a commit that referenced this pull request May 8, 2015

Merge pull request #2369 from Armada651/depth-percision

ca7fe09

VideoBackends: Use proper floating point depth precision.

dolphin-emu-bot merged commit ca7fe09 into dolphin-emu:master May 8, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VideoBackends: Use proper floating point depth precision. #2369

VideoBackends: Use proper floating point depth precision. #2369

CrossVR commented May 5, 2015

comex commented May 5, 2015

CrossVR commented May 5, 2015

FioraAeterna commented May 5, 2015

CrossVR commented May 5, 2015

CrossVR commented May 5, 2015

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

CrossVR commented May 6, 2015

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

degasus commented May 7, 2015

PatrickFerry commented May 8, 2015

CrossVR commented May 8, 2015

degasus commented May 8, 2015

degasus commented May 8, 2015

dolphin-emu-bot commented May 9, 2015

thegrb93 commented May 10, 2015

BhaaLseN commented May 10, 2015

VideoBackends: Use proper floating point depth precision. #2369

VideoBackends: Use proper floating point depth precision. #2369

Conversation

CrossVR commented May 5, 2015

comex commented May 5, 2015

CrossVR commented May 5, 2015

FioraAeterna commented May 5, 2015

CrossVR commented May 5, 2015

CrossVR commented May 5, 2015

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

CrossVR commented May 6, 2015

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

degasus commented May 7, 2015

PatrickFerry commented May 8, 2015

CrossVR commented May 8, 2015

degasus commented May 8, 2015

degasus commented May 8, 2015

dolphin-emu-bot commented May 9, 2015

thegrb93 commented May 10, 2015

BhaaLseN commented May 10, 2015