Texture array sometimes appears black in RD, but still draws

[Saftur]

Description

To be precise, it draws in my app just fine, and it even shows up in the capture preview, however, once I load the capture and check the texture viewer, the texture array in question is completely black, and the backbuffer color output looks like what the app would look like if that texture array was all black.

Here's what my app's screen looks like:

And here's what the backbuffer color output looks like when the texture becomes black:

The black version only shows up in RD, the app itself never shows up like that.

The texture array in question is my skybox texture array. Only my skybox object uses it, but both of the other textured objects visible in the first image are actually using an environment cube map I'm rendering, which is why they are also black in the second picture.

It might be an error with my own GL calls, but I don't see how it would work normally, but disappear in RD.

Another detail is that if I actually capture the frame in which the texture array gets created before capturing a random frame later on, there's a much higher chance the texture won't be full black in the random capture. And actually, even just capturing two random frames appears to give it a higher chance of not becoming full black.

I've tried commenting out the parts of my program where I was using the texture, so that I only loaded the texture, and then never touched it again, but it still became black. I thought it might've had to do with the way I was binding textures to different binding points, but when I simplified that part such that there was no possible way for it to cause an issue, it was still black.

Steps to reproduce

I've emailed you two captures, one where it shows up black, another where it doesn't. Both of them are captures at a random frame on two different runs. I ran it in the exact same way both times, and I did not recompile in between.

The texture array in question is labelled "Skybox Texture", so it should be easy to find in the resource inspector.

I'm not sure what other steps I can give here. The texture & backbuffer are black in one capture, and not black in the other, and both can be checked in the texture viewer.

Environment

• RenderDoc version: 1.10
• Operating System: Windows 10 Home; Version 2004
• Graphics API: OpenGL
• GPU: GeForce GTX 1060
• OpenGL Version: 4.6.0 NVIDIA 456.71
• OpenGL Driver Version: Geforce Drivers 456.71; 9-30-2020; Version86.6.59.0.c

[baldurk]

Unfortunately I can't tell from these captures exactly what went wrong. The snapshot of the skybox texture's contents that was taken at the start of the frame seems to contain data in one and not in the other, but why exactly that happened is not clear from the capture. The texture itself is over 400MB just for one mip since it's a 2048x2048 cubemap and it's 32-bit float per channel, so my only wild guess is some kind of out-of-memory error.

Are you able to share the application itself so I can debug the capture-time process? If not could you at least share the renderdoc diagnostic log from when a failed capture is created in case it contains any useful information?

[Saftur]

I've just sent a zip file with what should be everything needed to run the app to you by email. Excuse the weird file structure, that's just how it is right now because of where it builds the executable and whatnot.

[baldurk]

Do you follow any particular steps when the capture breaks? I tried 5-6 times restarting the application each time and all of the captures came out looking fine. I also tried one time to capture about 10 different frames and they were all successful too.

I can try on a different machine/GPU tomorrow but if it only breaks in certain circumstances that would be good. If it is machine specific can you share the diagnostic log from your run of capturing it?

[Saftur]

I don't do anything specific, no, I just start the app and capture. I've also had a friend have the same issue when he ran the app. The texture doesn't always become black, sometimes it looks fine.

I sent you the diagnostic log by email.

[baldurk]

I tried it on a couple of other machines and one was able to reproduce the problem. When I investigated more closely I quickly ran into internal errors from nvidia's driver indicating that the process is out of memory, and trying to do the texture readback fails explicitly.

Since the process is 32-bit I suspect that when RenderDoc copies and reads back texture data something internal is running out of memory (or at least VA space) but not reporting it upwards as an error, and it's just on the edge so it intermittently works depending on fragmentation. If I force the skybox texture to be created as RGBA8 instead of RGBA32F (which doesn't lead to any loss since the input data is only JPGs, it's just a free 4x size reduction) then it seems to capture correctly every time. It would make sense since as RGBA32F that texture is about 20% of the available VA.

If you reduce the size of the skybox texture directly, or better yet recompile as 64-bit, do you still have the same problems?

[Saftur]

Changing the texture to be RGBA8 appears to have fixed it. Thank you!

Is there no way for RenderDoc to detect something like this and at least say something rather than just silently failing?

Also, I don't quite understand how it is running out of memory. By my calculations, 2048 * 2048 * 6 sides * 32 bit float = 805,306,368 bits. 32 bit should be able to address at least that much, no?

[baldurk]

It wasn't failing in RenderDoc's code, somewhere in the GL commands to copy and readback the texture it was ending up with pure black instead of actual data, but that in itself is not necessarily a bug because the texture could just be black.

Your calculation is off by a factor of 4, 2048 * 2048 * 6 sides * 32-bit float * 4 components (RGBA) means 400MB. The program also had a 1024x1024 6-sided render target that was also 32-bit float RGBA which is another 100MB, and not to mention any other memory needed for other resources as well as other non-graphics related memory allocations. RenderDoc has overhead during capture including duplicating textures that may be modified, so given a typical windows 32-bit process only has 2GB of addressable user VA space the process doesn't necessarily have to be completely out of memory to be unable to allocate a contiguous 400MB range of memory.

Unless you have extremely specific requirements there's not really any reason to build code as 32-bit, 64-bit is a pretty safe minimum spec even if you want to target older systems.