Lost Graphics Device

Lost graphics device

Graphics drivers are not perfect, and now and then the GPU stops answering: the driver crashes, Windows resets the GPU after it took too long, or the GPU is removed. Vulkan calls this a lost device. The original Blitz3D lost its graphics when another program took over the display and told the program with GraphicsLost. BlitzLLVM makes the whole graphics context again instead of ending the program: the device, the window's swap chain, the pipelines, and every texture, image, mesh and font. It may land on another GPU than before.

What the program sees

The program loses one frame (what was drawn and not shown yet is gone) and then carries on. GraphicsLost is 1 once after the graphics were made again, and 0 after that. The calls that make the graphics again are Flip and GraphicsLost, so a program that never flips and never asks does not notice, and goes on as soon as it does.

If the device is lost a third time within 5 minutes of the loss before, the program ends with the error "The graphics device was lost 3 times". A driver that keeps failing does not get better with another try, and the program would only freeze in a loop; update the graphics driver, or restart the computer if the GPU was reset. The count starts again at 0 once 5 minutes passed since the previous loss, as many devices only fail when they are hot and take a while to get hot again; BLITZLLVM_DEVICE_LOSS_COOLDOWN_MS changes the 5 minutes (in milliseconds). If the graphics cannot be made again at all (the GPU is gone, no other is there) the program ends with an error that says so.

What is kept and what is not

Everything that was given to the GPU from the CPU is made again:

• Textures from files (LoadTexture) are read from their file again. A file that was changed since is read as it is now with a warning, and one that is gone, can't be decoded or is another size now gives a magenta texture of the old size with a warning, so that the program goes on. A texture that was written to afterwards has what was written.
• Textures and images from CreateTexture, CreateImage, LoadAnimTexture and the others, (and images from files) with the pixels they were last given, mip maps, cube map faces, atlases and their views, compressed (DDS) textures.
• Pixels written with WritePixel, WritePixelFast and UnlockBuffer, also after the textures were made.
• Meshes, sprites, terrains and fonts, from the vertices the program keeps; samplers, gamma and Dither settings, and the settings of Graphics (vsync, depth).

What only the GPU knew is gone:

• Whatever was drawn or copied into a texture, an image or a cube map face: SetBuffer TextureBuffer(...) and ImageBuffer(...) with Rect, DrawImage, RenderWorld, CopyRect and the like. The texture or image has the pixels it had when they were last put into it from the CPU (or is cleared if there never were any). A program that draws into images draws them again when GraphicsLost is 1, as it had to in the original.
• The front and the back buffer; both are black after the graphics were made again.
• The frame that was recorded and not yet shown.

The pixels of every texture that does not come from a file, and of every image, are kept in memory for this, so the program uses about as much memory more as those take on the GPU. Textures from files cost no memory, the file is read again. BLITZLLVM_DEVICE_RECOVERY=0 turns it all off: the pixels are not kept and a lost device ends the program with an error, like it did before.

Testing it

BLITZLLVM_SIMULATE_DEVICE_LOSS=3,8 makes the runtime lose the device at the 3rd and the 8th call of Flip, by making all of it again on a device that works, which is the same as what happens after a real loss. With BLITZLLVM_SIMULATE_DEVICE_LOSS_LATE=1 the loss is only found after the Flip, so the calls of the program up to the next Flip run against a device that is lost, as they do after a real one.