D3D11 back end details

This document contains all the implementation details, weird foibles, potential problems, notes and everything necessary (hopefully!) to understand the D3D11 back end.

This document is long. Seriously. I've written up a lot of information to try and make everything clear, so it will be dry reading - I've split it up by section so hopefully if you're only interested in one aspect you can check it out alone.

This page covers the main capture and replay code as that's harder to understand. The other half of the D3D11 driver does debug and analysis code like rendering textures, calculating histograms, fetching post-transform data, etc. You can find this in d3d11_debug.cpp and d3d11_analyse.cpp, it's mostly fairly easy to follow from reading the code.

DXGI wrapping

While important to make the D3D11 back end work, the DXGI wrapping is fairly simplistic. We wrap the objects (ie. implement the virtual interface for the types, encapsulating the 'real' object) mostly just so that we can enclose the ecosystem when user code queries for related objects or other interfaces - otherwise we would "leak" an unwrapped object out to use code and from there things could very quickly go badly.

In the files dxgi_wrapped.cpp and dxgi_wrapped.h the DXGI interfaces are wrapped. The majority of the interface functions just pass through to the real object, only the swapchain has any significant implementation, to handle the backbuffer and Present() calls and pass that through to the D3D device.

D3D11 Important Types

• WrappedID3D11Device
  • This is the core/root object. When D3D11CreateDevice is called we create one of these objects to return, and it will then own and control the lifetimes of all the other objects.
  • This object also implements much more than just the ID3D11Device interface. e.g. D3DPERF interface from d3d9 comes in here.
• WrappedID3D11DeviceContext
  • The WrappedID3D11Device owns a single context for the immediate context, and each deferred context is also wrapped this way
• WrappedID3D11* (resources, Texture1D, Texture2D, etc)
  • Every resource has its own wrapped class. Most of these are just to get SetPrivateData setting names, and to track the refcounting of the resources to know when they are released.
  • Each Texture class, the Shader class (parent of all types of shader) and the Buffer class all contain a static map that lists all live instances of that type. This is mostly used on replay, particularly for looking up IDs in the analyse/debug code.
• D3D11Resourcemanager
  • This is an instance of the Resource manager, implemented for D3D11 and the D3D11ResourceRecord. It doesn't add much functionality - the primary thing is that D3D11ResourceRecords allow multiple threads to allocate backing store for tracking Map()s of the same buffer on different threads independently (without trashing each other's data).
• D3D11RenderState
  • This is a class that tracks shadow state of the entire pipeline.
  • It's used for taking references on any resources bound (so that if they are then released they remain alive).
  • Any resource hazards can be tracked here, so that if a resource is bound for output somewhere we unbind it for input
  • We also use it when serialising so that we can serialise out the initial pipeline state, and also so that for each drawcall we mark the resources bound for read, write, etc as appropriate (for tracking).

Ref Counting

The ref counting behaviour in D3D11 is simple except for the edge cases, which are kind of painful to handle. Every wrapped D3D object derives from RefCounter which implements the AddRef/Release interface and will call delete (with a virtual destructor) on this. It also implements a base QueryInterface (many child classes overload this) that will handle basic types. It also provides functions for 'soft' refs which is explained below.

The most painful problem with the ref counting is the circular nature of logical 'references'.

1. The device holds a reference on the immediate context, as you can query for the immediate context even if you have no references on it as user code.
2. The immediate context holds references on any resources if they are currently bound to the pipeline, as you can retrieve them with the Get() functions and they have to be alive to be used as part of the pipeline.
3. The immediate context and any other resource or deferred context hold a reference on the device as you can do GetDevice() on any of them even if you release the device.

So this circle means we can't break the chain and release things to destruction. This is a real problem, as otherwise order-of-destruction could leave a random resource being the 'last' thing to be deleted which should take down the rest of the chain.

To solve this, the back-references from resources/immediate contexts are "soft" references. They're not done through the usual AddRef/Release but they're done through the SoftRef/SoftRelease interface on the WrappedID3D11Device.

The Device does not destruct even if it has no external references, as long as it has soft references around. In order to break the circle we keep track of how many 'internal' soft references the device has - by counting the self-references generated by the chain, with all the resources bound to the pipeline together with the immediate context. When the soft reference count equals this value, the device deletes itself which brings down the whole circle. See WrappedID3D11Device::CheckForDeath.

---

The other annoying thing is that some user code checks for refcount values. This is not well defined and the code shouldn't be doing it.. but it does anyway. The most common case for this is code running before ResizeBuffers checking for refcounts on resources or views to ensure that they are 1 - such that when the resource and views are released then ResizeBuffers will be safe. The specification says that you just have to release your resources - it doesn't say what the refcounts will be.

To match this behaviour, we keep 'internal' refs on resources from views, and also from the swapchain on the backbuffer textures. These aren't the same as the soft refs on the device as we simply treat them as identical to external refs - we just don't return these internal refs with the external refcount to user code.

Resource Records

We use resource records to keep track of everything related to a given resource - primarily textures or buffers, but a couple of other records exist.

The device holds a resource record. This resource record is only used to contain chunks that we will just always keep around - creation of states, queries/predicates, input layouts, class instance/linkage chunks, and other things like that. They also contain any releases of those resources, or name setting for those resources.

Each context also holds a resource record. This record is used for recording frame chunks - it remains empty outside of the WRITING_CAPFRAME state. For deferred contexts, their resource record is used for resource use tracking (see below).

Each texture, buffer, and shader has its own resource record. The chunks related to those resources - creation, data upload/copy/update, view creation and release will be written into their resource record. For textures and buffers the records do more work - shaders only have resource records so that they can be eliminated entirely from the final capture if they aren't used. The only reason for this is that shaders are expensive to create, and any game that creates a lot of shaders will lead to slow-loading logs for no reason.

For textures and buffers, a CPU shadow copy is kept when available and used for tracking updates. Buffers always create a CPU shadow copy - it's allocated in their creation chunk in the initial data. Textures only create a CPU shadow copy when necessary, in an UpdateSubresource or similar.

ResourceRecords can have dependencies on each other, and keep a record alive even while the the resource it corresponds to has been released. The reason for this is e.g. if a texture A is created, copied into texture B, then A is released. We keep the resource record for A alive so that we can serialise out its chunks to preserve this copy in the final capture. Each chunk has a universally ordered ID so that chunks when written to the final file are still in the right order, not ordered by dependency.

The context resource record references all resources that are bound at any point in the frame, and this way including only those resources allows us to only bring in the resources required for a log and reduces the size by quite a bit. The Ref All Resources capture option overrides this, and brings in all live resources regardless of whether they are used or not.

Special note - there is some extra handling around deferred contexts and command lists. The first thing is that when in WRITING_CAPFRAME the chunks for a context go into its resource record. The chunks in a deferred context don't necessarily get included in the capture. When FinishCommandList is called, we give the command list a resource record and swap both the chunks and resource references into it. Then when ExecuteCommandList is called, the command list's resource record becomes a dependency for the immediate context's record the same as any other resource that's referenced and so it's pulled in that way.

The second thing is that a buffer can be mapped on multiple deferred contexts (or deferred and immediate contexts). To do this and to do proper Map() tracking (see below) we must have a separate shadow buffer for each context. This is handled in the resource record, each context can allocate a shadow pointer for itself and be able to lock & modify it separately to any other context.

Resource use tracking

While not actively capturing we track the state of resources rather coarsely. For some we'll attempt to keep a CPU shadow copy around so that we can have their contents available immediately when we want to capture a frame, others we will leave and mark as "GPU dirty" which indicates thta we need their initial contents at the start of any captured frame. When capturing a frame, we grab the initial contents but don't serialise them out yet, as many of them may not be needed.

To track which contents are needed and which aren't we track the references to resources by whether they're reads or writes. This is done primarily through the render state that is set whenever we hit a drawcall, but other calls like copies or clears will also actively mark a resource as referenced.

By tracking read or write references and attempting to detect partial updates to resources (like if a viewport or scissor is set when a rendertarget is bound) we try and mark which resources are read-only, which are read-before-write and which are write-before-read (with the write being a complete write before any read). In the first two cases, we will need the initial contents available - in the case of read-only we only need the initial contents if they weren't already available. In the case of write-before-read, we can safely discard the initial contents as the frame will essentially clear/initialise the resource for us.

This is a slight deviation as technically the contents of the previous frame remain the resource, but since those contents are never used it shouldn't matter. This hinges critically on being able to detect when a resource is completely initialised, and this detection isn't perfect - the Save All Initials capture option forces all initial states to be saved, even if they don't appear to be necessary.

Log state

The device and each context contains the log state. The device is 'canonical' as it triggers captures which are the only time the state really changes (see later for how captures get triggered).

• READING
• EXECUTING
• WRITING_IDLE
• WRITING_CAPFRAME

The order of these matter and are used to separate < WRITING from >= WRITING.

• When a log is initially opened for replay and the initial contents and chunks are being read, everything is in the READING state. This is a hint that code can do one-off initialisation type operations, as well as potentially some more expensive processing that will be cached for later use. e.g. when reading Draw() chunks in the READING state, we create drawcall data to build the drawcall tree.

• The EXECUTING state is used for when a log is being replayed (partially or not) and the code will execute any of the chunks onto the real API as light-weight as possible.

In general, in either of those states, data is serialised in and acted on. In the below states, data is serialised out and we handle being within the captured app.

• While capturing an app, most of the time the state is in WRITING_IDLE. This is the state used for when we want to capture any data that we'll need for a future frame - like data upload, resource creation, such like. We don't need to capture anything that will be useless for a later frame like setting resource state or binding objects to the pipeline. However note that we do track the current shadow state so that it's ready when a frame is captured. In the WRITING_IDLE state we also try to track modifications to resources where possible in shadow copies on the CPU to save on copying lots of resource contents back from the GPU at the point of frame capture, and collapse redundant serialised chunks where necessary to make the final replay more efficient.

• At the point where we shift into WRITING_CAPFRAME we start capturing every API call literally and recording it into the relevant ResourceRecords. This is the only state where state setting and drawcalls are serialised out in any way. Typically we only stay in the WRITING_CAPFRAME state for a single frame, but as detailed below we can stay there for multiple frames if there's a problem capturing.

Handling Map() and shadow CPU resource copies

Maps are the most difficult kind of API call to handle in general, as a pointer is returned to the user code to do whatever with. We at least have the Map()/Unmap() pair that we know nothing else will happen with the resource other than the map. With other APIs the pointer can stay in user land and be modified without ever returning to the API - this is even more complex to handle!

For D3D11_READ maps, we don't have to do anything at all. In fact we use the opportunity to snoop the current contents of the resource and fetch it out for ourselves.

For other maps we do different things based on the type of map or whether we are actively capturing (WRITING_CAPFRAME) or idling (WRITING_IDLE). Also note that we count how many times a resource is mapped, and we stop tracking any resource that is mapped regularly as the interception has a fair amount of overhead.

WRITING_IDLE

If we map NO_OVERWRITE, we just mark the resource as GPU dirty and give up - it's not worth the time tracking, and in all likelihood this resource will be given up on anyway as NO_OVERWRITE mapped buffers tend to be mapped many times a frame.

We ensure that a CPU copy of the data is available, if necessary by allocating it. This data should always be up to date from whatever has happened earlier - if the data was dirtied by the GPU we don't bother intercepting any Map calls and simply leave this data GPU dirty. We return a copy to this CPU data to the user to modify, while keeping a copy of what the data looked like before the user modified it.

When Unmap() is called if the CPU copy was newly allocated, we serialise out an Unmap() chunk to hold this CPU copy, otherwise the CPU copy was naturally updated and we don't create a new chunk at all. For buffers example where the CPU copy always points to the initial data in the creation chunk, this means the creation chunk is now up to date.

If we are verifying Map() calls for buffer overruns (which is a capture option users can enable), we use a separate buffer with sentinel values and check them on Unmap(), rather than returning our CPU copy directly. Currently this means we don't verify any Map() calls that are MAP_NO_OVERWRITE, or come from a deferred context (except below, when capturing). When we're doing this verification, we also disable the "after many updates ignore any maps on this resource" check above, so we intercept every Map().

WRITING_CAPFRAME

While capturing a frame, we must capture all maps and cannot omit or coalesce any. The existing CPU copy that we were updating before is now reserved for holding the initial contents at the start of the frame capture. For the purposes of intercepting Maps, each context allocates a shadow buffer from the resource's resource record and copies the initial contents into it on allocation. Another shadow buffer is allocated and each map we update it to the 'latest' contents of the resource.

This buffer is then returned to the user to modify. When it comes back, we compare the buffer before to the buffer after to see what range was modified, then upload that modified range to D3D and store it in a serialised chunk.

The reason for checking to see what range was modified was for two reasons. 1) in the case of NO_OVERWRITE it ensures we don't write to something the GPU is using (even if the data would be the same) as this breaks the contract of NO_OVERWRITE. 2) it also provides a massive space saving in the case of e.g. a 4MB buffer being mapped for only a few hundred bytes at once, over and over again. If we stored the entire buffer rather than just the part that changed, the memory & filesize overhead would be huge.

Beginning capture, failing capture, ending capture

By default, captures are bookended by swapchain presents. Something that isn't implemented yet is to support triggering captures from code at arbitrary points - this would be easy to extend just by modifying where some of the below code executes instead of in WrappedID3D11Device::Present.

We only allow one swapchain to be 'active' at once. Via keyboard input the user can cycle between active swapchains with a key. A capture is defined as the events between two presents of the active swapchain, any other presents are essentially ignored. Currently we don't serialise anything when another swapchain presents, and we assume the active swapchain presents to end a frame. It'd be good to serialise these out explicitly - both to show where other presents happen, as well as to handle the case where a programmatic capture doesn't end with a present.

In WrappedID3D11Device::Present we render the in-application overlay, which is different depending on whether the swapchain is active or not. For any non-active swapchain, we then return. The overlay is only rendered when a capture is not on-going.

Capturing on an active swapchain is a simple state machine:

• If not currently capturing (WRITING_IDLE), and there's a flag indicating we should start capturing (either from a key being pressed, or the external UI triggering or queueing a capture for this frame), we then start capturing. To start capturing we enter WRITING_CAPFRAME on the device and all the device contexts, clear any frame references that have built up and prepare initial states by copying off any candidate resources.

• If we are currently capturing (WRITING_IDLE), we try and finish the capture. If the frame failed (see below), we clear off any referenced resources again, empty out the context records, and refetch the initial states to try again next frame. If the frame was successful:

  1. We copy out the backbuffer to prepare a thumbnail.
  2. We grab the initial contents and serialise them out for those resources that need them.
  3. Iterate through the resource records, starting with the device context, and pull in all dependencies to build a list of chunks to be written to disk.
  4. We write the referenced chunks, then the initial states, then the frame chunks to disk.

As mentioned above, frame captures can occasionally fail. There are two reasons for this currently on D3D11 -

• If a resource is Map()'d in one frame before the capture, and Unmap()'d in the frame that we want to capture, we won't necessarily have intercepted the Map and might be missing the data contained in it. Technically this isn't a fatal problem as we could then synchronously fetch the contents of the buffer instead but currently we fail the frame capture and retry next frame.
• If a deferred context creates a command list when we weren't capturing we won't have the chunks for it, and any time that command list is executed we have to fail the frame. This can be an annoyance if it's only one frame late before we catch up (e.g. for command lists created over a frame boundary to be executed once the next frame). It can be a serious problem if a command list is created persistently. Because there's no way to differentiate between the types of command lists, we avoid the overhead of constantly capturing all deferred command lists (which is not insignificant), and only begin capturing once the frame capture starts - so in some cases we could get stuck never being able to find a successful frame capture. The capture option Capture all cmd lists fixes this by forcing all deferred command lists to capture from the start of the application, so that we're ready to include a command list when a frame is captured. Ideally with sufficient optimisation, the overhead for capturing deferred command lists can be reduced enough that this option can be always on, but right now that's not practical. It is however one of the few times when capturing with renderdoc doesn't work "out of the box", so it would be great to fix.