DXMT: avoid initial data for depth/stencil textures#764
Conversation
Signed-off-by: Roberto Nibali <rnibali@gmail.com>
|
Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application. When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated. If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public. |
|
DXMT v0.80 audit update: Updating the vendored DXMT tree to v0.80 would not address this directly, because this fix is in VirtualBox's VMSVGA/D3D11 caller path. The issue is that VirtualBox can pass optional host-side backing data as initial data while creating depth/stencil resources; suppressing that for I did not open a DXMT upstream issue for this one, since it is not a DXMT source-tree fix. |
Summary
Avoid passing initial host-side backing data when creating DX11 depth/stencil textures in the VMSVGA backend.
Depth/stencil resources are normally initialized by later rendering commands such as clears. On the macOS/Arm DXMT path, passing the optional
pSurfaceDatabuffer as initial texture data can crash in Metal'sreplaceRegionvalidation when that backing memory is stale or otherwise unsuitable for the newly created depth/stencil texture.Root cause
vmsvga3dBackSurfaceCreateResource()currently passespaInitialDatawheneverpSurfaceDatais non-null, with exceptions for multisample and decoder resources. ThepSurfaceDatafield is optional and documented as potentially outdated after hardware realization. For depth/stencil textures, that upload is unnecessary in the reproduced path and can hand invalid source bytes to DXMT's Metal texture initialization.Validation
Tested locally on macOS/Arm with a Windows 11 Arm guest, VMSVGA WDDM, 3D acceleration enabled, and 256 MB VRAM.
Failing run before the change:
webgl-context, thenVirtualBoxVMaborted.VMSVGA CMDthread.AGXMetalG14X agxaAssertBufferIsValid,-[AGXG14XFamilyTexture replaceRegion:...],dxmt::InitializeTextureData<D3D11_TEXTURE2D_DESC1>,dxCreateTexture,vmsvga3dBackSurfaceCreateResource, anddxCreateDepthStencilView.Passing runs after the change:
Fixes #763.