Out memory error 4.25 with DX12 RHI

[duplexsystem]

Withe the latest version form source control I get an out of video memory error when trying to save. I have 11gb of video memory and the project is 1gb so there is no conceivable way it could run out. (don't know why it would even be using VRAM)

[Carbon-NateBowman]

To add some info to this issue.

Issue is easily reproducible by me and my team when checking out any asset config/texture/level etc with RHI set to DirectX12.

It only occurs for me and my team when using dx12, past dx11 projects work fine.

After opening a working existing dx11 project, switching RHI to DX12, and restarting the editor, the issue occurs upon first checkout.

The issue occurs with release 2.16-beta and still occurs after 214af6d

@SRombauts I cannot find a workaround for this, and as my team rely on an LFS locking workflow we are a bit stuck until it is resolved.

[duplexsystem]

@Carbon-NateBowman after getting this with another plugin I determined it was a UE 4.25 bug. You can attempt to verify this is you want. But I do no know if it can be fixed plugin side or has to be fixed engine side.

[SRombauts]

Thanks for reporting this and updating me.
The question is, is this happening without Source Control enabled ?

[Carbon-NateBowman]

Thanks for reporting this and updating me.
The question is, is this happening without Source Control enabled ?

It does not appear to happen without source control enabled.
For me, DX12, no source control, at least 5hrs a day since 4.25 release, not happened once.

[duplexsystem]

I've had it happen with the megascans plug-in (another plugin which would be doing similar UE side operations) only starting with 4.25, with dx12 which is far to specific to be unrelated.

[Carbon-NateBowman]

I've had it happen with the megascans plug-in (another plugin which would be doing similar UE side operations) only starting with 4.25, with dx12 which is far to specific to be unrelated.

Oh, that's a good point, when 4.25 came out, I also had that (but straight ctd, no error) when importing large megascans assets with their first 4.25 plugin versions. It was resolved after a few revisions. As a test, I just imported a few 8k surfaces and high poly models (with all lods) without issue.

[Carbon-NateBowman]

UE 4.24.3 with 2.15-beta RHI set to DX12, spikes on checkout but not fatal for small numbers of assets. However, checking out multiple items causes the same kind of spike and eventually a crash, just much slower.

Checking out 25 items.

[duplexsystem]

Which makes me think it's not a bug with update to 4.25 but a bug or change with ue that requires a plug-in side work around or change.

[crevetor]

I'm having the same issue :(

[SRombauts]

Sorry guys, I am having more things to deal with on my day job, and at the same time more things to do with the family at home.
Thanks a lot @crevetor for fixing this for us all

[Carbon-NateBowman]

Better results in 4.25 but unfortunately not fixed completely.

see #142 (comment)

[SRombauts]

Still present with UE5.0 & pre-UE5.1 built from sources

see SRombauts/UEPlasticPlugin#86

I have started reproducing the issue in Unreal without any Source Control operation, with a simple infinite loop in the Provider, meaning it's purely a render thread issue (and I can see how other progress bars in the Editor code seem to handle this)

[SRombauts]

A similar bug https://issues.unrealengine.com/issue/UE-93603 Directx 12 RHI can run out of video memory on long process (anims import)

has been fixed by adding End & BeginFrame on the RHI thread https://github.com/EpicGames/UnrealEngine/commit/4cbce19c2848723ccc01a4fba3133d663d64d4ae

[SRombauts]

And the official issue would be https://issues.unrealengine.com/issue/UE-100291 "Out of video memory" crash when attempting to save many assets with source control enabled.

A very recent one even mention the workaround I implemented in the plastic plugin! https://issues.unrealengine.com/issue/UE-160299 GPU memory leak during large source control operations:

The crash has been seen in Git and SVN, and can be avoided by removing FScopedSourceControlProgress::Tick. This will prevent Slate from being ticked during the operation, but will prevent the throbber widget from animating.

[SRombauts]

PR to Epic Games :)
https://github.com/EpicGames/UnrealEngine/pull/9492 [UE-100291][UE-160299] Fix source control progress bar crashing out of video memory in case of long-running operation #9492

[SRombauts]

The proper fix has been submitted to ue5-main https://github.com/EpicGames/UnrealEngine/commit/67dacf2baaafe0d9b6fb5eeb5b179fb5b871a4bf

It will be part of UE5.1 release :)