Indiana Jones and the Great Circle - DLSS Frame Gen

[SveSop]

#8

Since this pull did not fix the issue, even tho it may or may not be related to cuda, ill keep it open a while to see if there COULD be something there with Cuda Context creation or something.

I do not own this game, but maybe someone could come up with a link to a vulkan demo or something using DLFG? The NVIDIA "Donut" demo does use DLSS, but does not use Frame Gen to my knowledge.. and that does not work when running with the -vulkan option at all currently for me.
It does not load nvcuda either, so probably not comparable other than maaaaaybe something with winevulkan in general.

[Saancreed]

There is https://github.com/nvpro-samples/vk_streamline, but it will likely refuse to work because legacy Vulkan Reflex is not supported in Wine/Proton.

(I have a pile of hacks for that though, which I can't currently publish because of header licensing 😩)

Fwiw of all the Vulkan games I'm aware of that come with DLSS Frame Generation (Portal RTX, Portal Prelude RTX, No Man's Sky and Indy, please let me know if you're aware of other ones), only Indy ever tried to actually call into nvcuda. Maybe this is specific to the nvngx_dlssg snippet version that's shipped with this game, or maybe the game does this on its own on purpose, but I'm not sure.

There is, however, a concerning pattern where if the game is using Streamline (so, every one except Portal RTX) then Vulkan DLFG will refuse to work and die. Only Portal RTX, the sole game that was able to avoid Streamline due to dxvk-remix having a custom, blessed by Nvidia, DLFG integration, currently works if you pass WINE_DISABLE_HARDWARE_SCHEDULING=0 as environment variable to Proton.

[SveSop]

Yeah, i compiled vk_streamline and it does actually fire up. Using Bottles for this test with regular wine, but i should probably try GE-proton or something with that hardware_scheduling thing..

But it does actually start up. Enabling/disabling reflex does not indicate much but probably due to latencyflex or whatsitsname is not working.. dunno. Enabling DLSS does not work, and DLFG (DLSS-G it seems to be called in that log) shows this hardware scheduling thing.

viewing at some logs it does indicate something in the line of Native VK OFA feature not supported on this device! Falling back to OFA VK-Cuda interop feature. So.. maybe this is somewhat indicative of why it would attempt cuda usage?

Ill do some spying on what possible nvcuda calls if any is made in windows 🤔

[SveSop]

Maybe interesting.. dunno:
https://github.com/NVIDIAGameWorks/Streamline/blob/839eb0430d2b07e26f33be38f338d9811492557c/source/plugins/sl.common/commonEntry.cpp#L630-L664

[Saancreed]

Huh, nice find. Good to know there is actual Cuda interop there, that explains stuff… but we shouldn't be failing the check at https://github.com/NVIDIAGameWorks/Streamline/blob/f9fc648591a88d6accf859cd5c36010c25b6ab7b/source/platforms/sl.chi/vulkan.cpp#L2610 🤔

[SveSop]

So.. fiddling a bit with this using GE-proton-9.20, i atleast get a different error for DLFG "Error 6".. Running this in bottles i had to add the registry entry for the "RealPath" thing for NGXCore.. i thought the "DLSS script" in bottles actually would do this, but it did not.

Anyway, a small snippit from the log:

[12.12.2024 21-06-40][streamline][warn]commonentry.cpp:663[getNGXFeatureRequire
ments] Native VK OFA feature not supported on this device! Falling back to OFA
VK-CUDA interop feature.
0024:trace:nvcuda:DllMain (0x78062d650000, 1, (nil))
014c:trace:nvcuda:DllMain (0x78062d650000, 2, (nil))
014c:trace:nvcuda:DllMain (0x78062d650000, 3, (nil))

--

12.12.2024 21-06-41][streamline][warn]commonentry.cpp:663[getNGXFeatureRequire
ments] Native VK OFA feature not supported on this device! Falling back to OFA
VK-CUDA interop feature.
0150:trace:nvcuda:DllMain (0x78062d650000, 2, (nil))
0150:trace:nvcuda:DllMain (0x78062d650000, 3, (nil))
[12.12.2024 21-06-42][streamline][warn]dlss_gentry.cpp:274[updateEmbeddedJSON]
Failed to obtain DLSS-G min spec requirements from NGX, using SL defaults
[12.12.2024 21-06-42][streamline][warn]dlss_gentry.cpp:406[updateEmbeddedJSON]
Disabling DLSS-G since it is not supported on current hardware

I do believe that even tho nvcuda.dll is loaded in this case (for the vk_streamline demo thing), the nvngx.dll that NVIDIA provides with the driver actually loads libcuda.so behind the scenes, and that is why nothing of sense gets logged in the nvcuda log. It is probably some workaround and why most times the "empty" nvcuda.dll stub that is provided with proton just works.. The game just needs to load the dll, but internally when using nvngx.dll it uses libcuda.so directly.

I was looking into this on some other demos using a libcuda relay i made loading it with LD_PRELOAD and atleast that particular demo did use libcuda directly. Not entirely sure why it would fail to obtain minimum DLSS-G spec, but it could be some update needed to be done to nvngx. I seem to remember reading some PROTON_XXX option to allow updating nvngx? Arf.. cant find it..

EDIT: Doh.. if i had half a brain, it would be a better time for me.. ofc it wont work with DLFG on my stupid old linux-hack-box.. RTX2070 aint good enough. 😞

[Saancreed]

Yeah, the GPU needs to support VK_NV_optical_flow for this to work.

On my system with GeForce RTX 4080 Mobile, vk_streamline sample when launched with Proton Experimental never calls into nvcuda. Instead, I see stuff like

info:nvofapi64:DXVK-NVAPI experimental-9.0-20240718-71-g7b2cd347+ NVOFAPI/VK gcc 10.3.0 x86_64 plain (vk_streamline.exe)
info:nvofapi64:OFAPI Client Version: 5.0
info:nvofapi64:<-NvOFAPICreateInstanceVk: Success
info:nvofapi64:<-CreateOpticalFlowVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:<-RegisterResourceVk: Success
info:nvofapi64:OFExecuteVK params: inputFrame: 0x18d23a0 referenceFrame: 0x18d2400 externalHints: 0 disableTemporalHints: 1 hPrivData: 0x24cdf70 numRois: 0 roiData: 0 numWaitSync: 1 pWaitSyncs: 0x2544670
info:nvofapi64:<-ExecuteVk: Success
info:nvofapi64:OFExecuteVK params: inputFrame: 0x18d23a0 referenceFrame: 0x18d2400 externalHints: 0 disableTemporalHints: 1 hPrivData: 0x24cdf70 numRois: 0 roiData: 0 numWaitSync: 1 pWaitSyncs: 0x2546bc0

which then hangs. Oh well. But at least I do pass the sample's native OFA check, so I'm not sure why I would fail it with Indy.

[SveSop]

Does it hang immediately when you enable Frame Gen?
Default startup settings should be everything "off", so it should work just fine with various DLSS qualities except for "Ultra Quality" (that does not work in windows either).

I installed the latencyflex binaries to my distro, and used GE-Proton-20 binaries, and created a fresh prefix with whatever needed and ran with LFX=1 and i can run the sample with various DLSS qualities. Cant really see any visible difference on that demo thingy, but atleast i can hear the coil whine i have on my 2070 change in pitch when i change various qualities, so i suppose something is working 🤣

Ofc no Frame Gen due to the old card... just out of interest if the case is that it crashes "no matter what" for you and we can start blaming nvofapi64 🤣

[SveSop]

And using my linux libcuda.so relay, cuda is used directly by the nvngx.dll's:

Nvngx_func13: (0x55558afd2690, 0x55558af94aa0, 0x6ffff9ea8d60)
Nvngx_func11: (0x555589a0ba60, 0x55558afd2790, 0x6ffffc6805c0, 0x1, 0x1000ffb58)
Nvngx_func13: (0x55558aff4f30, 0x55558afdb740, 0x6ffff9ea8dd0)
Nvngx_func11: (0x555589a0ba60, 0x55558aff5030, 0x6ffffc62ade0, 0x1, 0x1000ffb58)
Nvngx_func13: (0x55558b027750, 0x55558b0034e0, 0x6ffff9ea8e50)
Nvngx_func38: (0x55558acdd2b0, 0x555589a0ba60, 0x55558ace0cf8)
Nvngx_func36: (0x1000ffb74, 0x1000ffb78, 0x1000ffb7c, 0x55558a5eddf0)
Nvngx_func21: (0x55558b05b550, 0x7725600, 0x55558a5eddf0, 0x1f, 0x5555892bd260, (nil))
Nvngx_func23: (0x55558b05b550, 0x7725600, 0x55558a5eddf0, 0x78, 0x43, 0x1, 0x8, 0x8, 0x1, (nil))
Nvngx_func22: (0x55558b05b550, 0x7725600, 0x55558a5eddf0, (nil), 0x11c790)
Nvngx_func43: (0x55558b05b550, 0x7725600)
Nvngx_func36: (0x1000ffb74, 0x1000ffb78, 0x1000ffb7c, 0x5555892aaae0)
Nvngx_func21: (0x55558b05b550, 0x7725a00, 0x5555892aaae0, 0x1f, 0x5555892bd260, (nil))
Nvngx_func23: (0x55558b05b550, 0x7725a00, 0x5555892aaae0, 0x3e2, 0x1, 0x1, 0x100, 0x1, 0x1, (nil))
Nvngx_func22: (0x55558b05b550, 0x7725a00, 0x5555892aaae0, (nil), 0x11c790)
Nvngx_func43: (0x55558b05b550, 0x7725a00)
Nvngx_func36: (0x1000ffb74, 0x1000ffb78, 0x1000ffb7c, 0x5555892aaae0)
Nvngx_func21: (0x55558b05b550, 0x7725e00, 0x5555892aaae0, 0x1f, 0x5555892bd260, (nil))
Nvngx_func23: (0x55558b05b550, 0x7725e00, 0x5555892aaae0, 0x2, 0x1, 0x1, 0x100, 0x1, 0x1, (nil))
Nvngx_func22: (0x55558b05b550, 0x7725e00, 0x5555892aaae0, (nil), 0x11c790)
Nvngx_func43: (0x55558b05b550, 0x7725e00)

This "Nvngx_funcX" functions is one of those internal hidden API thingys in nvcuda.

cuGetExportTable: Nvngx_UUID {7f9212d6-261d-dd4d-8af6-38dd1aeb10ae}

[shelterx]

I can chime in here, I've seen different results. For me it never worked, regardless if DLSS-FG is set to off.
As soon as I set GPU to AD100, the game crashes, with one exception, if nvcuda is not loaded or missing, the game will run fine with AD100 set but FG won't work, the option is there but it doesn't do anything.

For some people, the game seems to crash as soon as they enable FG, however I'm unable to verify this.

Saancreed got some logs from me...

Btw, Maybe Nvidia needs to step in and help here also...

[SveSop]

I don't have the game, so i cant test that... but what i tested was this vk_streamline demo linked above. Spoofing a AD100 card makes the demo run with the option to enable FG. Doing so however makes a call that is missing a cuda context - and hang on a black screen:

Nvngx_func11: (0x55555c255ee0, 0x55555d41be50, 0xa88340, 0x1, 0x1000ffb58)
Nvngx_func12: ((nil), 0x55555d7dd)

The func12 is supposed to take 2 "context" addresses, so something is missing.. But then again, spoofing AD100 wont make me have that optical flow vulkan extension anyway, so might be it.

There is also a call made to NvAPI_GPU_QueryNodeInfo immediately when enabling FG that could possibly be related to setting up something, but it is not documented so i do not know the struct fields that needs to be filled out.
NVAPI_INTERFACE NvAPI_GPU_QueryNodeInfo(NvLogicalGpuHandle hLogicalGpu, void* pGpuNodeInfo);
The first parameter IS a logical gpu, but the pGpuNodeInfo parameter is a struct holding some "node information", that could possibly be needed for something. Only thing i have somewhat gleamed from this is that it is using a _V2 kind of struct (from the .version field). But i have not been able to figure out much more other than it is somewhat huge (bytewise). Could be some array holding up to eg. 256 or whatever "max_nodes" is supposed to be perhaps.

Other calls that are made in this demo i do not think would be needed could be things like:
NvAPI_Vulkan_SetSleepMode
NvAPI_Vulkan_Sleep
I suppose NvAPI_Vulkan_SetLatencyMarker and NvAPI_Vulkan_InitLowLatencyDevice together with the two above might be related to Reflex usage.

NvAPI_QueryInterface (0xad298d3f): Unknown function ID seems to be a few times for this demo, but i have no info on what that is.
What i tend to do when looking for those "unknowns" is to look what calls are coming before and after, and in that case, (for this vk_streamline demo) it seems to call NvAPI_Initialize and this one.. and initialize again.. and so on. This could possibly indicate some fallback mechanism that if the unknown call is not "OK", it will try to initialize and one more go.. Just speculation ofc 😏

None of these are documented in the open API.. And i have not checked if they are used in Indiana Jones game, so in that sense it is a huge wall of text not really related to the game itself.. just figuring out what is needed for Frame Gen in the vk_streamline demo.

[Saancreed]

It doesn't help that we are effectively trying to troubleshoot three different issues here:

1. Something, either in nvcuda, nvapi or nvofapi is missing for DLFG's VK-CUDA interop to be happy.
2. Indy is trying to use VK-CUDA interop for DLFG, even though it shouldn't have to on my machine.
3. Streamline is doing something weird, to the point where none of Vulkan games with DLFG using it have Frame Generation working in Proton, even those that don't use this CUDA interop.

We should probably make sure that the issue we are trying to resolve is not hidden behind a manifestation of another issue, so to speak. There's a nonzero chance that any attempt to debug the first issue will be harder because of the third one.

Does it hang immediately when you enable Frame Gen?

I currently have r_streamlineDLSSGMode "1" in my settings so it hangs just as the main menu would be shown.

I installed the latencyflex binaries to my distro, and used GE-Proton-20 binaries, and created a fresh prefix with whatever needed and ran with LFX=1 and i can run the sample with various DLSS qualities.

LatencyFleX won't be able to help us here. It supports only D3D flavor of Reflex and only partially. Vulkan one is a slightly different beast that goes NvLowLatencyVk.dll → some private nvapi64.dll functions (ones you mention later) → Vulkan driver, and we currently can't support it due to missing headers for those private functions… but that will change soon-ish 🤫

cuda is used directly by the nvngx.dll's

This "Nvngx_funcX" functions is one of those internal hidden API thingys in nvcuda.

cuGetExportTable: Nvngx_UUID {7f9212d6-261d-dd4d-8af6-38dd1aeb10ae}

This could be the private API DLSS Frame Gen uses to support that VK-CUDA interop you found.

The func12 is supposed to take 2 "context" addresses, so something is missing.. But then again, spoofing AD100 wont make me have that optical flow vulkan extension anyway, so might be it.

Yeah, I think that's the case here.

There is also a call made to NvAPI_GPU_QueryNodeInfo immediately when enabling FG that could possibly be related to setting up something, but it is not documented so i do not know the struct fields that needs to be filled out.
NVAPI_INTERFACE NvAPI_GPU_QueryNodeInfo(NvLogicalGpuHandle hLogicalGpu, void* pGpuNodeInfo);
The first parameter IS a logical gpu, but the pGpuNodeInfo parameter is a struct holding some "node information", that could possibly be needed for something. Only thing i have somewhat gleamed from this is that it is using a _V2 kind of struct (from the .version field). But i have not been able to figure out much more other than it is somewhat huge (bytewise). Could be some array holding up to eg. 256 or whatever "max_nodes" is supposed to be perhaps.

🙁

Okay, I can imagine this being a problem.

Other calls that are made in this demo i do not think would be needed could be things like:
NvAPI_Vulkan_SetSleepMode
NvAPI_Vulkan_Sleep
I suppose NvAPI_Vulkan_SetLatencyMarker and NvAPI_Vulkan_InitLowLatencyDevice together with the two above might be related to Reflex usage.

Correct.

NvAPI_QueryInterface (0xad298d3f): Unknown function ID seems to be a few times for this demo, but i have no info on what that is. What i tend to do when looking for those "unknowns" is to look what calls are coming before and after, and in that case, (for this vk_streamline demo) it seems to call NvAPI_Initialize and this one.. and initialize again.. and so on. This could possibly indicate some fallback mechanism that if the unknown call is not "OK", it will try to initialize and one more go.. Just speculation ofc 😏

None of these are documented in the open API.. And i have not checked if they are used in Indiana Jones game, so in that sense it is a huge wall of text not really related to the game itself.. just figuring out what is needed for Frame Gen in the vk_streamline demo.

Well, I also have no idea what that is, or how critical it would be for DLFG to work. I should probably recheck if Portal RTX tries to call it.

[SveSop]

Since i dont own any of the games, if you could attach a DXVK-NVAPI log from both games i can see if any of those "unknown" addresses are similar. Preferrably with some indication with/without FG usage.

The reason i mentioned latencyflex in regards to this vk_streamline demo was not that i expected it to "work".. it just seemed a requirement for this particular demo to even start... Bottles has this as a toggle, but my "manual" wineprefix did not, and the demo did not even start up without having these. After adding the latencyflex binaries "distro wide" + LFX=1 option, i could run the vk_streamline demo with DLSS, but crashing if i enable FG. (Using GE-Proton-9.20)

If you think it could help in any way i could give you access to my libcuda.so relay library, and you could perhaps get some more info from that (just regular C code compiled with a makefile)? I dont have Linux on my 4070 gaming rig, or else i could have done more testing there WITH the proper vulkan extensions 😢

[Saancreed]

Here is a log from Portal RTX, with both Reflex and Frame Gen enabled and (as far as I can tell) working correctly: steam-2012840.log

Here is Indy, with GPU reported as Ampere: steam-2677660-ampere.log

Here is Indy, with GPU reported as Ada: steam-2677660-ada.log

And here is vk_streamline, which hangs the moment I click on the Frame Generation toggle:
steam-vkstreamline.log

I think at least the DLFG's VK-CUDA interop is bailing out in Indy because it could be loading nvofapi64.dll, looking with GetProcAddress for NvOFAPICreateInstanceCuda which we don't implement, not finding it and failing. Maybe we should try borrowing nvofapi64.dll from Nvidia's Windows driver 🙃

If you think it could help in any way i could give you access to my libcuda.so relay library, and you could perhaps get some more info from that (just regular C code compiled with a makefile)?

I can try, if I find some spare time.

[Saancreed]

Okay, so I spent the last few days finishing my Vulkan Reflex implementation, but with that out of the way I got the logs from Indy with libcuda relay: steam-2677660-relay.log

However, the behavior changed with this library preloaded. The game never loaded nvofapi64.dll in this case, and never got to make any nvcuda.dll call that would be logged by PROTON_LOG=+nvcuda (other than DllMain). Disabling DLSS Super Resolution does not affect this. Could the relay be interfering with Nvidia driver / NVNGX trying to setup VK_NVX_binary_import for CUDA interop?

[SveSop]

Could the relay be interfering with Nvidia driver / NVNGX trying to setup VK_NVX_binary_import for CUDA interop?

Short answer: Yes.

Theory:
I am having some similar things happening in windows when hard-replacing nvcuda64.dll in driverstore. There is a _RDATA segment of the original nvcuda that probably contain something that is needed.
I ofc do not know what it contains, but if that segment contains some lookup table, or some information that nvngx.dll & friends access, this would ofc fail misserably.

Same with libcuda.so.xxx.xxx (orignal driver versioned) library. There is a .rodata field like this:

[16] .rodata           PROGBITS         0000000000c2a000  00c2a000
       00000000020bb0e0  0000000000000000   A       0     0     32

compare that with my relay:

[16] .rodata           PROGBITS         0000000000043000  00043000
       000000000000bc72  0000000000000000   A       0     0     32

Quite more "data" in that segment for sure.. so no problem seing that there might be a lot more too it than just relaying the cu** calls.

I do not know if it is nvngx.dll that is responsible for setting up calls to nvofapi64.dll or not tho, but it is highly likely that some data in that read-only field contains something that the relay library does not have...

EDIT: Quick look with IDA kinda shows that this _RDATA segment in the windows .dll is some 4 jumptables with 10+ entries in each pointing to some internal offsets. I have no clue what this is.. Could be anything 😢 I do feel this is leaning towards nvngx.dll functionality that does not currently work - eg. NVIDIA. Implementing this .rdata segment is beyond me for sure.