[BUG] NVAPI64.dll not registered (Bottles is using Wine-Staging NVAPI instead of DXVK-NVAPI)

[Arcitec]

Installation

• Method: Flatpak
• Version: 2022.1.14-trento-4

Describe the bug
Bottles is not loading or installing DXVK-NVAPI properly. It's instead loading Wine-Staging's built-in NVAPI.

Details with evidence that the Wine-Staging NVAPI is being loaded:
jp7677/dxvk-nvapi#67

Could it be related to caffe being based on wine-tkg which contains wine-staging code and therefore having NVAPI built in? There may be some additional step required to force us to load DXVK-NVAPI properly. Maybe some "load native DLL first, never load built-in" setting is missing for this to work.

Edit: Yes, it's caused by Wine-TKG's built-in fake NVAPI DLL. It's being loaded instead of DXVK-NVAPI DLL.

[Arcitec]

I looked in winecfg. The DLL overrides look correct to me:

So perhaps the DLLs are not installed in the correct system location, I will try disabling and re-enabling the DLLs to see if that works...

Edit: I set Bottles to DXVK-NVAPI 0.4 and then back to 0.5.1 and observed the logs to see that there were no errors in the Bottles/Wine console. Then I tried running the game again. I get the error message (see linked ticket in 1st post) proving that DXVK-NVAPI is not being used, and that the built-in Wine-Staging NVAPI is being used instead. My conclusion is that Caffe is loading its own NVAPI instead of DXVK-NVAPI. Edit 2: Yes this is the correct conclusion. Caffe is affected by this.

[Arcitec]

I tried to run a vanilla version of Wine, so I switched runner to vaniglia-6.23 instead.

Noticed that its winecfg didn't list any nvapi overrides in its library.

So I swapped dxvk-nvapi version to 0.4 and back to 0.5.1 to make Bottles register the DLL overrides.

After that I noticed that only nvapi was overridden in winecfg (some vaniglia bug), so I manually added nvapi64.

Finally, I tried to open Origin. But it refuses to even launch in vaniglia 6.23, throwing lots of error dialogs on screen.

I also tried caffe 6.23 but Origin refuses to launch there either.

Next up, I switched runner to lutris-6.21-6. This one works. But I can't enable DLSS ingame so it may not be loading NVAPI at all actually, lol...

Either way, it seems like I have narrowed this bug down to Caffe's inclusion of a fake NVAPI from Wine-Staging. The question is how we can disable Wine-Staging's fake NVAPI when a real one exists!

[Arcitec]

Some ideas for fixes:

1. Maybe there's a way to tell Caffe (Wine-TKG) to compile itself without the fake NVAPI patch from Wine-Staging.
2. Maybe Wine-TKG requires some additional environment flag to load the native DLL.
3. Maybe Bottles isn't properly giving the path to the nvapi DLL folders for wine, so it's not finding the DXVK-NVAPI DLLs?

[mirkobrombin]

The problem should not be related to Caffe but Bottles: jp7677/dxvk-nvapi#67 (comment)

[Arcitec]

The issue is found, it's unreliable registration of nvapi64.dll: jp7677/dxvk-nvapi#67 (comment)

[mirkobrombin]

Nice, can we keep tracking this on the other issue?
Nvm I have seen now that such issue is in another repository hahaha

[mirkobrombin]

Is Bottles-related, I'll fix this in 2022.1.28 release according to jp7677/dxvk-nvapi#67 (comment)

[Arcitec]

@mirkobrombin I think your existing script attempts to register nvapi + nvapi64 as native DLLs already, at least if I remember the console logging output correctly when switching between different dxvk-nvapi versions.

But yeah there's some issue at the moment where only nvapi is registered without nvapi64. I hope it's not too hard to reproduce it. I have no idea why only one of the DLLs was registered for me.

[Arcitec]

I created a new Custom Bottle with Caffe-7.0.

• At first, no nvapi DLL is registered in winecfg.
• Clicked "Enable DLSS (DXVK-NVAPI)".
• Both nvapi and nvapi64 are registered in winecfg.
• Swapped to vaniglia-6.23 and checked winecfg too, both DLLs are still registered.
• So this fresh bottle was successful.

So the issue itself may be caused by one of these things:

1. I originally created my gaming bottle with vaniglia-6.23 and changed to caffe-7.0. Maybe that upgrade unregistered the nvapi64 DLL.
2. Maybe wine itself doesn't reliably register the DLL.
3. Maybe bottles doesn't reliably send the DLL registration to Wine.

I think it's number 2 or 3. Because as earlier testing revealed, I was switching between different versions of dxvk-nvapi in Bottles, and was watching the terminal output saying that it was registering the two DLLs, and winecfg (which I of course restarted before checking) still didn't list nvapi64 even after several supposed "Registering dll X" messages.

I dunno what you can really do about this. Maybe it's a bug in how Bottles registers "WINEDLLOVERRIDES" env variable, if you aren't writing the DLL registration directly into some internal Wine storage. Let's hope it's something as simple as Bottles forgetting to append nvapi64 to WINEDLLOVERRIDES. :) Because it would be terrible if this is a buggy DLL registration in Wine.

Hope this extra info helps.

[mirkobrombin]

I’ve checked the entire process and I think there is no bug (on both runner and Bottles sides).

There are two “type” of overrides:

• on the fly using WINEDLLOVERRIDES env var
• register keys (like how winecfg does)

Bottles handle both of them:

• when you install dxvk nvapi, Bottles adds the overrides on the register, so if you disable the component and enable it again, Bottles should automatically set the overrides
• when you set a custom override from bottle preferences, Bottles append it to the env var and use it with your next command (program execution, ..)

So there should no problems on Bottles front. Maybe something goes wrong during your tests?

Changing the runner should not reset overrides, so it should just works.

I’ll check again the entire process but all should works fine.

[Arcitec]

@mirkobrombin Great to hear that you did thorough tests and checked the process!

I can say with confidence that I didn't do anything wrong. Only one DLL was registered and I had to add the other manually. But perhaps it's a result of some extremely rare situation in Wine or Bottles code.

[mirkobrombin]

Bottles/src/backend/nvapi.py

Line 11 in d3b9a61

"nvapi.dll"

Bottles/src/backend/nvapi.py

Line 14 in d3b9a61

"nvapi64.dll"

🤔

I’ll do more tests tomorrow

[Arcitec]

@mirkobrombin Yeah the Bottles code looks good and it's probably not gonna lead anywhere to look more in Bottles code.

I suspect more that Wine unregistered one of the DLLs due to some corruption/resetting of the registry. It might have happened when I upgraded from vaniglia 6.23 (where I originally used dxvk-nvapi successfully) to caffe 6.23 to caffe 7.0 (this is where nvapi64 was suddenly missing). I never manually re-registered dxvk-nvapi between those steps.

The only weird part is this test I did on caffe-7.0 when I first tried to re-register the DLLs by forcing Bottles to do the registration, but nvapi64 was never re-registered by Bottles even though I swapped versions several times to trigger the register code: #951 (comment)

In the end, I had to re-add it manually.

I don't think we'll find the issue though. Looks like it's something very rare.

[jp7677]

Just a shot in the dark from reading the above: when using an existing wine prefix with a new wine version, the new wine version will update the prefix. May be that prefix update replaces any existing nvapi dlls with the ones provided by wine.

[mirkobrombin]

Just a shot in the dark from reading the above: when using an existing wine prefix with a new wine version, the new wine version will update the prefix. May be that prefix update replaces any existing nvapi dlls with the ones provided by wine.

I hadn't considered this, actually if the runner provides such DLLs it actually sets the overrides. In that case, I have to make sure Bottles overwrite the DLLs again on runner change.

[jp7677]

Just a shot in the dark from reading the above: when using an existing wine prefix with a new wine version, the new wine version will update the prefix. May be that prefix update replaces any existing nvapi dlls with the ones provided by wine.

I hadn't considered this, actually if the runner provides such DLLs it actually sets the overrides. In that case, I have to make sure Bottles overwrite the DLLs again on runner change.

Not sure if you are already doing this, but I guess you'll also have to trigger the prefix update manually first.

[Arcitec]

That is actually a really good theory regarding prefix updates. It would be safest if any switching of runner triggers a prefix update and then re-registers all DLL overrides.

Well, I decided to test if the runner update unregisters the DLLs:

1. Create custom bottle with vaniglia-6.23.
2. Open winecfg to force prefix update and look at overrides: Nothing overridden except winemenubuilder. Okay...
3. Went into preferences and toggled "Enable DLSS (DXVK-NVAPI)". Console said:

(12:24:08) INFO Setting Key: [dxvk_nvapi] to [True] for bottle: [nvtest2]… 
(12:24:08) INFO Adding Key: [HKEY_CURRENT_USER\Software\Wine\DllOverrides] with Value: [nvapi] and Data: [native,builtin] in register bottle: nvtest2 
002c:err:wineboot:process_run_key Error running cmd L"C:\\windows\\system32\\winemenubuilder.exe -a -r" (2).
(12:24:09) INFO reg: The operation completed successfully
	
 
(12:24:09) INFO Adding Key: [HKEY_CURRENT_USER\Software\Wine\DllOverrides] with Value: [nvapi64] and Data: [native,builtin] in register bottle: nvtest2 
(12:24:12) INFO reg: The operation completed successfully

4. Looked at winecfg. I see nvapi and nvapi64 both registered as native.
5. Pressed "kill all wine processes" and ensured old wineserver was dead.
6. Set runner to caffe-6.23.
7. Opened winecfg to trigger prefix update and look at the overrides. Both nvapi and nvapi64 are still registered.
8. Pressed "kill all wine processes" and ensured old wineserver was dead.
9. Set runner to caffe-7.00.
10. Opened winecfg to trigger prefix update and look at the overrides. Both nvapi and nvapi64 are still registered.

I then redid the test in another new bottle and skipped step 5-6-7, going directly from vaniglia-6.23 to caffe-7.00. Same result, DLLs remained registered.

Here's all I truly know in this thread:

1. I never manually deleted or added anything in Wine Libraries. I swear. I have no reason to touch the libraries. And if I do anything like that, I am always super careful to not remove the wrong thing. But I never removed anything here!
2. I didn't delete nvapi64. But somehow, nvapi was still registered and nvapi64 was missing from Wine Libraries. So the library was half-registered. I only discovered this when @jp7677 alerted me that I was using Wine-TKG (Caffe)'s built-in NVAPI stub instead of the real DXVK-NVAPI and I decided to check if all DLLs were registered. That's the first time I saw that the nvapi64 was not registered.
3. The only thing I did was change wine versions and launch software. So what can have caused this issue? We may never know...

The only real question now is if it's worth the effort to: Make a list of Bottle DLL overrides (just the custom toggles the user has manually enabled in Bottles), and re-register them as native when switching runner. Maybe as simple as "if switching runner, trigger prefix update and then loop through all Bottles settings and re-register just the DllOverrides". Is it worth it for extra peace of mind? We still don't know and probably will never know what unregistered nvapi64.dll but kept nvapi.dll. It's weird.

[mirkobrombin]

Just a shot in the dark from reading the above: when using an existing wine prefix with a new wine version, the new wine version will update the prefix. May be that prefix update replaces any existing nvapi dlls with the ones provided by wine.

I hadn't considered this, actually if the runner provides such DLLs it actually sets the overrides. In that case, I have to make sure Bottles overwrite the DLLs again on runner change.

Not sure if you are already doing this, but I guess you'll also have to trigger the prefix update manually first.

I don’t think

This seems to be automatically done on next command execution. Maybe it's better to do it right away at the runner change.

[mirkobrombin]

That is actually a really good theory regarding prefix updates. It would be safest if any switching of runner triggers a prefix update and then re-registers all DLL overrides.

Well, I decided to test if the runner update unregisters the DLLs:

1. Create custom bottle with vaniglia-6.23.
2. Open winecfg to force prefix update and look at overrides: Nothing overridden except winemenubuilder. Okay...
3. Went into preferences and toggled "Enable DLSS (DXVK-NVAPI)". Console said:

(12:24:08) INFO Setting Key: [dxvk_nvapi] to [True] for bottle: [nvtest2]… 
(12:24:08) INFO Adding Key: [HKEY_CURRENT_USER\Software\Wine\DllOverrides] with Value: [nvapi] and Data: [native,builtin] in register bottle: nvtest2 
002c:err:wineboot:process_run_key Error running cmd L"C:\\windows\\system32\\winemenubuilder.exe -a -r" (2).
(12:24:09) INFO reg: The operation completed successfully
	
 
(12:24:09) INFO Adding Key: [HKEY_CURRENT_USER\Software\Wine\DllOverrides] with Value: [nvapi64] and Data: [native,builtin] in register bottle: nvtest2 
(12:24:12) INFO reg: The operation completed successfully

4. Looked at winecfg. I see nvapi and nvapi64 both registered as native.
5. Pressed "kill all wine processes" and ensured old wineserver was dead.
6. Set runner to caffe-6.23.
7. Opened winecfg to trigger prefix update and look at the overrides. Both nvapi and nvapi64 are still registered.
8. Pressed "kill all wine processes" and ensured old wineserver was dead.
9. Set runner to caffe-7.00.
10. Opened winecfg to trigger prefix update and look at the overrides. Both nvapi and nvapi64 are still registered.

I then redid the test in another new bottle and skipped step 5-6-7, going directly from vaniglia-6.23 to caffe-7.00. Same result, DLLs remained registered.

Here's all I truly know in this thread:

1. I never manually deleted or added anything in Wine Libraries. I swear. I have no reason to touch the libraries. And if I do anything like that, I am always super careful to not remove the wrong thing. But I never removed anything here!
2. I didn't delete nvapi64. But somehow, nvapi was still registered and nvapi64 was missing from Wine Libraries. So the library was half-registered. I only discovered this when @jp7677 alerted me that I was using Wine-TKG (Caffe)'s built-in NVAPI stub instead of the real DXVK-NVAPI and I decided to check if all DLLs were registered. That's the first time I saw that the nvapi64 was not registered.
3. The only thing I did was change wine versions and launch software. So what can have caused this issue? We may never know...

The only real question now is if it's worth the effort to: Make a list of Bottle DLL overrides (just the custom toggles the user has manually enabled in Bottles), and re-register them as native when switching runner. Maybe as simple as "if switching runner, trigger prefix update and then loop through all Bottles settings and re-register just the DllOverrides". Is it worth it for extra peace of mind? We still don't know and probably will never know what unregistered nvapi64.dll but kept nvapi.dll. It's weird.

To be sure that all is working correctly I'll do this on every runner change:

• updating the prefix
• re-initializing active components (dxvk, dxvk-nvapi, vkd3d) overrides

[mirkobrombin]

Implemented 418143c

Thanks everyone for the debugging and info 😄

[Arcitec]

Implemented 418143c

Thanks everyone for the debugging and info smile

That looks beautiful, and makes Bottles even more robust! Thank you so much, you're awesome and Bottles is by far my favorite way to run apps and games! :D