Mimalloc v2.0.6 conflicts with Nvidia Nsight

[vauki]

Hello!

I found an issue while using mimalloc v2.0.6 and Nvidia Nsight.
Here are the steps:

1. You need an application with mimalloc integrated (D:/app.exe for example)
2. Run it using Nvidia Nsight:
   
3. In the debug configuration we get an assert:
   
4. Here is the mimalloc output:
   

It looks like the problem is that Nvidia Nsight conflicts with mimalloc. Both are injected into the DLL load order to intercept the messages they need, and in this case Nsight is loaded first and prevents mimalloc from properly intercepting malloc/free calls.

Any thoughts on how this can be fixed?

[vauki]

Can you provide source code for mimalloc-redirect.dll so I can try to find some workaround or some good fix?

[daanx]

Ah I see. I cannot check this but if it is indeed the case that the insight dll is injected earlier than mimalloc, then you should try to move the mimalloc-redirect.dll earlier in the import table of the application. This can be done manually on the exe using tools like CFF explorer. However, if you get a recent version of dev-slice the bin directory contains the minject program that you can run on you executable to do this automatically. (run minject --help for options). This may fix the problem

[vauki]

Ah I see. I cannot check this but if it is indeed the case that the insight dll is injected earlier than mimalloc, then you should try to move the mimalloc-redirect.dll earlier in the import table of the application. This can be done manually on the exe using tools like CFF explorer. However, if you get a recent version of dev-slice the bin directory contains the minject program that you can run on you executable to do this automatically. (run minject --help for options). This may fix the problem

I already tried this and it didn't help :(

[daanx]

Mmm, I'll try to run Nvidia Nsight and try to see what is going on. I guess because nsight runs the application it injects it's dll earlier than anything else so it may be difficult to get around this... tbc

[glebov-andrey]

Just in case someone comes across this in the future, I managed to get redirection to work under Nsight Systems by using minject.exe on ToolsInjectionHelper64.dll (the DLL that gets injected by Nsight Systems). Of course this needs to be undone before launching an application which shouldn't use mimalloc.