Building PhysX as multiple DLLs or a single DLL

[PathogenDavid]

This issue is mostly to note my thoughts and reasoning behind how I ended up at the solution I'll eventually be pushing here.

Special compilation target for a single DLL

This is what the original prototype used. It worked by adding a special CMake target to PhysX its self which used CMake shenanigans to combine the various parts of PhysX into a single DLL. This modified InfectedPhysX.Native.dll also included the equivalent of the inline export helpers at the time.

This approach had some major downsides:

• It required forking the PhysX repo
• It caused all of PhysX to be built twice (if you wanted to use the CMake INSTALL target.)
• It didn't evolve well to meet the introduction of LinkImportsTransformation
  • To use it properly we'd have to either build all of PhysX during generation which is slow
  • In theory we could've also hard-coded the DLL to InfectedPhysX.Native.dll, but then we miss out on validation.
• It gets kinda far away from how PhysX is normally built

Using individual DLLs like PhysX normally does

This feels the most natural and requires 0 modification to the PhysX repo. However it has a major downside of not handling the static components of PhysX very well:

• Only some PhysX components are actually built as DLLs when PX_GENERATE_STATIC_LIBRARIES is disabled.
  • In particular the following are never built as DLLs on any platform:
    • PhysXCharacterKinematic
    • PhysXExtensions
    • PhysXPvdSDK
    • PhysXTask
    • PhysXVehicle
  • At one point I attempted to minimally modify the affected libraries to build DLLs anyway, but things broke because the CMake configurations for building them as DLLs aren't exercised.
  • Additionally, only the Windows presets have PX_GENERATE_STATIC_LIBRARIES disabled by default.

I worked around the static library issue on Windows by generating linker response files along these lines:

/NOLOGO
/IGNORE:4001
/IGNORE:4102
/MACHINE:X64
/DLL
/DEBUG
"..\path\to\PhysXExtensions_static_64.lib"
/LIBPATH:"C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.30.30704\lib\x64\"
/LIBPATH:"C:\Program Files (x86)\Windows Kits\10\Lib\10.0.20348.0\ucrt\x64\"
/LIBPATH:"C:\Program Files (x86)\Windows Kits\10\Lib\10.0.20348.0\um\x64\"
/DEFAULTLIB:libcmt.lib
"..\path\to\other\PhysXComponent1_64.lib"
"..\path\to\other\PhysXComponent2_64.lib"
"..\path\to\other\PhysXComponentETC_64.lib"
/OUT:"PhysXExtensions_64.dll"
/EXPORT:symbol_needing_export_from_lib1
/EXPORT:symbol_needing_export_from_lib2
/EXPORT:symbol_needing_export_from_libETC

I also experimented with doing something similar (and much simpler) on Linux:

clang -shared -L. -Wl,--whole-archive -lPhysX_static_64 -o libPhysX_64.so -Wl,--no-whole-archive

However, either way you cut it both of these are adding quite a bit of complexity. (Particularly with the Windows version since in the above form it needs paths to a bunch of system junk.)

There's also the question of whether or not unintentional static non-sharing is occurring like we had with libclang. This isn't really easy to validate and could easily be happening between the inline export helper and the converted static libraries since the inline export helper is going to end up linking in its own copy of the static libraries it depends on. There may be nothing like this (there honestly probably isn't), but it's a HUGE pain when it pops up.

In theory we could gain some peace of mind here by converting each static library into a DLL along with its corresponding inline export helpers but that's even more added complexity and would require we either build the inline export helpers directly in-generator, have the generator emit individual CMake project, or hard-code individual CMake projects for the individual static libraries within PhysX (and have to keep up if they ever add news ones -- which is probably rare but still a pain.)

Finally, we really need to look at what this gains us:

• Modularity (only include the PhysX components you want.)
  • Great in theory, but due to how inline export helpers work you probably end up getting large chunks of the static libs anyway.)
  • Have to manually define the relationships between the various modules (not actually that bad, a quick skim suggests the all only depend on Common and/or Foundation which are required anyway.)
  • In theory the main motivation here is that the build will be smaller, right?
• Uhhh, not much else?

This would also mean we need to split up the C# bindings into multiple assemblies. Not that difficult, but this runs into the same issues mentioned earlier relating to multiple CMake projects for each static library. (Although emitting a small C# file isn't very hard.)

Let's look at that main motivation: Making the build smaller through modularity.

Here's the sizes of the individual PhysX modules on Windows: (All DLLs below are the checked builds.)

Size	File
310K	InfectedPhysX.Native.dll
842K	PhysXCharacterKinematic_64.dll
1.5M	PhysXCommon_64.dll
393K	PhysXCooking_64.dll
2.1M	PhysXExtensions_64.dll
267K	PhysXFoundation_64.dll
893K	PhysXPvdSDK_64.dll
627K	PhysXTask_64.dll
1.4M	PhysXVehicle_64.dll
2.2M	PhysX_64.dll
10.2M	Total
4.14M	Minimum

(Minimum includes InfectedPhysX.Native.dll, PhysX_64.dll, PhysXCommon_64.dll, and PhysXFoundation_64.dll since they are all absolutely mandatory.)

I wanted to test the size a runtime where everything is combined* and it came out to...5.96 MB. So you'd be saving a whopping 1.82 MB if you went with the bare minimum PhysX distribution, and you'd actually be paying more if you included each module separately. Furthermore I suspect most of this is probably from PVD which isn't even present in release builds. (Well, that's what the documentation says, I built in release and still got a PVD lib.)

(*To make this runtime, I created a PhysX build preset which built Win64 as static libraries, used Biohazrd to make a list of /EXPORT directives for all symbols in all of those static libraires, and them built the inline export helper with those directives and linked it to the static libraries.)

Building PhysX as static libraries normally and then combining them into one DLL

This is sort-of a hybrid of the above two options. We'll build PhysX with PX_GENERATE_STATIC_LIBRARIES enabled and combine the individual static libraries and the inline export helper into a single DLL using generated export directives (or maybe a def file -- and the equivalent on Linux.)

If it isn't obvious by now, this is the option I want to go with. IMO it's the right tradeoff between complexity and keeping our build similar to C++. It also probably gives a more natural progression towards static linking in NativeAOT scenarios. It also keeps the way we consume PhysX consistent between different platforms.

The downside of this is needing to get our presets into PhysX. I think we'll just provide a script that copies them into PhysX automagically. (Making our own preset isn't even considered non-standard, Nvidia specifically documents it as something you can do: https://gameworksdocs.nvidia.com/PhysX/4.1/documentation/physxguide/Manual/BuildingWithPhysX.html#customize-cmake-presets)

[AnderT]

Hi David,

When do you think you will have the DLLs / DLL ready for consumption?

Cheers,

Ander

[PathogenDavid]

Hello Ander!

Apologies for the radio silence. I had meant to send a GitHub Sponsors update last month but time really got away from me in October. (If you check your email you should have just gotten one for November)

I plan to get CI stood up on the PhysX bindings in the next week or so, and that will bring along with it binaries and eventually NuGet packages.

If you just want to tinker, here's a fresh zip of the current Windows x64 binaries built from the current main for the sample, the unmanaged bindings, and the native runtime: Mochi.PhysX.Sample.zip

• InfectedPhysX.dll is the .NET assembly you'll need to reference (the actual namespace for everything inside is Mochi.PhysX)
• Mochi.PhysX.Native.dll is the native runtime and will need to be present in your output directory (I'd just use CopyToOutputDirectory for it.)
• The sample runs the simulation without a graphical display. You can use the (Windows-only) PhysX Visual Debugger to see what the simulation is doing.

[PathogenDavid]

@AnderT, if you didn't already see I published v0.0.0-alpha0 to NuGet a few minutes ago. Apologies for the delay, let me know if you run into any issues!