Multithreaded shader compilation

[micossow]

Is it expected behavior, that using IDxcCompiler::Compile in multiple threads on the same IDxcCompiler object results in crash?

Stack trace:

dxcompiler.dll!llvm::PassRegistry::registerPass(const llvm::PassInfo & PI, bool ShouldFree) Line 82
dxcompiler.dll!initializeReducibilityAnalysisPassOnce(llvm::PassRegistry & Registry) Line 235
dxcompiler.dll!llvm::initializeReducibilityAnalysisPass(llvm::PassRegistry & Registry) Line 235
dxcompiler.dll!llvm::IsReducible(const llvm::Module & M, llvm::IrreducibilityAction Action) Line 245
dxcompiler.dll!hlsl::ValidateFlowControl(hlsl::ValidationContext & ValCtx) Line 3850
dxcompiler.dll!hlsl::ValidateDxilModule(llvm::Module * pModule, llvm::Module * pDebugModule) Line 3955
dxcompiler.dll!DxcValidator::RunValidation(IDxcBlob * pShader, llvm::Module * pModule, llvm::Module * pDebugModule, hlsl::AbstractMemoryStream * pDiagStream) Line 259
dxcompiler.dll!DxcValidator::ValidateWithOptModules(IDxcBlob * pShader, unsigned int Flags, llvm::Module * pModule, llvm::Module * pDiagModule, IDxcOperationResult * * ppResult) Line 133
dxcompiler.dll!DxcCompiler::Compile(IDxcBlob * pSource, const wchar_t * pSourceName, const wchar_t * pEntryPoint, const wchar_t * pTargetProfile, const wchar_t * * pArguments, unsigned int argCount, const DxcDefine * pDefines, unsigned int defineCount, IDxcIncludeHandler * pIncludeHandler, IDxcOperationResult * * ppResult) Line 2116

Repro gist:
https://gist.github.com/michalo2882/75691893f307162f3b40548d8d87f036

[marcelolr]

Correct, generally all of the objects we provide aren't thread-safe. They're also not re-entrant except for AddRef/Release (that is, if you implement an include handler for example, you can't call back into the compiler). They also don't have thread affinity (that is, you can make a call on one thread, then make a call on a different thread, as long as the calls aren't concurrent). Note that creating compiler instances is pretty cheap, so it's probably not worth the hassle of caching / sharing them.

However, please leave this issue open for a bit - this particular problem does show a race that we haven't seen before. For posterity, here's an explanation of this (this really should become a Wiki topic or blog post at some point).

LLVM has a notion of a registry pass, where pass metadata is available. Some functions do take the registry pass as an input, but there are many places where the reference is always obtained from a global singleton, so in practice the registry pass ends up being global.

The problem is that many passes will register on-demand, like in the stack you see above, which can lead to complications with correctness/completeness (enumerating passes results in incomplete / changing information) or with failure points (code doesn't expect to deal with registration errors when creating passes). Our solution is to register all the passes at DLL load time, and then the registry acts as a frozen, read-only object during the life of the library.

Long story short: if we're registering during compilation, it's too late and we forgot to register a pass at DLL load time.

[micossow]

Thanks for info! I'll add some critical sections to my code.