Shared Arena Env Allocator Usage Across Modules

[RyanRio]

Describe the issue

This is a precursor to #19795, I've created a full demo for this one, and will have a full demo for the first issue soon. I'd like to verify that the error code I'm receiving is 1. expected, and 2. safe to ignore.

When an executable (demo.exe) dynamically loads two DLLs (a.dll and b.dll) that separately create a session through usage of a shared static library (lib) the following code causes a ORT_INVALID_ARGUMENT status -

m_ort->CreateArenaCfgV2(keys, reinterpret_cast<const size_t*>(values), sizeof(values) / sizeof(size_t), &arena_cfg);
m_ort->CreateAndRegisterAllocator(GetEnv(), mem_info, arena_cfg); // ERROR HERE in whatever DLL's 'Run' function was called second

Normally to ensure the allocator is only created once in the process, I could for instance use thread local storage indices with TlsAlloc, TlsGetValue, etc, but each module has it's own thread local state. So it seems like the best I can do (without pulling out some of the library to it's own shared DLL which I don't want to do) is ignore the error, which from the logs seems benign as no extra allocations occur.

To reproduce

I've attached a complete example, model and onnxruntime (1.17) -

demo.zip

A few things of note -

1. I create the env every time (CreateEnvWithGlobalThreadPools), but it seems that as long as I don't call ReleaseEnv until the very end onnx will hand back the first env created
2. I configure the session to use the env allocator AddSessionConfigEntry(m_session_options, kOrtSessionOptionsConfigUseEnvAllocators, "1")
3. I disable the cpu mem arena (DisableCpuMemArena(m_session_options)) for the session as without doing so it recreates the BFCArena according to the logs, but then ends up using the one in the env anyway, so this seems undesired
4. In dll_a I demonstrate that you can use TLS to avoid double initialization within the module

To see the error, uncomment the ORT_ABORT_ON_ERROR macro -

// ORT_ABORT_ON_ERROR(m_ort->CreateAndRegisterAllocator(GetEnv(), mem_info, arena_cfg));
std::cout << "\nCreating and registering allocator..." << std::endl;
m_ort->CreateAndRegisterAllocator(GetEnv(), mem_info, arena_cfg);
std::cout << "Created and registered allocator..." << std::endl;

Urgency

No response

Platform

Windows

OS Version

10

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17

ONNX Runtime API

C

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[snnn]

CreateEnv should only be called once per process.

[RyanRio]

Is there a way I can query if it's already been created so the two modules can cooperate?

[snnn]

I'm sorry. No, there isn't such a function.

[RyanRio]

Ah ok, and when you say "should", is it undefined behavior to call it more than once? Same question as for calling CreateAndRegisterAllocatorV2.

I'm also all ears for alternative solutions if anyone has one, although I understand that's not really onnx's problem.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.