Skip to content

Fix functionInstanceInjector null on concurrent cold start (#879)#880

Merged
ahmedmuhsin merged 1 commit into
Azure:devfrom
ahmedmuhsin:fix/injector-null-concurrent-init
Jul 23, 2026
Merged

Fix functionInstanceInjector null on concurrent cold start (#879)#880
ahmedmuhsin merged 1 commit into
Azure:devfrom
ahmedmuhsin:fix/injector-null-concurrent-init

Conversation

@ahmedmuhsin

Copy link
Copy Markdown
Contributor

What this fixes

Fixes #879. Since worker 2.19.2, an instance can start up with functionInstanceInjector left null for its whole lifetime, and then every invocation on that instance fails with an NPE.

java.lang.NullPointerException: Cannot invoke
  "com.microsoft.azure.functions.spi.inject.FunctionInstanceInjector.getInstance(java.lang.Class)"
  because "this.functionInstanceInjector" is null
    at com.microsoft.azure.functions.worker.binding.ExecutionContextDataSource.getFunctionInstance(ExecutionContextDataSource.java:108)

Root cause

JavaFunctionBroker.initializeOneTimeLogics() uses double-checked locking behind the volatile oneTimeLogicInitialized flag. Since 2.19.2 it set the flag to true before building the injector.

oneTimeLogicInitialized = true;          // flag set first
initializeFunctionInstanceInjector();    // slow ServiceLoader scan runs after

Function load requests are dispatched concurrently, since JavaWorkerClient submits every message to a cached thread pool. At cold start two threads run loadMethod at the same time. Thread A enters the locked block and sets the flag. Thread B hits the lock-free outer check, sees the flag already true, returns early, and starts invoking while the injector is still null. The ServiceLoader scan for a custom injector (for example the Spring Cloud Function adapter) widens the window.

This is the same NPE that #684 fixed. #819 (OpenTelemetry support) reordered the two lines and reopened it.

The fix

Build the injector before flipping the flag, which restores the #684 ordering.

initializeFunctionInstanceInjector();
oneTimeLogicInitialized = true;

Test

Adds JavaFunctionBrokerConcurrentInitTest, which freezes a thread at the point the injector is about to be built and asserts oneTimeLogicInitialized is still false there. It fails on the old ordering and passes with the fix.

Affected versions

Regressed in 2.19.2 and present through 2.19.4. 2.19.1 and earlier are fine.

initializeOneTimeLogics set oneTimeLogicInitialized to true before initializeFunctionInstanceInjector ran. The outer flag check is lock-free, so a second load thread could observe the flag as set, skip init, and invoke while functionInstanceInjector was still null. That failed every invocation on the affected instance with an NPE in ExecutionContextDataSource.getFunctionInstance.

Restore the ordering from Azure#684 that Azure#819 undid, so the injector is built before the flag flips. Add a regression test that freezes a thread at injector init and checks the flag stays false until init completes.

@TsuyoshiUshio TsuyoshiUshio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

One tip for CountDownLatch If it works, that's great, but when I use similar solution in C#, CI has very few thread, so that blocked long and eventually timed out. But It is also possible not the case. Let's merge it and see if it works. If it works, I don't have any objection and CountDownLatch will remove flaky testing.

@ahmedmuhsin
ahmedmuhsin merged commit a91018d into Azure:dev Jul 23, 2026
19 checks passed
ahmedmuhsin added a commit that referenced this pull request Jul 24, 2026
initializeOneTimeLogics set oneTimeLogicInitialized to true before initializeFunctionInstanceInjector ran. The outer flag check is lock-free, so a second load thread could observe the flag as set, skip init, and invoke while functionInstanceInjector was still null. That failed every invocation on the affected instance with an NPE in ExecutionContextDataSource.getFunctionInstance.

Restore the ordering from #684 that #819 undid, so the injector is built before the flag flips. Add a regression test that freezes a thread at injector init and checks the flag stays false until init completes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression: functionInstanceInjector left null on concurrent cold-start load (2.19.2+) — reintroduces the NPE fixed by #684

3 participants