feat(service): async context propagation for task executor by flyingImer · Pull Request #4061 · apache/polaris

flyingImer · 2026-03-25T21:00:22Z

This PR adds RequestIdHolder and a concrete context propagation helper for TaskExecutorImpl. Fixes #3444

Problem

When TaskExecutorImpl schedules async work, the task runs on a different thread with a fresh CDI request scope. Request-scoped context (realm, principal, request ID) was previously propagated via ad-hoc hardcoded logic, and request IDs were not propagated at all, since the only way to read them was through RESTEasy's internal CurrentRequestManager API, which is unavailable on task threads.

Solution

RequestIdHolder is a new @RequestScoped CDI bean replacing the removed ServiceProducers.requestIdSupplier() that depended on RESTEasy internals. It produces RequestIdSupplier via CDI so any component can inject it without depending on JAX-RS types. RequestIdFilter now writes to this holder on each request.

TaskContextPropagator is a package-private helper that captures realm, principal, and request ID on the request thread and restores them into the task thread's fresh CDI request scope. It directly injects RealmContextHolder, PolarisPrincipalHolder, and RequestIdHolder. No new SPI or extension point is introduced. The implementation follows the same pattern as Bootstrapper.

CurrentRequestManager is no longer referenced anywhere in the codebase.

Out of scope (follow-up candidates)

MDC propagation: request ID is not currently written to SLF4J MDC on task threads. Can be added in a follow-up.
X-Request-ID header validation: client-supplied header is used verbatim. Pre-existing behavior in RequestIdFilter, not introduced by this PR.

Checklist

Don't disclose security issues! (contact security@apache.org)
Clearly explained why the changes are needed, or linked related issues: Fixes Provide a robust CDI way to inject request_ids #3444
Added/updated tests with good coverage, or manually tested (and explained how)
- Unit tests for TaskContextPropagator (capture, restore, round-trip)
- TaskExecutorImplTest updated for new constructor signature
Added comments for complex logic
Updated CHANGELOG.md (if needed)
Updated documentation in site/content/in-dev/unreleased (if needed)

Disclaimer

Javadoc is mainly assisted by coding agent.

dimas-b

Hi @flyingImer ! Good idea to normalize the code that deals with async context values propagation! Some comments and suggestions below.

...ime/service/src/main/java/org/apache/polaris/service/events/PolarisEventMetadataFactory.java

runtime/service/src/main/java/org/apache/polaris/service/task/TaskExecutorImpl.java

runtime/service/src/main/java/org/apache/polaris/service/task/PrincipalContextPropagator.java

runtime/service/src/main/java/org/apache/polaris/service/task/TaskExecutorImpl.java

runtime/service/src/main/java/org/apache/polaris/service/task/RequestIdPropagator.java

runtime/service/src/main/java/org/apache/polaris/service/task/RealmContextPropagator.java

CHANGELOG.md

jbonofre

Overall good to me.

Can you please fix the spotless issues ? I will do a new pass after.

Thanks !

flyingImer · 2026-03-30T19:47:30Z

Thanks @dimas-b and @jbonofre. I’ve addressed the earlier feedback and fixed the CI issues. The latest run is green now.

When you have a chance, could you please take another look at the current diff? Thanks!

runtime/service/src/main/java/org/apache/polaris/service/context/catalog/RequestIdHolder.java

dimas-b · 2026-03-30T22:55:06Z

...ime/service/src/main/java/org/apache/polaris/service/events/PolarisEventMetadataFactory.java

  @Inject Clock clock;
  @Inject CurrentIdentityAssociation currentIdentityAssociation;
  @Inject Instance<RealmContext> realmContext;
+  @Inject RequestIdSupplier requestIdSupplier;


Why not RequestIdHolder?.. It's simpler, no?

RequestIdSupplier is the read-only contract and this class only reads. It's also what PersistingMetricsReporter uses, so keeping them consistent felt right. Happy to revisit if you see a reason to diverge.

I think RequestIdHolder is enough (see my other comment).

Yes, in this case the "factory" can be thought of as infrastructure code, so using RequestIdHolder is perfectly fine.

runtime/service/src/main/java/org/apache/polaris/service/task/AsyncContextPropagator.java

runtime/service/src/main/java/org/apache/polaris/service/task/PrincipalContextPropagator.java

runtime/service/src/main/java/org/apache/polaris/service/task/RealmContextPropagator.java

flyingImer · 2026-04-02T17:15:27Z

@dimas-b I think your latest comments convinced me the shape issue is central enough to address in this PR.

My current cut is to keep the scope narrow, but switch this SPI to a state/action model so callers no longer deal with raw Object state, and TaskExecutorImpl only deals with one captured object per propagator.

I may still keep a generic cleanup hook with a default no-op, but the intent there would be generic lifecycle cleanup, not letting MDC shape the main propagation contract.

I also plan to clean up the smaller @Unremovable and RequestIdHolder points while touching this area.

Does that sound like the right cut?

dimas-b · 2026-04-02T20:42:35Z

@flyingImer : I'd prefer to keep MDC out of this PR (feel free to improve MDC data in a follow-up PR). I believe it is a totally different concern, not related to Request Context. Let's keep this PR focused on CDI concerns.

Looking forward to a new diff.

… for context management

runtime/service/src/main/java/org/apache/polaris/service/task/AsyncContextPropagator.java

runtime/service/src/main/java/org/apache/polaris/service/task/PrincipalContextPropagator.java

runtime/service/src/main/java/org/apache/polaris/service/task/TaskExecutorImpl.java

runtime/service/src/main/java/org/apache/polaris/service/task/AsyncContextPropagator.java

runtime/service/src/main/java/org/apache/polaris/service/task/RequestIdPropagator.java

dimas-b · 2026-04-02T23:26:55Z

runtime/service/src/main/java/org/apache/polaris/service/task/TaskExecutorImpl.java

+      // earlier ones, matching standard stacked-context conventions.
+      for (int i = restored - 1; i >= 0; i--) {
+        try {
+          actions.get(i).close();


I think I understand what you'd like to build, but I believe the term "context propagation SPI" (from PR title) would be really confusing in that case.

If the idea is to establish a convenience framework for running code inside some context and make sure the current thread removes traces of that context on completion, I believe a better approach would to follow the pattern of Bootstrapper (suggested new name: ContrextualTaskRunner).

polaris/runtime/service/src/main/java/org/apache/polaris/service/config/Bootstrapper.java

Line 39 in 6c1a12b

class Bootstrapper {

Inject RequestIdHolder, PolarisPrincipalHolder and RealmContextHolder into it.

Use them to propagate context internally. No need for a new SPI.

TaskExecutorImpl would call:

(on task creation) ContextualAction action = ContrextualTaskRunner.capture()

(on execution) action.runWithNewContext( { ... } );

MDC can be handled internally and cleared upon exit from runWithContext

hmmm, on second thought, makes sense. I now agree this PR doesn't have a strong enough case for a new SPI. The main value is the propagation itself, not the extension point.

I'll rework this to a concrete helper along the lines of your ContextualTaskRunner suggestion, wiring in the three holders directly. MDC will be a separate follow-up.

…e helper

dimas-b

Thanks for bearing with me, @flyingImer 🙂

dimas-b · 2026-04-03T23:48:55Z

runtime/service/src/main/java/org/apache/polaris/service/task/TaskContextPropagator.java

+   */
+  CapturedTaskContext capture() {
+    return new CapturedTaskContext(
+        realmContextHolder.get(),


nit: for good measure, it might be best to create a copy for the realm context too. It's similar to the principal in many aspects.

Request ID is just a String, so it's fine to reuse it.

Thanks for calling this out! Let me tackle it in a follow up pr later

flyingImer · 2026-04-07T00:39:49Z

Hi @jbonofre, I addressed the earlier feedback and the latest CI is green now.
Dimas already approved the current diff.
When you have a chance, could you take a quick final look? Thanks!

adutra · 2026-04-07T16:35:28Z

runtime/service/src/main/java/org/apache/polaris/service/task/TaskContextPropagator.java

+  /**
+   * Immutable snapshot of request-scoped context captured for propagation across async boundaries.
+   */
+  record CapturedTaskContext(


This imo should be public as it is being exposed outside the package scope: in the protected handleTaskWithTracing method.

It should probably be made top-level as well.

adutra · 2026-04-07T16:37:43Z

runtime/service/src/main/java/org/apache/polaris/service/task/TaskExecutorImpl.java

-    tryHandleTask(taskEntityId, clone, eventMetadata, null, 1);
+
+    // Capture request-scoped context for propagation into the task thread.
+    TaskContextPropagator.CapturedTaskContext captured = taskContextPropagator.capture();


With this change, do we still need to clone CallContext on line 148? Do we need CallContext in this class at all?

Good point. I missed it 🤦

adutra · 2026-04-07T16:39:37Z

CHANGELOG.md

 - Changed from Poetry to UV for Python package management.
 - Exclude KMS policies when KMS is not being used for S3.
 - Improved default KMS permission handling to better distinguish read-only and read-write access.
+- Request IDs are now propagated to async task threads via CDI-injectable `RequestIdSupplier`, so task log messages carry the originating request's ID. Context propagation across task boundaries is now handled by the pluggable `AsyncContextPropagator` SPI.


Suggested change

- Request IDs are now propagated to async task threads via CDI-injectable `RequestIdSupplier`, so task log messages carry the originating request's ID. Context propagation across task boundaries is now handled by the pluggable `AsyncContextPropagator` SPI.

- Request IDs are now propagated to async task threads via CDI-injectable `RequestIdSupplier`, so task log messages carry the originating request's ID. Context propagation across task boundaries is now handled by the pluggable `TaskContextPropagator` SPI.

adutra · 2026-04-07T16:40:40Z

runtime/service/src/main/java/org/apache/polaris/service/context/catalog/RequestIdHolder.java

+  @Produces
+  @RequestScoped
+  public RequestIdSupplier getRequestIdSupplier() {
+    return () -> requestId.get();


nit:

Suggested change

return () -> requestId.get();

return requestId::get

adutra · 2026-04-07T16:41:40Z

runtime/service/src/main/java/org/apache/polaris/service/context/catalog/RequestIdHolder.java

+   */
+  @Produces
+  @RequestScoped
+  public RequestIdSupplier getRequestIdSupplier() {


Do we need both a RequestIdHolder and a RequestIdSupplier? I think the holder could now just expose the raw request ID. The supplier creates an unnecessary indirection, it seems.

RequestIdSupplier is for injection into other beans, as far as I can tell. It's more specific than a Supplier<Integer>.

IMHO, RequestIdHolder is meant to be used only by infrastructure code.

Alternatively, we could produce a @Named("RequestId") Integer here and get it injected into code that calls RequestIdSupplier.get().

If "holder" interfaces are meant for use in infrastructure code only, in this case I would just rename RequestIdSupplier to RequestId – it would then look quite similar, in shape and purpose, to the RealmContext interface.

IMHO it's easier to have an injectable interface encapsulating the string, rather than injecting the raw string itself annotated with @Named.

+1 to RequestId

However, RequestIdSupplier is an old class (from #3385). It might be best to do that renaming in a separate PR 🤔

To be more specific about "infrastructure" code, I think RequestIdSupplier is for use in polaris-core and runtime/service classes that "consume" requests IDs without participating in the construction of request-scoped objects.

Factories and CDI producers should be able to use RequestIdHolder. Its role is pretty clear in that context, I guess.

adutra · 2026-04-07T16:44:01Z

...ime/service/src/main/java/org/apache/polaris/service/events/PolarisEventMetadataFactory.java

  @Inject Clock clock;
  @Inject CurrentIdentityAssociation currentIdentityAssociation;
  @Inject Instance<RealmContext> realmContext;
+  @Inject RequestIdSupplier requestIdSupplier;


I think RequestIdHolder is enough (see my other comment).

adutra · 2026-04-08T13:00:44Z

Worth noting: this PR introduces a new TaskContextPropagator bean. The bean is functionally equivalent to a custom org.eclipse.microprofile.context.spi.ThreadContextProvider.

I wonder if it wouldn't be cleaner to just wire up a custom ThreadContextProvider instead of providing an equivalent class that must be invoked explicitly at the beginning of each task.

The custom ThreadContextProvider would still require to clear ThreadContext.CDI + @ActivateRequestContext + manual population of beans. So it wouldn't look radically different for sure, but we would at least benefit from implicit execution.

dimas-b · 2026-04-08T14:21:20Z

I wonder if it wouldn't be cleaner to just wire up a custom ThreadContextProvider instead of providing an equivalent class that must be invoked explicitly at the beginning of each task.

+1

dimas-b · 2026-04-08T14:24:56Z

A note on the "Holder" class pattern (RealmContextHolder, PolarisPrincipalHolder, etc.):

These classes are meant to allow Polaris code to manage corresponding request-scoped data without relying on the REST framework (ContainerRequestContext) during async task execution.

github-project-automation bot added this to Basic Kanban Board Mar 25, 2026

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Mar 25, 2026

flyingImer mentioned this pull request Mar 25, 2026

Provide a robust CDI way to inject request_ids #3444

Open

flyingImer added 4 commits March 25, 2026 21:03

feat(service): async context propagation SPI for task executor

2ea3123

docs: add async context propagation to changelog

b9f3849

refactor(service): align RequestIdHolder with sibling holder patterns

aa051f1

chore(service): align test method names with repo testXxx convention

0968869

flyingImer force-pushed the async branch from dfb3a00 to 0968869 Compare March 25, 2026 21:10

dimas-b reviewed Mar 25, 2026

View reviewed changes

flyingImer added 2 commits March 25, 2026 21:35

chore(service): apply spotless formatting

d580fd8

chore: incorporate with comments

4ed37c7

flyingImer requested a review from dimas-b March 25, 2026 22:16

jbonofre reviewed Mar 27, 2026

View reviewed changes

fix: CI

733b65b

flyingImer requested a review from jbonofre March 27, 2026 18:50

dimas-b reviewed Mar 30, 2026

View reviewed changes

refactor(service): update AsyncContextPropagator to use RestoreAction…

ff063e9

… for context management

flyingImer requested a review from dimas-b April 2, 2026 21:22

dimas-b reviewed Apr 2, 2026

View reviewed changes

refactor(service): replace async context propagation SPI with concret…

e1e1005

…e helper

flyingImer changed the title ~~feat(service): async context propagation SPI for task executor~~ feat(service): async context propagation for task executor Apr 3, 2026

flyingImer requested a review from dimas-b April 3, 2026 21:09

dimas-b approved these changes Apr 3, 2026

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Apr 3, 2026

adutra reviewed Apr 7, 2026

View reviewed changes

dimas-b mentioned this pull request Apr 10, 2026

JDBC: Replace coarse-grained synchronized methods with per-realm locking #4054

Open

6 tasks

	- Request IDs are now propagated to async task threads via CDI-injectable `RequestIdSupplier`, so task log messages carry the originating request's ID. Context propagation across task boundaries is now handled by the pluggable `AsyncContextPropagator` SPI.
	- Request IDs are now propagated to async task threads via CDI-injectable `RequestIdSupplier`, so task log messages carry the originating request's ID. Context propagation across task boundaries is now handled by the pluggable `TaskContextPropagator` SPI.

Conversation

flyingImer commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Out of scope (follow-up candidates)

Checklist

Disclaimer

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jbonofre left a comment

Choose a reason for hiding this comment

Uh oh!

flyingImer commented Mar 30, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flyingImer commented Apr 2, 2026

Uh oh!

dimas-b commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dimas-b Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyingImer commented Apr 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

flyingImer commented Mar 25, 2026 •

edited

Loading

dimas-b commented Apr 2, 2026 •

edited

Loading

dimas-b Apr 2, 2026 •

edited

Loading

dimas-b Apr 8, 2026 •

edited

Loading