feat: add payload offloader core runtime

[zhongkechen]

Stack

Layer	PR	Scope
Architecture	#678	ADR-006 and design decision
1	#649 (this PR)	Core API, envelopes, runtime, operations
2	#681	Filesystem implementation and retries
3	#682	Testing utilities and integration coverage
4	#683	Examples, E2E infrastructure, and implementation docs

Depends on #678, which owns ADR-006 and the architecture decision.

Scope

• add the dedicated PayloadOffloader API, SDK-owned payload envelope, producer context, integrity metadata, and invocation-scoped codec/cache
• integrate payload policies with root execution, steps, child contexts, map/parallel, wait-for-condition, callbacks, and operation-level overrides
• add explicit chained-invoke request/output framing while preserving ordinary Lambda payloads by default
• preserve storage failures as invocation-level failures rather than user-function outcomes
• add core unit and replay/concurrency coverage

Intentionally excluded from this PR:

• filesystem storage and retry decorators
• local/cloud testing utility integration
• examples, E2E infrastructure, and implementation documentation

Validation

• full eight-module Maven reactor on Java 17
• git diff --check
• mvn spotless:check

Related to #463.

[github-actions]

Claude AI review

Payload Offloader (Approach B) — review

The change is well-structured: a versioned @aws-durable-payload:v1: envelope keeps legacy raw checkpoints readable, offloading runs on a dedicated blocking-I/O executor, per-invocation caching prevents duplicate loads, attempt-aware storage keys keep retries isolated, and the disabled sentinel plus per-operation overrides are threaded consistently through every config/operation. Replay semantics (reference stored in the envelope, resolved via cache-or-load) look correct, and the attempt - 1 used when resuming waitForCondition state matches the attempt the state was written under.

Two confirmed defects, both surfacing only when an offloader is configured (the intended production setup):

1. Null payloads throw — PayloadCodec.serialize forwards a null serialized value into offloader.offload(...), and FileSystemPayloadOffloader / OffloadedPayload.inline reject null. Any step/child/parallel/map result, waitForCondition state, or root output that serializes to null now fails with PayloadOffloadException where it previously checkpointed cleanly. See sdk/src/main/java/software/amazon/lambda/durable/execution/PayloadCodec.java:56.
2. OVERFLOW threshold ignores envelope/escaping overhead — inline overflow payloads are still enveloped and JSON-escaped, so a payload just under the raw threshold can produce a checkpoint over the 256 KB limit the mode exists to respect. See extra-filesystem-offloader/.../FileSystemPayloadOffloader.java:33.

Residual test risk: No test covers a null operation result / root output under a configured offloader (the integration tests only use non-null values), nor the near-threshold OVERFLOW boundary. Separately worth noting (not flagged inline, as it appears intentional and is documented as requiring a shared mount): offloading invoke payloads/results is on by default under a global offloader, which requires the invoked function to run a compatible SDK + offloader with access to the same storage, or it will fail to parse the envelope.

Reviewed commit 625b1d3c4d3f01dcc1a9dc3dc451817f5c0832ff. Workflow run

[github-actions]

Codex AI review · Finding arf_v1_f3atcijds7gzk56a5jhfbcgnaz

[P1] Latch payload failures before exposing them to handler code. On replay, this runs inside DurableFuture.get(). A handler can catch a load, digest, or ownership PayloadOffloadException and return a fallback, allowing SUCCEEDED despite unreadable checkpoint state. Call ExecutionManager.failInvocation before rethrowing operation codec failures, and test a handler that catches a failing replay load.

[github-actions]

Codex AI review · Finding arf_v1_etwwdfzpfwr42v2wd3posft7nz

[P1] Claim ordinary checkpoint failures while holding this lock. A custom DurableExecutionClient.checkpoint may throw a plain runtime exception, which bypasses this UnrecoverableDurableExecutionException branch. The lock is then released with completion still available, allowing a waiting coordinator to persist early parent success before the child terminates. Normalize all checkpoint publication failures to an invocation-level type or otherwise distinguish and claim them here; add a one-shot IllegalStateException race test.

[github-actions]

Codex AI review · Finding arf_v1_im5jt6pypylf7lqfdngnda7mil

[P2] Do not offload transient condition initialization. This helper now offloads, but WaitForConditionOperation.start() still uses it only to normalize initialState; that envelope is never checkpointed. Each first execution can therefore create an orphan external payload and fail on unnecessary storage access before the first check. Use normalizeResult for the initial state and test the offload count and payload contexts.

[github-actions]

Codex AI review

Found three correctness issues: payload failures can be masked, checkpoint races can persist invalid parent success, and condition initialization creates orphan offloads.

Reviewed commit a52a63745d2f0ef2031adef2aeb872568f0fafbf. Workflow run