[runtime] Reject RuntimeExecutionMode.BATCH with the default batch state backend - #1104
Open
Ashfaqbs wants to merge 1 commit into
Open
[runtime] Reject RuntimeExecutionMode.BATCH with the default batch state backend#1104Ashfaqbs wants to merge 1 commit into
Ashfaqbs wants to merge 1 commit into
Conversation
…ate backend ActionExecutionOperator keeps pending action tasks in keyed state across mailbox continuations that can span multiple keys of the same subtask. Flink's batch keyed-state backend (execution.batch-state-backend.enabled, enabled by default in BATCH mode) assumes a key is fully processed before the operator moves to the next one and clears keyed state on every key switch, so a continuation for one key can be silently discarded when the operator processes another key first. The job still reports FINISHED; only the affected records go missing. CompileUtils.connectToAgent now fails fast with a clear error at job-construction time when it detects an explicitly configured BATCH runtime mode together with the batch state backend enabled, instead of letting the job silently drop records. Points the caller at the two documented ways out: RuntimeExecutionMode.STREAMING, or explicitly setting execution.batch-state-backend.enabled to false. This only catches an explicitly configured BATCH mode; AUTOMATIC mode that later resolves to batch execution because every source happens to be bounded is not detected, since that resolution has not happened yet at graph-construction time. Fixes apache#939 Generated-by: Claude Code 2.1.226 (Claude Sonnet 5)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: #939
Purpose of change
A job configured with
RuntimeExecutionMode.BATCHand Flink's batch keyed-state backend (execution.batch-state-backend.enabled, enabled by default inBATCHmode) now fails immediately at job-graph construction with a clear error, instead of running to completion and silently dropping records.ActionExecutionOperatorkeeps pending action tasks in keyed state across mailbox continuations that can span multiple keys of the same subtask. The batch keyed-state backend assumes a key is fully processed before the operator moves to the next one and clears keyed state on every key switch, so a continuation for one key can be discarded when the operator processes another key first — the job still reportsFINISHED; only the affected records go missing.Runtime flow:
CompileUtils.connectToAgent(both the Python and Java entry points funnel into the same private overload) now callscheckBatchStateBackendCompatibilitybefore constructing theActionExecutionOperator. That readsExecutionOptions.RUNTIME_MODEandExecutionOptions.USE_BATCH_STATE_BACKENDoffkeyedInputStream.getExecutionEnvironment().getConfiguration()and throws before the operator is ever wired into the graph.Key decisions:
BATCHmode is detected.RuntimeExecutionMode.AUTOMATIC(the default) that later resolves to batch execution because every source happens to be bounded is not caught, since that resolution hasn't happened yet at graph-construction time — inspecting source boundedness to predictAUTOMATIC's eventual resolution was rejected as disproportionate for a case the explicit, documentedBATCHsetting already covers.Behavioral Semantics
Interaction decisions
RUNTIME_MODEUSE_BATCH_STATE_BACKENDSTREAMINGtrue/falseBATCHBATCHtrue(default)IllegalStateExceptionbefore the operator is constructedBATCHfalseAUTOMATICtrue/falseBehavioral contracts
connectToAgentthrowsIllegalStateExceptionbefore constructing the operator whenRUNTIME_MODE=BATCHandUSE_BATCH_STATE_BACKEND=true.connectToAgentbehaves exactly as before this change for every otherRUNTIME_MODE/USE_BATCH_STATE_BACKENDcombination.RuntimeExecutionMode.STREAMING, or settingexecution.batch-state-backend.enabled=false) and links to [Bug] Bounded workflows silently drop records in batch mode with multiple keys #939.Failure behavior
The only new failure path: an unsupported
BATCH+ batch-state-backend-enabled configuration raisesIllegalStateExceptionsynchronously, before any operator is created — not a retry, not silently absorbed. For the Python entry point, this check runs after the existing pickled-input-type and agent-plan-JSON validations, so it doesn't change their ordering or messages. No other failure path changes.Tests
rejectsExplicitBatchModeWithDefaultBatchStateBackendallowsExplicitBatchModeWithBatchStateBackendDisabledtestJavaNoKeyedStreamConnectToAgent/testJavaKeyedStreamConnectToAgentrejectsExplicitBatchModeWithDefaultBatchStateBackendasserts the message contains bothRuntimeExecutionMode.BATCHandexecution.batch-state-backend.enabledNot verified:
AUTOMATICmode resolving to batch execution via all-bounded sources; an actual executedBATCHjob reproducing the original multi-key data-loss scenario (would need a heavier MiniCluster run) — the causal chain for why the collision happens relies on the issue's own trace of Flink's batch state-backend semantics, not a reproduction inside this PR.Verification evidence
Ran
CompileUtilsTest(9/9 pass) plus the fulloperator,context, andskillpackages after a clean rebuild (mvn -pl runtime clean, thenmvn -pl api,plan -am install -DskipTeststo pick up unrelated upstream API changes already onmain). 249 tests, 7 pre-existing failures unrelated to this change: 4 are a Windows-only temp-directory-deletion quirk inClasspathSkillRepositoryTest(locked JAR files), and 3 (FileSystemSkillRepositoryTest,SkillManagerTestx2) pick up this machine's own local Claude Code skills directory contents (github,nano-banana-pro) rather than the repo's test fixtures — both reproduce identically on unmodifiedmainin this environment.mvn -pl runtime spotless:checkclean.API
No public API changes.
connectToAgent's signature and behavior for every previously-supported configuration are unchanged; this adds a new, additive rejection for a configuration that was previously accepted but silently broken.Documentation
doc-not-neededWas this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code 2.1.226 (Claude Sonnet 5)