Skip to content

Add Dag run context fields to the Java SDK Context#69100

Open
FrankYang0529 wants to merge 1 commit into
apache:mainfrom
FrankYang0529:airflow-java-context
Open

Add Dag run context fields to the Java SDK Context#69100
FrankYang0529 wants to merge 1 commit into
apache:mainfrom
FrankYang0529:airflow-java-context

Conversation

@FrankYang0529

@FrankYang0529 FrankYang0529 commented Jun 28, 2026

Copy link
Copy Markdown
Member

The Java SDK's Context only exposed dagId and runId from the Dag run, even though the supervisor already delivers the full run context in the same StartupDetails that Python tasks receive. Add fields logicalDate, dataIntervalStart / dataIntervalEnd, runAfter, ds, runType, and conf to Context.


Was generative AI tooling used to co-author this PR?
  • Yes - Claude Code with Opus 4.8

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@FrankYang0529 FrankYang0529 marked this pull request as ready for review June 30, 2026 00:32
@uranusjr

Copy link
Copy Markdown
Member

I was partly doing this intentionally since all those context keys are quite redundant IMO. Maybe it’d be easier if we create a DagRun object (from the model in ti context, but not that type directly; similar to how Connection and Variable are bridged) to build Context with, and tell users to just use e.g. context.dagRun.runId instead?

Note that Java SDK does not need to support Jinja templating, so a top-level key in the context is not particularly useful.

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was partly doing this intentionally since all those context keys are quite redundant IMO. Maybe it’d be easier if we create a DagRun object (from the model in ti context, but not that type directly; similar to how Connection and Variable are bridged) to build Context with, and tell users to just use e.g. context.dagRun.runId instead?

Isn't this PR doing exactly what the above part you mentioned?

Note that Java SDK does not need to support Jinja templating, so a top-level key in the context is not particularly useful.

Or do you mean since Java SDK does not need to support Jinja templating so we don't need to expose logicalDate, dataIntervalStart, etc, in the context?

IMO, no harm to expose those snapshotted attributes to user in the injected context.

Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Context.kt Outdated
@FrankYang0529 FrankYang0529 force-pushed the airflow-java-context branch from 9a8c2d1 to cc2a671 Compare July 2, 2026 13:40
@FrankYang0529

Copy link
Copy Markdown
Member Author

I was partly doing this intentionally since all those context keys are quite redundant IMO. Maybe it’d be easier if we create a DagRun object (from the model in ti context, but not that type directly; similar to how Connection and Variable are bridged) to build Context with, and tell users to just use e.g. context.dagRun.runId instead?

Agree, I use DagRun object (context.dagRun.dataIntervalStart) rather than top-level context keys, so I think we're aligned there.

Note that Java SDK does not need to support Jinja templating, so a top-level key in the context is not particularly useful.

Yes, Java tasks have no Jinja/operator-templating path, so a task doing a backfill by data interval can only reach those through the context object.

Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/execution/Logger.kt Outdated
Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Context.kt Outdated
Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Context.kt Outdated
Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Context.kt Outdated
Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Context.kt Outdated
@FrankYang0529 FrankYang0529 force-pushed the airflow-java-context branch from cc2a671 to 2b65abe Compare July 3, 2026 07:06
Java tasks could only read the dag id and run id from Context, even though the
supervisor already sends the full run context (logical date, data interval, run
type, trigger conf) in StartupDetails.

Signed-off-by: PoAn Yang <payang@apache.org>
@FrankYang0529 FrankYang0529 force-pushed the airflow-java-context branch from 2b65abe to 2b008ee Compare July 3, 2026 09:11
@jason810496 jason810496 added the AIP-108: java-sdk Change this to an 'area:' label after AIP acceptance. label Jul 6, 2026
Comment on lines +56 to +59
runCatching { RunType.valueOf(dagRunType.name) }.getOrElse {
logger.warning("Ignoring unrecognized run type in run context", mapOf("value" to dagRunType.name))
null
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would drift out of sync too easily if DagRunType receives new values. Use a simple when to make the compiler check for missing items instead.

}
}

enum class RunType {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would name this DagRunType to match the API and the Python type.

@Test
fun fromDecodesRunContextFields() {
// [2, msg, null] with msg coming from
// https://github.com/astronomer/airflow/blob/f39c8da8/task-sdk/tests/task_sdk/execution_time/test_comms.py#L73-L108

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference the apache/airflow repo instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIP-108: java-sdk Change this to an 'area:' label after AIP acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants