Skip to content

Enable mapped task argument overrides in airflow tasks test#67356

Open
kevinhongzl wants to merge 2 commits into
apache:mainfrom
kevinhongzl:fix-task-test-mapped-param-override
Open

Enable mapped task argument overrides in airflow tasks test#67356
kevinhongzl wants to merge 2 commits into
apache:mainfrom
kevinhongzl:fix-task-test-mapped-param-override

Conversation

@kevinhongzl
Copy link
Copy Markdown
Contributor

Summary

Task parameters were immutable when debugging dynamic task mappings using the CLI airflow tasks test. This meant we couldn't override parameters on the fly, nor could we easily test mapped tasks that relied on upstream XCom arguments. This PR updates the Airflow CLI to support local testing of dynamically mapped tasks.

closes: #56953

Params Overrides Behavior

The tables below outline how our updated airflow tasks test handles combinations of expansion types, argument sources, and the presence of CLI task parameter overrides.

1. Positional Arguments from expand()

Argument Type With CLI Task Params Without CLI Task Params
Literal Values Overrides the original task parameters with the single provided parameter value. Runs tests using the original values defined for that specific map index in the DAG.
XCom Arguments Uses the user-provided input value to successfully execute and test the task. Fails immediately with the expected XCom not found exception.

2. Keyword Arguments from expand_kwargs()

Argument Type With CLI Task Params Without CLI Task Params
Literal Values Updates the mapped keyword arguments for that index with the new CLI task parameters. Runs tests using the baseline keyword arguments defined for that map index in the DAG.
XCom Arguments Uses the user-provided input dictionary to successfully execute and test the task. Fails immediately with the expected XCom not found exception.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@kevinhongzl kevinhongzl force-pushed the fix-task-test-mapped-param-override branch from 174b645 to fd301cc Compare May 22, 2026 19:08
@kevinhongzl kevinhongzl force-pushed the fix-task-test-mapped-param-override branch from fd301cc to ede42de Compare May 23, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

'airflow tasks tests' for mapped tasks with dependencies

1 participant