Skip to content

Add task instance management commands to airflowctl CLI#67947

Open
Suraj-kumar00 wants to merge 10 commits into
apache:mainfrom
Suraj-kumar00:feat/task-instance-cli-support
Open

Add task instance management commands to airflowctl CLI#67947
Suraj-kumar00 wants to merge 10 commits into
apache:mainfrom
Suraj-kumar00:feat/task-instance-cli-support

Conversation

@Suraj-kumar00
Copy link
Copy Markdown

Adds Task Instance management support to the airflowctl CLI, enabling users to get, list, clear, and update task instances directly from the terminal.

Closes: #61547
Related: #66173

Changes

airflowctl/api/operations.py

  • Added TaskInstanceOperations class with get, list, clear, update methods
  • Uses a _parse_task_instance_response helper to handle the different response shapes (single instance, list, or collection) from the API
  • Restored PluginsOperations class that was lost during merge with main (Added plugins command to airflowctl #64935)

airflowctl/api/client.py

  • Added task_instances property to the Client for auto-generated CLI command wiring
  • Restored plugins property that was lost during merge with main

airflowctl/ctl/cli_config.py

  • Added "clear" to output_command_list for correct CLI output handling
  • Added _is_list_annotation() helper that properly unwraps Annotated[list[...] | None, Field(...)] to convert comma-separated CLI inputs into lists when the Pydantic model expects a list field
    (e.g. --task-ids t1,t2 → ["t1", "t2"])
  • Excluded _parse_task_instance_response from auto-generated CLI commands

tests/airflow_ctl/api/test_operations.py

  • Added TestTaskInstanceOperations with 10 unit tests covering all response shapes for get, list, clear, and update
  • Restored TestPluginsOperations tests lost during merge

tests/airflow_ctl/ctl/commands/test_pool_command.py

  • Moved import re to module scope

airflow-ctl-tests/tests/airflowctl_tests/test_airflowctl_commands.py

  • Added end-to-end integration tests for all 4 taskinstance commands

New CLI Commands

airflowctl taskinstance list   --dag-id <dag_id> --dag-run-id <run_id>
airflowctl taskinstance get    --dag-id <dag_id> --dag-run-id <run_id> --task-id <task_id>
airflowctl taskinstance clear  --dag-id <dag_id> [--dry-run] [--only-failed] [--task-ids ...]
airflowctl taskinstance update --dag-id <dag_id> --dag-run-id <run_id> --task-id <task_id> [--new-state <state>] [--note <note>]
image image
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@Suraj-kumar00 Suraj-kumar00 changed the title Feat/task instance cli support Add task instance management commands to airflowctl CLI Jun 3, 2026
@Suraj-kumar00 Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from bfb913a to 193ed6a Compare June 3, 2026 10:57
@Suraj-kumar00
Copy link
Copy Markdown
Author

Hello @potiuk,
Could you please review now and the CI Pipeline can be run?

Also could you help me get this that do i have to do the testing manually on local? This is taking so much time locally!!!

@bugraoz93 bugraoz93 added the full tests needed We need to run full set of tests for this PR to merge label Jun 3, 2026
@bugraoz93 bugraoz93 closed this Jun 3, 2026
@bugraoz93 bugraoz93 reopened this Jun 3, 2026
Copy link
Copy Markdown
Contributor

@bugraoz93 bugraoz93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some foundational things needs changing, thanks for the PR!

"taskinstance clear --dag-id=example_bash_operator --dry-run",
'taskinstance update --dag-id=example_bash_operator --dag-run-id="manual__{date_param}" --task-id=runme_0 --new-state=success',
# XCom commands - need a DAG run with completed tasks
'xcom add --dag-id=example_bash_operator --dag-run-id="manual__{date_param}" --task-id=runme_0 --key={xcom_key} --value=\'{{"test": "value"}}\'',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if fields are mandatory they should be path parameter without any param name to be optional. Can you rebase? Maybe you don't have those. Because we shouldn't update old commands in this work, they should work as is and we should ensure required fields are not passed as optional for taskinstance commands too

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, all required params (dag_id, dag_run_id, task_id, key, value) are now positional args in the test commands, matching how CommandFactory generates them. Rebased on main.




class TaskInstanceOperations(BaseOperations):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operations name shpuld be tasks as this auto generated should be same as Airlfow core cli

This creating is correct even though the static checks failing and PR not finished. Please check the operation class naming
https://github.com/apache/airflow/pull/66926/changes

return False


def _parse_task_ids_cli_arg(value: str) -> list:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks bespoke to me, I don't think we should put task base exceptional bespoke implementations here. We should add them to operation side if needed, I assume we shouldn't have this much changes here

@Suraj-kumar00 Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from 0b76de6 to 400f426 Compare June 3, 2026 18:56
@Suraj-kumar00 Suraj-kumar00 requested a review from jscheffl as a code owner June 3, 2026 19:22
…tanceOperations

- Remove bare re-raise try/except from get, clear, and update methods
- Fix ruff E302: add missing blank line before TaskInstanceOperations class
- Fix ruff F841: remove unused variable in pool command export test
- Run ruff format on operations.py, cli_config.py, test_operations.py
…onal args

Required path params (dag_id, dag_run_id, task_id, key, value) are
generated as positional CLI arguments by CommandFactory, not optional
--flags. Also fix ruff-format: remove extra blank lines before
TaskInstanceOperations and TestTaskInstanceOperations class definitions.
@Suraj-kumar00 Suraj-kumar00 force-pushed the feat/task-instance-cli-support branch from 4da1910 to a584bd1 Compare June 3, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants