Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update airflow tasks * commands to lookup TaskInstances from DagRun Table #16030

Merged
merged 22 commits into from
Jun 17, 2021
Merged

Update airflow tasks * commands to lookup TaskInstances from DagRun Table #16030

merged 22 commits into from
Jun 17, 2021

Conversation

ephraimbuddy
Copy link
Contributor

@ephraimbuddy ephraimbuddy commented May 24, 2021 •
edited

This change adds the ability to lookup TaskInstances using the DagRun.run_id

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@mik-laj
Copy link
Member

mik-laj commented May 24, 2021

It looks like a breaking change. Am I right?

@github-actions
Copy link

The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the workflow link to check the reason.

@ephraimbuddy
Copy link
Contributor Author

ephraimbuddy commented May 24, 2021 •
edited

It looks like a breaking change. Am I right?

I can't really tell right now. From what I understand, the execution date sent to the tasks command is not really used in the context of execution date but to just get TaskInstance. I will shed more light as I work on this.

Update:
I have allowed backward compatibility @mik-laj

@ephraimbuddy ephraimbuddy force-pushed the modify-task-commands branch 2 times, most recently from 2836b62 to 3dbfdb3 Compare May 28, 2021 17:53
@ephraimbuddy ephraimbuddy marked this pull request as ready for review May 28, 2021 22:09
@ashb ashb added the AIP-39 Timetables label May 28, 2021
@ashb ashb self-assigned this Jun 1, 2021
uranusjr
uranusjr requested changes Jun 1, 2021
View reviewed changes
airflow/cli/commands/task_command.py Show resolved Hide resolved
airflow/cli/commands/task_command.py Outdated Show resolved Hide resolved
airflow/cli/commands/task_command.py Outdated Show resolved Hide resolved
airflow/models/dag.py Outdated Show resolved Hide resolved
tests/cli/commands/test_task_command.py Show resolved Hide resolved
ashb
ashb reviewed Jun 1, 2021
View reviewed changes
Copy link
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(small changes, fuller review comming)

airflow/cli/commands/task_command.py Outdated Show resolved Hide resolved
airflow/models/dag.py Outdated Show resolved Hide resolved
@github-actions
Copy link

github-actions bot commented Jun 7, 2021

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Jun 7, 2021
ephraimbuddy and others added 19 commits June 16, 2021 00:20
@ephraimbuddy
fixup! fixup! fixup! Update airflow task * commands to lookup TaskIns…
0b4f830 
…tances from DagRun Table
@ephraimbuddy
add some tests
f47d7f7
@ephraimbuddy
fixup! add some tests
c764592
@ephraimbuddy
fixup! fixup! add some tests
bf5d699
@ephraimbuddy @ashb
Apply suggestions from code review
2d84b82 
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
@ephraimbuddy
fixup! Apply suggestions from code review
65432d9
@ephraimbuddy
fixup! fixup! Apply suggestions from code review
71a6d22
@ephraimbuddy @ashb
Update airflow/cli/commands/task_command.py
6278169 
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
@ephraimbuddy
add TI attributes
2093151
@ephraimbuddy
Apply suggestions from code review
cef27a1
@ephraimbuddy
Update airflow task * commands to lookup TaskInstances from DagRun Table
fef4a59 
This change removes execution_date from task commands and uses DagRun.run_id instead
@ephraimbuddy
fixup! Update airflow task * commands to lookup TaskInstances from Da…
a3acf58 
…gRun Table
@ephraimbuddy
fixup! fixup! Update airflow task * commands to lookup TaskInstances …
1b145bb 
…from DagRun Table
@ephraimbuddy
add some tests
44c0424
@ephraimbuddy
fixup! fixup! add some tests
3992d05
@ephraimbuddy @kaxil
Update airflow/cli/commands/task_command.py
2efa0a8 
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@ephraimbuddy
fix conflict
7767e96
@ephraimbuddy
Remove duplicate console print introduced during conflict resolution
94b616a
@ephraimbuddy
remove minscheduler in task test
df6293b
@ephraimbuddy ephraimbuddy force-pushed the modify-task-commands branch 2 times, most recently from a056f8f to 0bca458 Compare June 16, 2021 08:29
AIP-39 Pluggable schedule_interval automation moved this from In progress to Done Jun 16, 2021
@ephraimbuddy ephraimbuddy reopened this Jun 16, 2021
AIP-39 Pluggable schedule_interval automation moved this from Done to In progress Jun 16, 2021
@ephraimbuddy ephraimbuddy merged commit 26b3440 into apache:main Jun 17, 2021
AIP-39 Pluggable schedule_interval automation moved this from In progress to Done Jun 17, 2021
@ephraimbuddy ephraimbuddy deleted the modify-task-commands branch June 17, 2021 10:46
@ashb ashb added this to the Airflow 2.2 milestone Jun 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@uranusjr uranusjr uranusjr requested changes

@ashb ashb ashb approved these changes

@kaxil kaxil kaxil approved these changes

@turbaszek turbaszek Awaiting requested review from turbaszek

@XD-DENG XD-DENG Awaiting requested review from XD-DENG XD-DENG is a code owner

@dimberman dimberman Awaiting requested review from dimberman

@mik-laj mik-laj Awaiting requested review from mik-laj

@potiuk potiuk Awaiting requested review from potiuk

@vikramkoka vikramkoka Awaiting requested review from vikramkoka

Assignees

@ashb ashb

Labels
AIP-39 Timetables area:CLI full tests needed We need to run full set of tests for this PR to merge
Projects
No open projects
AIP-39 Pluggable schedule_interval
Done
Milestone
Airflow 2.2.0
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@ephraimbuddy @mik-laj @ashb @uranusjr @kaxil