Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dag (un)pausing won't work on environment where dag files are missing #40345

Merged
merged 1 commit into from
Jul 1, 2024

Conversation

boushphong
Copy link
Contributor

closes: #38834

#38265 added bulk pause and resume of DAGs. However, this PR seems to reuse the cli util method that collects DAGs from the default dag folder but not from the metadata DB. Hence, this would cause the unpause command to fail on environments where the dag folder is missing.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@eladkal eladkal added this to the Airflow 2.9.3 milestone Jun 20, 2024
@eladkal eladkal added the type:bug-fix Changelog: Bug Fixes label Jun 20, 2024
@boushphong boushphong marked this pull request as draft June 20, 2024 14:32
@boushphong boushphong marked this pull request as ready for review June 20, 2024 15:08
@boushphong boushphong changed the title Fix dag pausing won't work on environment where dag files are missing Fix dag (un)pausing won't work on environment where dag files are missing Jun 20, 2024
@boushphong
Copy link
Contributor Author

@ephraimbuddy Could you take a look? 🙇

Copy link
Contributor

@shahar1 shahar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, but I'd be happy if @ephraimbuddy could double-check.
I reused this logic from the backfilling method, does it make sense to have it also there?

@boushphong
Copy link
Contributor Author

LGTM overall, but I'd be happy if @ephraimbuddy could double-check. I reused this logic from the backfilling method, does it make sense to have it also there?

@shahar1 I would say so.

Maybe you could submit a Pull Request for the backfilling part? 🚀
I would love to look into it though. Which part of the code should I look into?

@ephraimbuddy ephraimbuddy merged commit e3d62c3 into apache:main Jul 1, 2024
49 checks passed
@shahar1
Copy link
Contributor

shahar1 commented Jul 1, 2024

LGTM overall, but I'd be happy if @ephraimbuddy could double-check. I reused this logic from the backfilling method, does it make sense to have it also there?

@shahar1 I would say so.

Maybe you could submit a Pull Request for the backfilling part? 🚀 I would love to look into it though. Which part of the code should I look into?

dag_backfill method, around line 157 :)
I'll try to work it out later

utkarsharma2 pushed a commit that referenced this pull request Jul 2, 2024
…sing (#40345)

closes: #38834

#38265 added bulk pause and resume of DAGs. However, this PR seems to reuse the cli util method that collects DAGs from the default dag folder but not from the metadata DB. Hence, this would cause the unpause command to fail on environments where the dag folder is missing.
(cherry picked from commit e3d62c3)
ephraimbuddy pushed a commit that referenced this pull request Jul 2, 2024
…sing (#40345)

closes: #38834

#38265 added bulk pause and resume of DAGs. However, this PR seems to reuse the cli util method that collects DAGs from the default dag folder but not from the metadata DB. Hence, this would cause the unpause command to fail on environments where the dag folder is missing.
(cherry picked from commit e3d62c3)
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
…sing (apache#40345)

closes: apache#38834

apache#38265 added bulk pause and resume of DAGs. However, this PR seems to reuse the cli util method that collects DAGs from the default dag folder but not from the metadata DB. Hence, this would cause the unpause command to fail on environments where the dag folder is missing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:CLI type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cli | airflow dags unpause - fail cause not founding dags
4 participants