Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deferrable mode for RunPipelineJobOperator #37969

Conversation

e-galan
Copy link
Contributor

@e-galan e-galan commented Mar 7, 2024

List of changes:

  • Add the deferrable mode to RunPipelineJobOperator;
  • Move persisting the pipeline job link to the beginning of the operator's execution;
  • Add new unit tests, refactor some of the existing unit tests.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:providers provider:google Google (including GCP) related issues labels Mar 7, 2024
@VladaZakharova
Copy link
Contributor

Hi @potiuk @eladkal @Lee-W !
Can you please check the changes in this PR? Thank you so much!

Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good to me. Left a few nitpicks

@e-galan e-galan force-pushed the add-deferrable-mode-to-run-pipeline-job-operator branch from 1d54a9b to f482fd9 Compare March 18, 2024 10:25
@e-galan e-galan requested a review from Lee-W March 19, 2024 10:41
@e-galan e-galan force-pushed the add-deferrable-mode-to-run-pipeline-job-operator branch from f482fd9 to dd46e98 Compare March 21, 2024 10:11
@VladaZakharova
Copy link
Contributor

Hi @potiuk !
Can you please check changes here? :) Thanks you <3

Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few nitpicks. but overall looks good to me

@e-galan e-galan force-pushed the add-deferrable-mode-to-run-pipeline-job-operator branch from dd46e98 to 18272e6 Compare March 25, 2024 22:58
@VladaZakharova
Copy link
Contributor

Hi @potiuk @eladkal!
Can you please merge the changes in this PR? Thank you!

@potiuk potiuk merged commit 067fedb into apache:main Mar 26, 2024
46 checks passed
Copy link

boring-cyborg bot commented Mar 26, 2024

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants