Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add decorators for external and venv python branching operators #35043

Merged
merged 6 commits into from Oct 20, 2023

Conversation

jscheffl
Copy link
Contributor

@jscheffl jscheffl commented Oct 18, 2023

This PR is a follow-up of PR #33356 to add also python decorators for additional branching tasks for:

  • external python
  • python virtual environments
    To make the feature set complete also for taskflow style DAGs.

How to test?

  • See docs and in general CI is green
  • Run the example_branch_operator
  • Run the example_branch_python_operator_decorator

@jscheffl jscheffl requested a review from potiuk as a code owner October 18, 2023 21:50
@jscheffl jscheffl added type:new-feature Changelog: New Features area:core-operators Operators, Sensors and hooks within Core Airflow labels Oct 18, 2023
@jscheffl jscheffl added this to the Airflow 2.8.0 milestone Oct 18, 2023
@jscheffl jscheffl requested review from uranusjr and potiuk and removed request for potiuk October 18, 2023 21:51
@potiuk
Copy link
Member

potiuk commented Oct 18, 2023

Looks really nice. But how about adding Howtos/Examples to https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html folllowing other operators? I think otherwise they will not be easily discoverable

@jscheffl
Copy link
Contributor Author

Looks really nice. But how about adding Howtos/Examples to https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html folllowing other operators? I think otherwise they will not be easily discoverable

Good idea... should have had the idea myself with examples... DIFF is larger now. I re-worked the existing branching examples and made it all together in one place - let me know if this was the correct idea.
See in "How to test" which DAGs as example are modified now. Also made this for the classic operators for consistency.

@potiuk
Copy link
Member

potiuk commented Oct 20, 2023

tests need fixing but It LGTM otherwise.

@potiuk
Copy link
Member

potiuk commented Oct 20, 2023

All Good. Failed check fixed in main,

@potiuk potiuk merged commit c2a733a into apache:main Oct 20, 2023
43 of 44 checks passed
@jscheffl jscheffl deleted the feature/add-python-branch-decorators branch October 28, 2023 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core-operators Operators, Sensors and hooks within Core Airflow kind:documentation type:new-feature Changelog: New Features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants