New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add deferrable mode to DbtCloudRunJobOperator
#29014
Conversation
def provide_account_id(func: T) -> T: | ||
""" | ||
Decorator which provides a fallback value for ``account_id``. If the ``account_id`` is None or not passed | ||
to the decorated function, the value will be taken from the configured dbt Cloud Airflow Connection. | ||
""" | ||
function_signature = signature(func) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem with this wrapper (as well as other same decorators which already exists in community providers) that every time we access to method which decorated by this function we need to lookup for connection in Secrets Backend (if configured), Environment Variables and Airflow Database.
Is any way to make it cashable? Something like that: #28716
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will check if it can be made cacheable
165ea0c
to
834bb09
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's do caching in a separate PR
Related: apache#28890 apache#29014 There was a recent enhancement of DbtCloudRunJobOperator to include deferrable/async functionality. Unfortunately the `tenant` evaluation in the DbtCloudHook was outdated and didn't include the most recent change to properly handle domain specification. This PR consolidates the tenant eval logic to a common method to be used by both sync and async methods in the hook.
* Correct tenant eval within async logic of DbtCloudHook Related: #28890 #29014 There was a recent enhancement of DbtCloudRunJobOperator to include deferrable/async functionality. Unfortunately the `tenant` evaluation in the DbtCloudHook was outdated and didn't include the most recent change to properly handle domain specification. This PR consolidates the tenant eval logic to a common method to be used by both sync and async methods in the hook. * Remove Connection.schema use * Update provider.yaml and changelog
This PR donates the
DbtCloudRunJobOperatorAsync
from astronomer-providers repo^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.