-
Notifications
You must be signed in to change notification settings - Fork 14.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not require all extras for SalesforceHook #19530
Conversation
06a2487
to
491f6cc
Compare
a56f7e3
to
c01cf43
Compare
c01cf43
to
5c54b47
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, pending the cached_property import fix
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. |
5c54b47
to
96c2227
Compare
The import is fixed |
excellent, thank you |
Thank you @alexbegg for getting this across the line! |
thanks for the change. in which version this will be released? https://pypi.org/project/apache-airflow-providers-salesforce/#history |
In the next one released ~ the end of the month |
and was changing extra parameter names absolutely necessary? I have to change this 40 times in our airflow setup, not everyone is creating connections from ui.
|
I totally feel your pain. I do not like the decision. My very uninformed guess is that more often than not creds are not managed in UI and I would prefer if we never implemented the UI customizations. I think @alexbegg may also have been working on a PR to make it so that you can optionally use the "normal" names instead. But clearly that won't help you in time. BUT you have an option. Since all the conn extra parsing is done in |
Yes, I am planning to make a separate PR to have There is a current discussion on the dev list about why the names changed: https://lists.apache.org/thread/ofwxrx7rcwcn2h6z6m51bfymoqktnbbq |
This PR is using the same extra names as the previous version of the method. Once I make the PR I'll link it here. Also I'm a bit busy at the moment if someone want to handle making a PR in my place for that I'll appreciate that! |
The
SalesforceHook
was previously requiring all connection extras, however with secrets backends it is now possible to provide one extra at a time. The current setup was directly accessing keys of the extras json which would result in aKeyError
if you leave out one of them.This PR allows for just the minimum required extras to be provided for each Salesforce connection method as defined in the docstring of the
SalesforceHook
:airflow/airflow/providers/salesforce/hooks/salesforce.py
Lines 55 to 60 in e9a72a4
In addition to allowing for only the required extras it also still allows for all of the extras to be provided, such as the case currently when the connection is added via the UI.
When making the connection the hook will explicitly default to
None
for all extras except for "version" (which defaults to the default API version). The reason for this is becausesimple-salesforce
already has built-in authentication-choosing methods that relies on which arguments areNone
and without havingor None
in the code then setting this connection in the UI will result in the blank extras being empty strings instead of None, which would break the connection ifget
was used on its own.The tests for the hook are also updated to only test for the required connection params when testing connections and also test that if all extras are empty strings they will be defaulted to
None
.Also, a thank you @gigatexal, @dstandish, @josh-fell for prior PRs and PR discussions regarding this fix. This PRs incorporates the discussed points.
closes: #19506
completes PR #18929 by adding tests
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.