-
Notifications
You must be signed in to change notification settings - Fork 13.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Making SFTPHook's constructor consistent with its superclass SSHHook #20164
Conversation
I think adding a test for that would be nice too. |
(and fixing static checks - I heartily recommend installing pre-commit. It helps with the iterations a LOT. |
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. |
Looks good. I thin what it needs is a bit more docstring description on which parameter is preferred but looks good otherwise. |
You can also take a look at examples and see which connection is used there (and align with the preferred one) |
I am writing some tests and I will send a complete PR as soon as I'm done!
Do you mean example dags? |
it should be possible to create tests for the two versions of the hook constructor using 'parametrized'
4b83f56
to
5c7f413
Compare
Yep. |
I didn't find any example dag including the |
😱 How about adding one ? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah.. I think example for that one can be added later :)
Will do! |
I believe this introduces a regression on any use of the SFTPHook that doesn't set a specific sftp_conn_id. Since the parameter has a default value, this is then used to override ssh_conn_id with it. This happens for instance on the SFTPToGCSOperator. It worked on version 2.3.0 of the provider, but now on 2.4.0, no matter what connection is passed in its constructor, the hook will use "sftp_default". One simple fix would be to remove the default value on sftp_conn_id but I'm not sure it's completely safe and backwards compatible. Or maybe switch the order of the two parameters so that sftp_conn_id is first. What do you think ? |
This should be fixed by #20756 |
Yeah. I yanked the 2.4.0 release and 2.4.1rc1 with the fix included is up for voting. We will release it likely on Wednesday |
Thanks Jarek! Sorry for the mistake in my PR |
No worries :) Happens. That's why we have so flexible mechanims and split the release to separate providers that we can easily handle those kind of situations :). |
This is just a first dummy solution that probably scores low for elegance since I'm making both the parameters Optional.
What do you guys think? I will adjust according to your suggestions :)
related issue: #20144
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.