WebHDFSHook Bugfix/optional port
#24550
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
|
|
Hey, Can you please enable the workflow which is awaiting approval. It will help me understand if If there is anything else needed in code and then I can work on the documentation of the change. |
| if extra_dejson.get('use_ssl', False): | ||
| connection_str = f'https://{namenode}:{port}' | ||
| session.verify = extra_dejson.get('verify', True) | ||
| if extra_dejson.get('use_ssl', 'False') == 'True': |
There was a problem hiding this comment.
I think connection extra can be both strings as well as dict/JSON so not sure, actually, this change requires or not https://github.com/apache/airflow/blob/main/airflow/models/connection.py#L119 .
There was a problem hiding this comment.
When using the application , it takes json input from the UI where it only accepts String . Should I change it to accept both ?
|
Can you please add some description like case or some dummy extra where it not working |
Do you want me to add more tests ? bascially when you send a string "False" from the UI , it doesnt work , you have to send an empty string to make it work |
…airflow into bugfix/optionalPort
|
Thanks for the run , I have fixed all the issues that I could find. Please enable the workflows awaiting approval once again to check if everything is working fine |
|
Awesome work, congrats on your first merged pull request! |
closes: #24022
This requests makes the port number in a connection to hdfs as optional. It also allowes user to add schema to the url if he wants to.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragement file, named
{pr_number}.significant.rst, in newsfragments.