-
Notifications
You must be signed in to change notification settings - Fork 14.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add missing conn_id to string representation of ObjectStoragePath #39313
add missing conn_id to string representation of ObjectStoragePath #39313
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
5acc478
to
cb98b09
Compare
Hello @eladkal 🙂 Very sorry for the late poke about this but is there a chance to include a 1.3.2 release of the common io provider with this bugfix in the current wave of releases (or the next one, since the May 12th one already two votes)? cc: @pankajkoti |
If I'm not mistaken, we'll need to wait for the next airflow release (not provider release) to include this fix. |
This is Airflow core not providers. |
Oh! Apologies 🤦 . Thank you! |
Just a security concern, is it too risky if we add conn info to xcom value return to UI? IMO xcom values should not have any sensitive value relate to internal process can be misuse/leak when user reach that, and user still need to access xcom value to debug/check etc so we not remove |
As long as there's no credential, I think we're good. But it really depends on what you think sensitive values are 🤔 By users, do you mean DAG author or just DAG user? I'm not sure we should grant DAG user XCom permission. For the DAG author, it again depends on the team's policy. |
@Lee-W with user I mean DAG author. Credential not expose to user in this case but it's a system configuration then I think by default should not expose to anyone except Admin and Op role, with honor with flag |
As stated in apache-airflow-providers-common-io - Object Storage XCom Backend, we'll need to set
xcom_objectstorage_path
to something likes3://conn_id@mybucket/key
to use customized xcom backend. As the string representation ofObjectStoragePath
does not containconn_id
, this serialization will lost this information which cause this comparison to fail.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.