New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Azure DataStore] Handle storage options as secrets #1206
[Azure DataStore] Handle storage options as secrets #1206
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hayesgb Looks good to me, just note you have some conflict
Fixed merge conflict. Thanks! |
@hayesgb I now noticed that the test file you added is almost an exact duplicate of |
@Hedingber -- I considered combining the tests, but was concerned about the interpretability. The two approaches serve entirely different use-cases, and I was worried that if the tests were blended (storing secrets or retrieving environmental variables vs user-passed credentials, which will most likely be done during exploratory analysis) it would lead to confusion and create potential maintainability issues. Thoughts? |
@hayesgb I see your point but I feel like leaving the tests duplicated will hurt maintainability more 😬 , @theSaarco WDYT ? |
@Hedingber -- No worries. I consolidated the tests into a single file and updated the PR. LMKWYT. |
@hayesgb Looks great! |
This add the ability to pass standard dictionary keys from fsspec's storage_options parameter into mlrun.run.get_dataitem() as secrets.
Enabling this will more easily allow users engaging in exploratory analysis to leverage the mlrun api to fetch data_items from Azure by enabling the following