Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: az context to access data in Azure blob storage #937

Open
raybellwaves opened this issue Aug 11, 2020 · 3 comments
Open
Projects

Comments

@raybellwaves
Copy link

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
bc.az('blazingsql-colab', blob_name='blazingsql-colab')

Note: could also be named bc.abfs() but we are discussing using az for the fsspec here fsspec/adlfs#78

Describe alternatives you've considered
Not tested: some work around of reading the data in as a dask.dataframe using aldfs -> cudf -> BlazingContext

Additional context
For better or worse we are a Microsoft shop at the company I work for. I would like to use Blazing SQL to potentially speed up a spark ETL

----For BlazingSQL Developers----
How and where should this be implemented?
What part of the code should be feature be implemented? What should the APIs and/or classes look like?

**Other design considerations **
What components of the engine could be affected by this? What functions should we make sure we use/reuse?

Testing considerations?
What sort of unit tests and/or End to End tests be implemented to test this?

@raybellwaves raybellwaves added the ? - Needs Triage needs team to review and classify label Aug 11, 2020
@roaramburu roaramburu added this to Needs prioritizing in Scrum board Aug 19, 2020
@wmalpica wmalpica removed the ? - Needs Triage needs team to review and classify label Aug 19, 2020
@wmalpica
Copy link
Contributor

Hello @raybellwaves , thank you for filing this issue. This is the future we actually already had in our roadmap and we plan on implementing it in our next release.

@wmalpica wmalpica moved this from Needs prioritizing to Next Release in Scrum board Aug 19, 2020
@raybellwaves
Copy link
Author

Awesome. Sounds good. az just got implemented in fsspec https://github.com/intake/filesystem_spec/blob/master/fsspec/registry.py#L141
When the new version of fsspec is released we'll work on fsspec/adlfs#91

@raybellwaves
Copy link
Author

FYI az just landed in the latest adlfs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Scrum board
  
Low priority Backlog
Development

No branches or pull requests

2 participants