Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use of 'az' as a shorter version of 'abfs' #78

Closed
raybellwaves opened this issue Aug 7, 2020 · 7 comments
Closed

use of 'az' as a shorter version of 'abfs' #78

raybellwaves opened this issue Aug 7, 2020 · 7 comments

Comments

@raybellwaves
Copy link
Contributor

raybellwaves commented Aug 7, 2020

Moved from filesystem_spec to here

This is more of a discussion than a feature request.

Looking at known_implementations in fsspec there are gcs and gs which both point to gcsfs.GCSFileSystem. Could az be created as a shorter version of abfs?

az is common with Azure CLI syntax e.g. docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10#run-azcopy.
As someone who is very lazy I prefer two key strokes instead of four.

Caveats

Unlike gs and s3, Azure has two classes (adlfs.AzureDatalakeFileSystem and adlfs.AzureBlobFileSystem). I imagine you can't use az for both classes.

Proposed solution

Use az as a shortcut for 'abfs': {'class': 'adlfs.AzureBlobFileSystem'}. Given this is for the latest version of storage (Azure Datalake Gen2 and Azure Blob Storage). In addition my 50 % reduction in key strokes motivation doesn't apply to adl (Azure Datalake Gen1).

@hayesgb
Copy link
Collaborator

hayesgb commented Aug 7, 2020

I'm open to the idea, but I'd like to get feedback from @martindurant. When we settled on abfs as the protocol, it was based on the fact that abfs:// is the defined protocol for AzureBlobFileSystem, as described here.

@hayesgb
Copy link
Collaborator

hayesgb commented Aug 7, 2020

I just saw that this discussion started in fsspec. I'm OK with it. As a side-note, I've also started to see the "az://" show up in at least one other place as well.

@martindurant
Copy link
Member

You can, of course, have both protocols.

@raybellwaves
Copy link
Contributor Author

Yeah I was thinking of making it akin to gcs and gs. I imagine replacing abfs with az would probably lead too many breaking changes for users.

I'm thinking of seeing the following in fsspec.registry.known_implementations:

'abfs': {'class': 'adlfs.AzureBlobFileSystem', 'err': 'Install adlfs to access Azure Datalake Gen2 and Azure Blob Storage'},
'az': {'class': 'adlfs.AzureBlobFileSystem', 'err': 'Install adlfs to access Azure Datalake Gen2 and Azure Blob Storage'}

@martindurant
Copy link
Member

OK, so go ahead!
I think the protocol attribute of the implementation class in this repo should also change to a tuple of the possible strings, ("az", "abfs") at the same time

@hayesgb
Copy link
Collaborator

hayesgb commented Aug 15, 2020

@raybellwaves -- I'm good with this. Are you planning to submit a PR?

@raybellwaves raybellwaves mentioned this issue Aug 22, 2020
3 tasks
@raybellwaves
Copy link
Contributor Author

Closed via #111

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants