-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Enhancement] Proposal for introducing centralized URI handling #793
Comments
Meta comments
Having said all of that, we should indeed have more consistency. We need centralised
All of these should live in fsspec, I think, and most of the functionality already exists in some form there. I am -1 on trying to enumerate all possible protocols and credential types here, or having protocol-dependant sets of |
Thank you for the correction, sometimes our typing tools fool us :) I agree that dedicated classes with their methods like in My main idea is that we could simplify the code, by introducing an abstraction on top of the strings and using these objects as a base type for internal communication between the different parts. |
Whilst intake can afford to be relaxed about requirements and add extra packages if warranted, we'd have to be clear about what we're getting from them. As I said, I think that path string handling should ideally live over in fsspec, including the functions I outlined above, and that would solve everything for those readers that accept fsspec/file-like objects or filesystems. Other readers likely don't need any path handling at all, and Intake just passes the strings along unaltered. |
I'm not familiar with the structure of the project, but I have the impression that there are too many places where we do URI string parsing, protocol extraction, checking URI format, converting paths to POSIX format, and so on...
It seems tricky to maintain, and it could cause some weird bugs. I think there should be the only place where we do all the operations with URI strings.
For example, it could be done like that, in this very draft implementation of the
URI
class:Probably,
intake/utils.py
could be the right place for it...What do you think if we create this missing abstraction layer where we do all the manipulations, and then we just pass the
URI
class instances everywhere?I think this should simplify the code, ease maintenance, and promote code reuse.
The text was updated successfully, but these errors were encountered: