-
Notifications
You must be signed in to change notification settings - Fork 119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add git to filesystem source 301 #954
Draft
deanja
wants to merge
8
commits into
dlt-hub:devel
Choose a base branch
from
deanja:add-git-to-filesystem-source-301
base: devel
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Commits on Feb 15, 2024
-
Add gitpythonfs fsspec implementation
- Intended to be used in filesystem verified Source. -read-only - due to depedency of gitpython on git binaries, gitpythonfs should only be conditionally imported, for example by registering the module using `register_implementation_in_fsspec()`
Configuration menu - View commit details
-
Copy full SHA for b04d563 - Browse repository at this point
Copy the full SHA b04d563View commit details -
Configuration menu - View commit details
-
Copy full SHA for 43c6180 - Browse repository at this point
Copy the full SHA 43c6180View commit details -
Urls do not require a netloc (in rfc1808)
- See https://www.rfc-editor.org/rfc/rfc1808.html#section-2.1 . So, for example, gitpythonfs:// is valid and works in fsspec. - However in bucket-based systems, the bucket name is where the netloc would be - eg s3://bucket_name. But luckily s3fs (and probably az, gcs etc) already gives helpful error messages if no bucket name given, wrong bucket name given etc. - As a ToDo, this rule could test only those protocols whose fsspec implementation needs netloc, rather than excluding a few protocols as is done here. But we don't know which protocols this rule was initially added for.
Configuration menu - View commit details
-
Copy full SHA for c16a9dc - Browse repository at this point
Copy the full SHA c16a9dcView commit details -
Use 'mtime' as default input for
modification_date
-reduces need to modify code every time a new fsspec implementation is added - `mtime` is idiomatic in *nix file systems.
Configuration menu - View commit details
-
Copy full SHA for ea0557f - Browse repository at this point
Copy the full SHA ea0557fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 87ec256 - Browse repository at this point
Copy the full SHA 87ec256View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5d644fc - Browse repository at this point
Copy the full SHA 5d644fcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 62790c1 - Browse repository at this point
Copy the full SHA 62790c1View commit details
Commits on Feb 16, 2024
-
use distinct filesystem instances in pipeline threads
- clearer naming in FileItemDict
Configuration menu - View commit details
-
Copy full SHA for b473fb5 - Browse repository at this point
Copy the full SHA b473fb5View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.