ENH: GitRepo -- make use of --pathspec-from-file where possible and chunking needed#6932
Merged
Merged
Conversation
yarikoptic
added
severity-minor
yarikoptic
force-pushed
the
bf-long-CLI
branch
from
August 11, 2022 18:50
f80a141 to
ffbfa33
Compare
yarikoptic
added a commit
to yarikoptic/datalad
that referenced
this pull request
Aug 11, 2022
https://github.com/datalad/datalad/runs/7793747019?check_suite_focus=true and others in datalad#6932 are now failing all of a sudden! Smells the like newest and greatest setuptools added some deprecation
Member
Author
|
ok, the issue is reported pypa/setuptools#3501 ... so seems to relate to versionner and workarounds seems to be suggested such as https://github.com/shapely/shapely/pull/1475/files from pypa/setuptools#3501 (comment) . I guess I will submit a dedicated PR with that one. |
yarikoptic
added a commit
to yarikoptic/datalad
that referenced
this pull request
Aug 12, 2022
https://github.com/datalad/datalad/runs/7793747019?check_suite_focus=true and others in datalad#6932 are now failing all of a sudden! Smells the like newest and greatest setuptools added some deprecation one more after presumably addresed in setuptools https://github.com/pypa/setuptools/releases/tag/v64.0.2 pypa/setuptools#3506
yarikoptic
added a commit
to yarikoptic/datalad
that referenced
this pull request
Aug 12, 2022
…present A little more discussion at datalad#6930 and initially identified while working on datalad#6932 and being unable to get CommandError.stdout for matching while using call_git (but works with _call_git using different code path, which does not have this git_ignore_check). One of the two locations where this function is used has stdout=None, so this lead to wiping out of e.stdout even if it was present. I think it was not intentional but @christian-monch would know better
yarikoptic
force-pushed
the
bf-long-CLI
branch
from
August 12, 2022 17:59
ffbfa33 to
cc859ae
Compare
Codecov Report
@@ Coverage Diff @@
## master #6932 +/- ##
===========================================
+ Coverage 78.29% 90.95% +12.65%
===========================================
Files 354 354
Lines 46259 46276 +17
===========================================
+ Hits 36219 42090 +5871
+ Misses 10040 4186 -5854
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
…hunking needed Chunking is a kludge to workaround the problem of DataLad needing to provide a long list of files to git/git-annex commands. Whenever git-annex has --batch mode which we use typically for commands which could get lots of file paths arguments, we have not used any special means like that for git commands. Many commands support --pathspec-file-nul and --pathspec-from-file=FILE options to pass a list of pathspec(s) to operate on via a FILE instead of command line. In this PR we decide to pass via such a file in cases where more than a single chunk is necessary.
To guarantee that we do test operation of datalad whenever paths are to be provided to git commands (we think) already support --pathspec-from-file option(s). I added that run to existing runs of Travis which already check for nonmiltiplexed ssh
… the logs Now it logs with all the variables not expanded etc, so hard to replicate invocation locally
yarikoptic
force-pushed
the
bf-long-CLI
branch
from
August 16, 2022 13:41
6b367a1 to
a2b00af
Compare
|
Code Climate has analyzed commit a2b00af and detected 2 issues on this pull request. Here's the issue category breakdown:
View more on Code Climate. |
mih
approved these changes
Aug 20, 2022
mih
left a comment
mih
left a comment
Member
There was a problem hiding this comment.
This makes sense and is a verz useful feature. Thx!
Collaborator
|
PR released in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Chunking is a kludge to workaround the problem of DataLad needing to provide a
long list of files to git/git-annex commands. Whenever git-annex has --batch mode
which we use typically for commands which could get lots of file paths arguments, we
have not used any special means like that for git commands. Many commands support
--pathspec-file-nul and --pathspec-from-file=FILE options to pass a list of pathspec(s)
to operate on via a FILE instead of command line.
In this PR we decide to pass via such a file in cases where more than a single
chunk is necessary.
The main change is to
GitRepo.commitwhich used to resort to sequence of commits with--amendto establish the desired commit.maint, might not matter)pathspec_from_file, I will establishdatalad.runtime.prefer_pathspec_from_file: boolwhich would cause use of--pathspec-from-fileregardless on either chunking is needed. Then one of the matrix runs of travis will enable that config setting to get commands covered (probably the one for "bleeding edge everything")Closes #6922