Prepare for git-annex's removal of direct mode #3631
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
test_annex_version_handling patches our information on git annex so that its version matches the minimum requirement, its version is outdated, or annex is unavailable entirely. The first scenario will interfere with the logic in AnnexRepo.__init__() that decides whether the git-annex version supports direct mode, so we need to skip the test if we're running a git-annex version that doesn't support direct mode and datalad.repo.direct is true. Split this part off so that we can more easily skip it without skipping the other scenarios.
Several of the tests patch the git-annex version. One of these, test_annex_version_comparison, leaks state because it doesn't return AnnexRepo.git_annex_version back to its original value. Add a helper that takes care of this, and use it in all the spots that temporarily override the git-annex version.
Using a class method here is a bit ugly. The main motivation of doing this---instead of adding a property or regular method to AnnexRepo or adding a top-level helper to annexrepo.py---is to allow the method to be called without instantiating the class while still sharing this information with instantiated objects.
When the caller instructs us to use direct mode and git-annex doesn't support it, issue a warning and ignore the request.
git-annex dropped direct mode after version 7.20190819. Skip tests that rely on it being supported. Closes datalad#3627.
|
Looks good to me! Anything else left todo to take it out of Draft mode? I will add it to upcoming 0.11.7 milestone so datalad could be ready to "accept" new git annex in *debians |
I'll take it out of draft mode now, but I think we still need a Travis build with an unreleased annex. |
|
I was building a fresh snapshot, will upload later to NeuroDebian devel, we could test this PR against it. I will push a temp commit for that later today |
Codecov Report
@@ Coverage Diff @@
## 0.11.x #3631 +/- ##
===========================================
+ Coverage 57.66% 81.02% +23.36%
===========================================
Files 94 256 +162
Lines 15083 33882 +18799
===========================================
+ Hits 8697 27454 +18757
- Misses 6386 6428 +42
Continue to review full report at Codecov.
|
Codecov Report
@@ Coverage Diff @@
## 0.11.x #3631 +/- ##
===========================================
+ Coverage 57.66% 77.07% +19.41%
===========================================
Files 94 256 +162
Lines 15083 33882 +18799
===========================================
+ Hits 8697 26114 +17417
- Misses 6386 7768 +1382
Continue to review full report at Codecov.
|
Thank you. All green: https://travis-ci.org/datalad/datalad/builds/579444778 I'll drop the tip commit and merge. |
kyleam
force-pushed
the
directless-annex
branch
from
542b677
to
e2f918b
Compare
kyleam
added a commit
that referenced
this pull request
Sep 2, 2019
yarikoptic
added a commit
that referenced
this pull request
Sep 6, 2019
0.11.7 (Sep 02, 2019) -- python2-we-still-love-you-but-...
Primarily bugfixes with some optimizations and refactorings.
Fixes
- [addurls][]
- now provides better handling when the URL file isn't in the
expected format. ([#3579][])
- always considered a relative file for the URL file argument as
relative to the current working directory, which goes against the
convention used by other commands of taking relative paths as
relative to the dataset argument. ([#3582][])
- [run-procedure][]
- hard coded "python" when formatting the command for non-executable
procedures ending with ".py". `sys.executable` is now used.
([#3624][])
- failed if arguments needed more complicated quoting than simply
surrounding the value with double quotes. This has been resolved
for systems that support `shlex.quote`, but note that on Windows
values are left unquoted. ([#3626][])
- [siblings][] now displays an informative error message if a local
path is given to `--url` but `--name` isn't specified. ([#3555][])
- [sshrun][], the command DataLad uses for `GIT_SSH_COMMAND`, didn't
support all the parameters that Git expects it to. ([#3616][])
- Fixed a number of Unicode py2-compatibility issues. ([#3597][])
Enhancements and new features
- The [annotate-paths][] helper now caches subdatasets it has seen to
avoid unnecessary calls. ([#3570][])
- A repeated configuration query has been dropped from the handling of
`--proc-pre` and `--proc-post`. ([#3576][])
- Calls to `git annex find` now use `--in=.` instead of the alias
`--in=here` to take advantage of an optimization that git-annex (as
of the current release, 7.20190730) applies only to the
former. ([#3574][])
- [addurls][] now suggests close matches when the URL or file format
contains an unknown field. ([#3594][])
- Shared logic used in the setup.py files of Datalad and its
extensions has been moved to modules in the _datalad_build_support/
directory. ([#3600][])
- Get ready for upcoming git-annex dropping support for direct mode
([#3631][])
* tag '0.11.7': (87 commits)
DOC: Added an entry to changelogn on merged 3631
ENH: finalizing changelog for 0.11.7
TST: Update tests for a git-annex without direct mode
TST: utils: Add decorator that skips when direct mode is unsupported
ENH: annexrepo: Refuse to initialize in direct mode if unsupported
ENH: annexrepo: Add check_direct_mode_support method
BF+TST: Avoid leaking patched git-annex version
TST+RF: test_annexrepo: Split up a test
CHANGELOG.md: Second batch for 0.11.7
TST: run_procedure: Mark test_spaces() as known Windows failure
TST: run_procedure: Mark test_quoting as known windows failure
TST: run_procedure: Test more arguments that need quoting
BF(py2): run_procedure: Avoid encoding error in log message
TST: add run_procedure test with spaces in file name
TST/RF: non-hardcoded Python executable
RF: newline at end of file
RF: helper instead of conditional
RF: remove superfluous imports
BF/TST: remove quoting
ENH: replace conditionals with helper function
...
yarikoptic
added a commit
that referenced
this pull request
Sep 6, 2019
0.11.7 (Sep 02, 2019) -- python2-we-still-love-you-but-...
Primarily bugfixes with some optimizations and refactorings.
Fixes
- [addurls][]
- now provides better handling when the URL file isn't in the
expected format. ([#3579][])
- always considered a relative file for the URL file argument as
relative to the current working directory, which goes against the
convention used by other commands of taking relative paths as
relative to the dataset argument. ([#3582][])
- [run-procedure][]
- hard coded "python" when formatting the command for non-executable
procedures ending with ".py". `sys.executable` is now used.
([#3624][])
- failed if arguments needed more complicated quoting than simply
surrounding the value with double quotes. This has been resolved
for systems that support `shlex.quote`, but note that on Windows
values are left unquoted. ([#3626][])
- [siblings][] now displays an informative error message if a local
path is given to `--url` but `--name` isn't specified. ([#3555][])
- [sshrun][], the command DataLad uses for `GIT_SSH_COMMAND`, didn't
support all the parameters that Git expects it to. ([#3616][])
- Fixed a number of Unicode py2-compatibility issues. ([#3597][])
- [download-url][] now will create leading directories of the output path
if they do not exist ([#3646][])
Enhancements and new features
- The [annotate-paths][] helper now caches subdatasets it has seen to
avoid unnecessary calls. ([#3570][])
- A repeated configuration query has been dropped from the handling of
`--proc-pre` and `--proc-post`. ([#3576][])
- Calls to `git annex find` now use `--in=.` instead of the alias
`--in=here` to take advantage of an optimization that git-annex (as
of the current release, 7.20190730) applies only to the
former. ([#3574][])
- [addurls][] now suggests close matches when the URL or file format
contains an unknown field. ([#3594][])
- Shared logic used in the setup.py files of Datalad and its
extensions has been moved to modules in the _datalad_build_support/
directory. ([#3600][])
- Get ready for upcoming git-annex dropping support for direct mode
([#3631][])
* tag '0.11.7':
Changelog entry for download-url paths handling
ENH: downloaders: Ensure directories for target exist
kyleam
added a commit
to kyleam/datalad
that referenced
this pull request
Sep 6, 2019
This is a partial cherry pick of 66cf13b (ENH: annexrepo: Add check_direct_mode_support method, 2019-08-29). When the 0.11.x series (dataladgh-3631) that contained it was merged in a19e5c8, I dropped the method because it had no callers. But it turns out that I must've botched my local testing with the unreleased annex and part of test_direct_cfg does needs to be skipped. 0.11.x calls check_direct_mode_support() in __init__() to ensure that the supports_direct_mode attribute is set to a non-None value, but don't bother doing that here given that the only place in master that needs to check direct mode support is one test.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This series updates annexrepo.py and the test suite for git-annex's removal of direct mode (gh-3627). It resolves issues that I've caught from running the test suite locally with a git-annex built from d6e1f09ed, but we'd of course want a run on Travis before merging this.