BF: Don't crash on no reported percentage #2891
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bpoldrack
force-pushed
the
bf-addurl-progress
branch
from
fed5bd0
to
8a300d3
Compare
Codecov Report
@@ Coverage Diff @@
## master #2891 +/- ##
==========================================
- Coverage 90.28% 86.23% -4.06%
==========================================
Files 246 246
Lines 31932 31932
==========================================
- Hits 28831 27535 -1296
- Misses 3101 4397 +1296
Continue to review full report at Codecov.
|
yarikoptic
reviewed
Oct 3, 2018
datalad/support/annexrepo.py
Outdated
| try: | ||
| portion = (float(perc) / 100.) | ||
| except ValueError: | ||
| return None |
There was a problem hiding this comment.
ATM it should be ok to introduce None as possible output here (at the use point it is or-ed with 0 after all), but I guess it would have been safer to just return 0 (although None is more semantically correct).
I am just afraid that if above code is changed under assumption that this one returns an int, we could be in trouble without realizing it (since there is no dedicated "integration" test for this use case).
So - I am ok with it as is, but ideally - should we just return 0 here (or just portion = None and leave return to logic below)
There was a problem hiding this comment.
(although None is more semantically correct)
That's my point, yes. But I'm fine with returning zero as well, if you consider this appropriate. And I agree to do so via portion=None. Will do.
yarikoptic
added a commit
that referenced
this pull request
Oct 24, 2018
Upgrade of [git-annex] to the most recent available to your release is advisable since a number of issues were resolved at that level. ### Major refactoring and deprecations - `datalad.consts.LOCAL_CENTRAL_PATH` constant was deprecated in favor of `datalad.locations.default-dataset` [configuration] variable ([#2835]) ### Minor refactoring - `"notneeded"` messages are no longer reported by default results renderer - [run] no longer shows commit instructions upon command failure when `explicit` is true and no outputs are specified ([#2922]) - `get_git_dir` moved into GitRepo ([#2886]) - `_gitpy_custom_call` removed from GitRepo ([#2894]) - `GitRepo.get_merge_base` argument is now called `commitishes` instead of `treeishes` ([#2903]) ### Fixes - [update] should not leave the dataset in non-clean state ([#2858]) and some other enhancements ([#2859]) - Fixed chunking of the long command lines to account for decorators and other arguments ([#2864]) - Progress bar should not crash the process on some missing progress information ([#2891]) - Default value for `jobs` set to be `"auto"` (not `None`) to take advantage of possible parallel get if in `-g` mode ([#2861]) - [wtf] must not crash if `git-annex` is not installed etc ([#2865]), ([#2865]), ([#2918]), ([#2917]) - Fixed paths (with spaces etc) handling while reporting annex error output ([#2892]), ([#2893]) - `__del__` should not access `.repo` but `._repo` to avoid attempts for reinstantiation etc ([#2901]) - Fix up submodule `.git` right in `GitRepo.add_submodule` to avoid added submodules being non git-annex friendly ([#2909]), ([#2904]) - [run-procedure] ([#2905]) - now will provide dataset into the procedure if called within dataset - will not crash if procedure is an executable without `.py` or `.sh` suffixes - Use centralized `.gitattributes` handling while setting annex backend ([#2912]) - `GlobbedPaths.expand(..., full=True)` incorrectly returned relative paths when called more than once ([#2921]) ### Enhancements and new features - Report progress on [clone] when installing from "smart" git servers ([#2876]) - Stale/unused `sth_like_file_has_content` was removed ([#2860]) - Enhancements to [search] to operate on "improved" metadata layouts ([#2878]) - Output of `git annex init` operation is now logged ([#2881]) - New - `GitRepo.cherry_pick` ([#2900]) - `GitRepo.format_commit` ([#2902]) - [run-procedure] ([#2905]) - procedures can now recursively be discovered in subdatasets as well. The uppermost has highest priority - Procedures in user and system locations now take precedence over those in datasets. * tag '0.11.0': Make it a 0.11.0 release since there were some API RFings REL: slight tune up to Changelog following the advices DOC: v0.10.4: Mention change in procedure precedence (a0cbcba) DOC: v0.10.4: Fix description of db715b7 DOC: v0.10.4: Improve description of 6f615a4 DOC: v0.10.4: Remove duplicate word
yarikoptic
added a commit
that referenced
this pull request
Oct 24, 2018
## 0.11.0 (Oct 23, 2018) -- Soon-to-be-perfect [git-annex] 6.20180913 (or later) is now required - provides a number of fixes for v6 mode operations etc. ### Major refactoring and deprecations - `datalad.consts.LOCAL_CENTRAL_PATH` constant was deprecated in favor of `datalad.locations.default-dataset` [configuration] variable ([#2835]) ### Minor refactoring - `"notneeded"` messages are no longer reported by default results renderer - [run] no longer shows commit instructions upon command failure when `explicit` is true and no outputs are specified ([#2922]) - `get_git_dir` moved into GitRepo ([#2886]) - `_gitpy_custom_call` removed from GitRepo ([#2894]) - `GitRepo.get_merge_base` argument is now called `commitishes` instead of `treeishes` ([#2903]) ### Fixes - [update] should not leave the dataset in non-clean state ([#2858]) and some other enhancements ([#2859]) - Fixed chunking of the long command lines to account for decorators and other arguments ([#2864]) - Progress bar should not crash the process on some missing progress information ([#2891]) - Default value for `jobs` set to be `"auto"` (not `None`) to take advantage of possible parallel get if in `-g` mode ([#2861]) - [wtf] must not crash if `git-annex` is not installed etc ([#2865]), ([#2865]), ([#2918]), ([#2917]) - Fixed paths (with spaces etc) handling while reporting annex error output ([#2892]), ([#2893]) - `__del__` should not access `.repo` but `._repo` to avoid attempts for reinstantiation etc ([#2901]) - Fix up submodule `.git` right in `GitRepo.add_submodule` to avoid added submodules being non git-annex friendly ([#2909]), ([#2904]) - [run-procedure] ([#2905]) - now will provide dataset into the procedure if called within dataset - will not crash if procedure is an executable without `.py` or `.sh` suffixes - Use centralized `.gitattributes` handling while setting annex backend ([#2912]) - `GlobbedPaths.expand(..., full=True)` incorrectly returned relative paths when called more than once ([#2921]) ### Enhancements and new features - Report progress on [clone] when installing from "smart" git servers ([#2876]) - Stale/unused `sth_like_file_has_content` was removed ([#2860]) - Enhancements to [search] to operate on "improved" metadata layouts ([#2878]) - Output of `git annex init` operation is now logged ([#2881]) - New - `GitRepo.cherry_pick` ([#2900]) - `GitRepo.format_commit` ([#2902]) - [run-procedure] ([#2905]) - procedures can now recursively be discovered in subdatasets as well. The uppermost has highest priority - Procedures in user and system locations now take precedence over those in datasets. * tag '0.11.0': CHANGELOG adjusted to reflect new minimal version of git-annex
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request fixes a failure discovered in https://github.com/datalad/datalad-container. Test was failing in https://travis-ci.org/datalad/datalad-container/jobs/436153807, because
annex-addurldidn't report progress in the JSON report. ThereforeProcessAnnexProgressIndicatorsset the reported percentage to default to an empty string, which was then casted to afloat, yielding aValueError.Independently on why exactly
addurldoesn't report it (seems to just take a while for the first report to contain a field "percent-progress"), WE are setting the default to an empty string and therefore have to be able to deal with that value in the function we pass it to.This pull request proposes to just catch the
ValueErrorand returnNoneinAnnexRepo._get_size_from_perc_complete().Changes
Please have a look @datalad/developers