push: Assume 0 bytes pushed if git-annex does not provide bytesize#7049
push: Assume 0 bytes pushed if git-annex does not provide bytesize#7049yarikoptic merged 2 commits intodatalad:maintfrom
Conversation
Otherwise we can get KeyError and underlying code can provide records without bytesize
Inspired by a fix we needed for push: datalad#7049 I looked into status to realize that we do not even consider those files for which git-annex would not report bytesize. This commit makes them considered in count but would assume 0 bytesize for "recorded" etc. It would not address the issue that we would still not consider/interrogate sizes for those files even if their content available: $> datalad status --annex=all 2 annex'd files (0.0 B/0.0 B present/total size) $> ls -lL * -r-------- 1 yoh yoh 0 Sep 20 15:52 123.dat -r-------- 1 yoh yoh 723 Sep 20 15:57 sample3.csv where sample3.csv is the relaxed one.
Codecov ReportBase: 45.45% // Head: 88.36% // Increases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## maint #7049 +/- ##
===========================================
+ Coverage 45.45% 88.36% +42.90%
===========================================
Files 354 354
Lines 58559 46336 -12223
Branches 6615 0 -6615
===========================================
+ Hits 26618 40943 +14325
+ Misses 31779 5393 -26386
+ Partials 162 0 -162
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
|
uff all travis fails seems to be due to unrelated |
|
no objections, the fix is trivial, let's proceed and retry releasing again! |
|
@jwodder -- note that here as well we didn't have github actions ran and changelog entry was pushed by github-actions bot although I thought we should have switched to that yarikoptic-gitmate token/account... will need to check/keep an eye... |
|
PR released in |
Inspired by a fix we needed for push: datalad#7049 I looked into status to realize that we do not even consider those files for which git-annex would not report bytesize. This commit makes them considered in count but would assume 0 bytesize for "recorded" etc. It would not address the issue that we would still not consider/interrogate sizes for those files even if their content available: $> datalad status --annex=all 2 annex'd files (0.0 B/0.0 B present/total size) $> ls -lL * -r-------- 1 yoh yoh 0 Sep 20 15:52 123.dat -r-------- 1 yoh yoh 723 Sep 20 15:57 sample3.csv where sample3.csv is the relaxed one.
Otherwise we can get KeyError and underlying code can provide records without bytesize. Reported in https://neurostars.org/t/datalad-push-causes-keyerror-bytesize-keyerror/23570 with
datalad==0.15.4
git==2.35.0
git-annex==8.20210903
first I failed to reproduce with the following script following user report with --fast
but then reproduced nicely with --relaxed
and verified that with this PR we do not crash. Given that git-annex unlikely to even provide progress report if not enough time passed, I think I would not bother crafting a unittest which would trigger this condition atm.