ENH(UX): set "smoothing=0" for any progress bar with "total" in its label #5438
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…abel Original odd experience was reported in datalad#5421 (comment) and it was due to interplay of two factors: 1 - Total is updated only after git-annex verifies the checksum. Total progress bar is updated only upon the final "ok" from git-annex that we got the file. That also involves validation of the checksum by git-annex. So, even though file is finished downloading, and its progress bar disappeared, Total would not be updated until that file is "ok"ed by annex. That takes time, so perceptually to the user (me) it was odd that Total shows only some little total although a good number of files already "downloaded" Good news: since Joey implemented "on the fly" checksumming (at least for built in "special remotes") -- with e.g. `8.20210223+git1-g33af7d193-1~ndall+1` experience is much better since there is no delay from progressbar disappearing and the final message arriving and thus Total gets updated. 2 - Default tqdm setting is not good for "rare" updates See `smoothing` at https://github.com/tqdm/tqdm . Default `0.3` is a compromise between providing user with smth between "momentary" and "mean" speed. When Total is updated only once a minute, that leads to "current" speed being indeed quite low if most of the time there were no progress at all ;) Since "smoothing" is ATM quit specific to tqdm and we have quite a number of subclasses from ProgressBarBase I decided to just go for an ad-hoc solution here and make any progress bar which has "total" in its label to use smoothing=0, i.e. to report the mean. Altogether with a new git-annex and this total smoothing of 0, Total looks quite decent now ;) Closes datalad#5434
Codecov Report
@@ Coverage Diff @@
## maint #5438 +/- ##
==========================================
- Coverage 89.56% 89.56% -0.01%
==========================================
Files 296 296
Lines 41756 41758 +2
==========================================
+ Hits 37398 37399 +1
- Misses 4358 4359 +1
Continue to review full report at Codecov.
|
kyleam
approved these changes
Feb 25, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Original odd experience was reported in #5421 (comment) and it was
due to interplay of two factors:
1 - Total is updated only after git-annex verifies the checksum.
Total progress bar is updated only upon the final "ok" from git-annex
that we got the file. That also involves validation of the checksum by
git-annex. So, even though file is finished downloading, and its progress bar
disappeared, Total would not be updated until that file is "ok"ed by annex.
That takes time, so perceptually to the user (me) it was odd that Total shows
only some little total although a good number of files already "downloaded"
Good news: since Joey implemented "on the fly" checksumming (at least for
built in "special remotes") -- with e.g.
8.20210223+git1-g33af7d193-1~ndall+1(*)experience is much better since there is no delay from progressbar disappearing
and the final message arriving and thus Total gets updated.
2 - Default tqdm setting is not good for "rare" updates
See
smoothingat https://github.com/tqdm/tqdm . Default0.3is acompromise between providing user with smth between "momentary" and
"mean" speed. When Total is updated only once a minute, that leads to
"current" speed being indeed quite low if most of the time there were no
progress at all ;)
Since "smoothing" is ATM quit specific to tqdm and we have quite a number of
subclasses from ProgressBarBase I decided to just go for an ad-hoc solution
here and make any progress bar which has "total" in its label to use
smoothing=0, i.e. to report the mean.
Altogether with a new git-annex and this total smoothing of 0, Total looks
quite decent now ;)
Closes #5434
*: reminder - installation of a bleeding edge build of git-annex on debians is as difficult as
datalad-installer git-annex -m datalad/git-annexnow (you would need hub.oauthtoken though in your git config)