Unzip progress bars #3545
Unzip progress bars #3545
Conversation
ghost
assigned 
|
This is the less invasive way to introduce |
|
BTW, we have to investigate the 17.6MB vs 16.8M file sizes 😕 |
|
I'm thinking about forwarding all unknown attributes/methods to the underlying |
|
Mb difference might be if using SI or not. |
|
Given the output here, I'm sure we have to disable progress bars on CI systems, too much noise (use |
| from tqdm import tqdm | ||
| from contextlib import contextmanager | ||
|
|
||
| TIMEOUT_BEAT_SECONDS = 30 |
There was a problem hiding this comment.
Do we want to make this value configurable? It is used to print a heartbeat character instead of the progress bar. Which value to use?
There was a problem hiding this comment.
If that is for the "dot" being printed every few seconds, both for the user and CI not aborting, I would say hardcode it by now, to something that makes sense from a UI perspective (every 10 seconds?)
There was a problem hiding this comment.
This is the "dot" being printed only if not output.is_terminal, so tipically it will be shown in CI and tests.
|
Using the
|
|
There is a bug (py2) in this version of tqdm, output shows the following:; Exception KeyError: KeyError(<weakref at 0x7f5cd8944fc8; to 'tqdm' at 0x7f5cd8a1c650>,) in <bound method tqdm.__del__ of Decompressing conan_export.tgz: 18.0B [00:00, 5.84kB/s]> ignoredIssue reported here: tqdm/tqdm#549 (PR ongoing: tqdm/tqdm#607) Solution: may downgrade to a working version; or I think that catching the exception, closing the pb and re-raising it could work (in the |
|
@jgsogo, I am a developer of tqdm, let's wait for my PR to be merged. Currently, if you use py2 with tqdm, tqdm v4.20 should be fine (or v4.19) Catching this exception (in tqdm>=4.21) might be tricky. |
conans/client/remote_manager.py
Outdated
| @@ -29,6 +29,7 @@ | |||
| from conans.util.env_reader import get_env | |||
| from conans.search.search import filter_packages | |||
| from conans.client.cmd.uploader import UPLOAD_POLICY_SKIP | |||
| from conans.util import progress_bar as pb | |||
There was a problem hiding this comment.
I think I'd prefer the full progress_bar, explicit name.
conans/util/progress_bar.py
Outdated
|
|
||
| def __init__(self, fileobj, output, desc=None): | ||
| pb_kwargs = self.tqdm_defaults.copy() | ||
| self.ori_output = output |
conans/util/progress_bar.py
Outdated
| TIMEOUT_BEAT_CHARACTER = '.' | ||
|
|
||
|
|
||
| class BinaryFileWrapper(object): |
There was a problem hiding this comment.
A class name more descriptive of its purpose, like ProgressBarFileReader
|
About the py2 issue, @chengs' PR is pending yet. Maybe we should downgrade the tqdm version in order to test this progress bar in the next release. |
| self._pb = tqdm(total=self.tell(), desc=desc, file=output, **pb_kwargs) | ||
| self.seek(0) | ||
|
|
||
| def seek(self, *args, **kwargs): |
There was a problem hiding this comment.
should it implement other fileojb methods, e.g. fileno?
There was a problem hiding this comment.
Hi! This is just a proof of concept (to see if someone reports a weird behavior in CI or interactive). The implementation will be far more complete (following this idea: #3608), but probably it will remain only for binary files (files opened in binary mode) as this is the type of content be have to deal with in Conan.
* POC for progress bars using tqdm * remove unused imports * add seek function * support only binary read, add a contextmanager to better control flow * remove unused argument * new line in non-terminal progress bar * global var for the timeout char * remove invalid file * flush has no params * more meaninful names to classes and variables * downgrade tqdm to 4.20 to avoid error message
FileObjectProgressI've tried to introduce it in the less invasive way possible.
Example output doing a
conan install boost/1.68.0@conan/stable:Changelog: Feature: Progress bars for files unzipping.