pull: output is not human-readable within Bamboo (tqdm)

[sebastienbonami]

Bug Report

Description

I'm running dvc pull within Bamboo (a tool by Atlassian similar to Jenkins) and the output looks like this:

...
build	18-Mar-2021 15:25:18	Downloading:   0% 0.00/294k [00:00<?, ?file/s]
build	18-Mar-2021 15:25:19	Downloading:   0% 0.00/294k [00:00<?, ?file/s{'info': ''}]
build	18-Mar-2021 15:25:19	
build	18-Mar-2021 15:25:19	  0% 0.00/30.9k [00:00<?, ?B/s]�[A
build	18-Mar-2021 15:25:19	
build	18-Mar-2021 15:25:19	  0% 0.00/30.9k [00:00<?, ?B/s{'info': ''}]�[A
build	18-Mar-2021 15:25:19	
build	18-Mar-2021 15:25:19	
build	18-Mar-2021 15:25:20	  0% 0.00/31.3k [00:00<?, ?B/s{'info': ''}]�[A�[A
build	18-Mar-2021 15:25:20	
build	18-Mar-2021 15:25:20	
build	18-Mar-2021 15:25:20	
build	18-Mar-2021 15:25:20	  0% 0.00/34.8k [00:00<?, ?B/s{'info': ''}]�[A�[A�[A
build	18-Mar-2021 15:25:20	
...

Any idea on how to improve that? Maybe there's a way to deactivate tqdm or something?

Reproduce

1. Create a build plan in Bamboo linked to a Git repo that use DVC
2. Add a script task that run dvc pull
3. Run a build
4. Look at its Logs

Expected

A human can read the Logs and understand the download progress of the data.

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 2.0.6 (pip)
---------------------------------
Platform: Python 3.9.2 on Linux-5.4.0-1028-aws-x86_64-with-glibc2.28
Supports: http, https, s3
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p1
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/nvme0n1p1
Repo: dvc, git

Thanks for the help!

[pared]

@sebastienbonami Thank you for reporting the issue!
I am afraid, thath in case of pull(push too) we did not provide a way to disable tqdm.

This seems like a feature request - it seems to me that in some cases (like CI/CD) progress bar might not be that important. More value might come from logs, and progress can be just occluding important information.

[sebastienbonami]

For now I'm using --quiet on the pull command as without it the output is a mess in Bamboo (CI/CD). I consider this as a Bug because I think it's essential to have a readable output in CI/CD, including a progress bar ideally.

I have other projects using tqdm and their output is fine in CI/CD so I'm not sure what's wrong here.

Thanks.

[pmrowla]

0% 0.00/30.9k [00:00<?, ?B/s{'info': ''}]�[A

This makes it look potentially like some kind of terminal encoding issue to me?

@sebastienbonami it's going to be difficult for us to debug this since it appears to be specific to Bamboo, could you provide some example logs for your other projects which use tqdm and work as-expected?

[sebastienbonami]

@pmrowla about the encoding, I see UTF-8 for "System encoding" on Bamboo's System information page. I'm guessing Bamboo is using that to show build logs.

Here's an example of build logs from another project using tqdm that are showing correctly in Bamboo:

...
Progress:   0%|          | 2998/654613 [00:03<08:38, 1257.83 element/s]
Progress:   0%|          | 3127/654613 [00:03<08:34, 1265.84 element/s]
Progress:   0%|          | 3254/654613 [00:03<08:34, 1265.84 element/s]
Progress:   1%|          | 3381/654613 [00:03<08:37, 1258.06 element/s]
Progress:   1%|          | 3507/654613 [00:03<08:38, 1254.80 element/s]
Progress:   1%|          | 3633/654613 [00:03<08:40, 1250.64 element/s]
...

[daavoo]

A Discord user reported similar behavior but on GitLab C.I:

https://discord.com/channels/485586884165107732/485596304961962003/915639761530650655 .

[skshetry]

Related #2664.

[skshetry]

Closing in favor of #2664.