[Feature] Disable download progress bars

[austinmw]

What is the feature?

Hi, when not using an interactive terminal (for example CloudWatch logs), each iteration of progress bars is printed to logs. So for example I typically have several hundred lines in my training logs that just print each iteration of progress bar for downloading checkpoint file(s).

Is there already any way to disable progress bars for file downloads? If not, I'd like propose a feature for disabling this through a config option.

Any other context?

For example (this goes on for several hundreds of lines):

[1,mpirank:8,algo-2]:#15 0%| | 0.00/97.8M [00:00<?, ?B/s]
[1,mpirank:15,algo-2]:#15 0%| | 0.00/97.8M [00:00<?, ?B/s]
[1,mpirank:14,algo-2]:#15 0%| | 0.00/97.8M [00:00<?, ?B/s][1,mpirank:11,algo-2]:#15 0%| | 0.00/97.8M [00:00<?, ?B/s]
[1,mpirank:12,algo-2]:#15 0%| | 0.00/97.8M [00:00<?, ?B/s]
[1,mpirank:10,algo-2]:#15 0%| | 0.00/97.8M [00:00<?, ?B/s]
[1,mpirank:13,algo-2]:#15 0%| | 0.00/97.8M [00:00<?, ?B/s]
[1,mpirank:9,algo-2]:#15 0%| | 0.00/97.8M [00:00<?, ?B/s]
[1,mpirank:15,algo-2]:#15 5%|▍ | 4.70M/97.8M [00:00<00:02, 48.7MB/s]
[1,mpirank:14,algo-2]:#15 5%|▍ | 4.69M/97.8M [00:00<00:01, 49.1MB/s]
[1,mpirank:11,algo-2]:#15 11%|█ | 10.5M/97.8M [00:00<00:00, 110MB/s]
[1,mpirank:10,algo-2]:#15 8%|▊ | 7.95M/97.8M [00:00<00:01, 83.3MB/s]
[1,mpirank:12,algo-2]:#15 8%|▊ | 7.44M/97.8M [00:00<00:01, 77.1MB/s]
[1,mpirank:8,algo-2]:#15 11%|█▏ | 11.1M/97.8M [00:00<00:00, 106MB/s]
[1,mpirank:13,algo-2]:#15 8%|▊ | 7.66M/97.8M [00:00<00:01, 79.5MB/s]
[1,mpirank:9,algo-2]:#15 2%|▏ | 2.18M/97.8M [00:00<00:05, 19.4MB/s]
[1,mpirank:10,algo-2]:#15 17%|█▋ | 17.1M/97.8M [00:00<00:00, 90.5MB/s]
[1,mpirank:15,algo-2]:#15 14%|█▍ | 13.6M/97.8M [00:00<00:01, 72.6MB/s]
[1,mpirank:14,algo-2]:#15 12%|█▏ | 11.8M/97.8M [00:00<00:01, 61.7MB/s]
[1,mpirank:8,algo-2]:#15 22%|██▏ | 21.2M/97.8M [00:00<00:00, 106MB/s]

[zhouzaida]

Hi @austinmw , thanks for your feedback. The progress bar is caused by load_from_http loading checkpoints from the Internet. We will find a proper way to support closing the progress bar.

[HAOCHENYE]

@austinmw Hi! it seems the dirty progress bar is caused by

mmengine/mmengine/utils/dl_utils/hub.py

Line 53 in d3d7528

def load_url(url,

load_url accepts the progress argument with the default value True, which causes the problem. Maybe we could set its default value according to whether it is in an interactive terminal or not?

[austinmw]

Hi, thanks, that sounds like a great solution if possible.

Not sure of the best way to do that, but this seems to correctly produce False in my environment:

cmd = 'if [[ $- == *i* ]]; then echo "True"; else echo "False"; fi'
response = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
interactive = response.stdout.read().decode('utf-8').strip()
if interactive == 'True':
    progress = True
else:
    progress = False

[HAOCHENYE]

Will this solution https://stackoverflow.com/questions/6108330/checking-for-interactive-shell-in-a-python-script be more elegant? I'm not sure if it will work for your environment.

[austinmw]

Ah yes that solution works in my environment (os.isatty(0) returns False) and is much cleaner!

[HAOCHENYE]

Ah yes that solution works in my environment (os.isatty(0) returns False) and is much cleaner!

Great 😎😎 !, Would you mind proposing a PR to fix this this?

[austinmw]

@HAOCHENYE Hey, let me know how #770 looks. It works to remove the progress bars in my test environment.