Logs from tqdm python progress bar

[hadim]

Summary

What happened/what you expected to happen?

Some of my workflows are displaying progress using the Python tqdm library. I have tried to play with various settings of the library without success.

The progress bar only displays itself at the end of the workload when it's at 100%. Which make it quite useless.

What version of Argo Workflows are you running? 3.1.10

Diagnostics

Workflow to reproduce:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: tqdm-test-
spec:
  entrypoint: entrypoint
  templates:
    - name: entrypoint
      script:
        image: python:3.8
        command: [bash]
        source: |

          pip install tqdm

          read -r -d '' PYTHON_CODE << EOM
          from tqdm import tqdm
          from time import sleep

          # the progress bar will only be displayed after 40s when it's at 100%
          for i in tqdm(range(20)):
              sleep(2)
          EOM

          python -c "$PYTHON_CODE"

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[terrytangyuan]

This is a common issue with progress bars with K8s logging. I am not sure if there's anything we can do here. As a workaround for your specific issue, you should be able to log the progress bar periodically in your Python code though.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[hadim]

Indeed this is not an argo issue but more related to k8s itself with a potential workaround to be provided by tqdm.

I have opened a ticket about it on the tqdm repo: tqdm/tqdm#1319