How to get list of docker images used in one argo workflow pipeline?

[larrycai]

Use case

In CI flow, we want to baseline the docker images we used for each workflow pipeline, so we know what is really executed for it.

What I expect

In the end of step like OnExit, I can get all docker images in list

for step in all_steps:
     print(step.name, step.image)
checkout alpine:latest
unittest/1 python:3.9
unittest/2 python:3.10
summary ubuntu:focal

What I checked/tried

• https://argoproj.github.io/argo-workflows/variables/
• also try to parse the yaml files directly, but it has issues for parameter (then difficult to get real docker image) and workflow template (it refers to another file)

Or possible?

• Not sure whether I can get kubectl to list all related pods -> images in easy way

[terrytangyuan]

You should be able to parse the workflow YAML and get the images from each template.

[larrycai]

like my example https://github.com/larrycai/hello-python/blob/main/.argo/workflows/ci-python.yaml

      image: "python:{{inputs.parameters.python-version}}"

How can I get correct version if I just parse the yaml file directly, are there any API in argo workflow, I can fetch the real yaml during execution.

And for template, if they are stored in argo workflowtemplate instead of this yaml file, how can I get it correctly?

[ajkaanbal]

you can use jq, something like:

kubectl get wf {{workflow.name}} -o json | jq '..|.image? | select( . != null )'

or to inspect all related pods:

kubectl get po -l workflows.argoproj.io/workflow={{workflow.name}}  -o json |  jq '..|.image? | select( . != null )'

[larrycai]

since I need to collect those information automatically during workflow execution, so for this, I need to inject .kube config into latest step, but could be a solution

[larrycai]

OK, the good for this answer is that I can get workflow.name and namespace easily inside the flow, see example code below. Thx @ajkaanbal

spec:
  entrypoint: ci-python-example
  volumeClaimTemplates:
  - metadata:
      name: workdir
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
  volumes:
  - name: kubecfg
    secret:
      # kubectl create secret generic kube-config --from-file=config=readonly-config
      secretName: readonly-kubecfg
  templates:
  templates:
  - name: ci-python-example
    steps:
    - - name: checkout
        template: checkout
    - - name: static-check
        template: flake8
    - - name: unittest
        template: pytest
        arguments:
          parameters:
          - name: python-version
            value: "{{item.tag}}"
        withItems:
        - { tag: '3.8' }
        - { tag: '3.9' }
        - { tag: '3.10' }
    - - name: summary
        template: run-hello
# some steps are skipped here
  - name: run-hello
    container:
      image: bitnami/kubectl:1.20
      command: [sh, -c]
      args: 
      - |
        kubectl --kubeconfig=/kube_config/config get po -n {{workflow.namespace}} -l workflows.argoproj.io/workflow={{workflow.name}} -o json |  jq -r '..|.image? | select( . != null )' | sort |  uniq
        echo " store those images as baseline for this pipeline"
      volumeMounts:
      - name: workdir
        mountPath: /workspace
      - name: kubecfg
        readOnly: true
        mountPath: /kube_config

The result looks like below in the logs

ci-python-example-7qbgv-2392645666: bitnami/kubectl:1.20
ci-python-example-7qbgv-2392645666: golang:1.10
ci-python-example-7qbgv-2392645666: python:3.10
ci-python-example-7qbgv-2392645666: python:3.8
ci-python-example-7qbgv-2392645666: python:3.9
ci-python-example-7qbgv-2392645666: quay.io/argoproj/argoexec:v3.2.4