Trying to run an argo workflow from a selected step

[Rachel-Rabinowitz]

I have this pipeline configuration:
@dsl.pipeline(name='hello-workflow', description='Learning argo workflow') def pipeline(message1: str = 'message1', message2: str = 'message2', message3: str = 'message3'): step1 = HelloWorflowStep1(message=message1) step2 = HelloWorflowStep2(message=message2) step3 = HelloWorflowStep3(message=message3).after(step2)

I'm trying to run the following command:
argo submit --from workflowtemplate/hello-workflow --entrypoint=hello-workflow-step2 --kubeconfig {KUBE_CONFIG_PATH} --namespace {namespace}
hello-workflow-step2 is the name of step2,
Only step2 running but I expect step3 to run also

Any idea what that happens or any idea how to run a workflow from middle step

[terrytangyuan]

You step 3 might not be connected with step 2. Could you paste the raw workflow YAML generated by KFP?

[Rachel-Rabinowitz]

metadata:
name: hello-workflow
namespace: production
uid: d8464cdc-86e9-4331-8820-c2e69beed6cc
resourceVersion: '37300395'
generation: 1
creationTimestamp: '2022-07-16T22:43:43Z'
labels:
app.kubernetes.io/version: 1.0.0
annotations:
pipelines.kubeflow.org/pipeline_spec: >-
{"description": "Learning argo workflow", "inputs": [{"default":
"message1", "name": "message1", "optional": true, "type": "String"},
{"default": "message2", "name": "message2", "optional": true, "type":
"String"}, {"default": "message3", "name": "message3", "optional": true,
"type": "String"}], "name": "hello-workflow"}
managedFields:
- manager: argo
operation: Update
apiVersion: argoproj.io/v1alpha1
time: '2022-07-16T22:43:43Z'
fieldsType: FieldsV1
fieldsV1:
'f:metadata':
'f:annotations':
.: {}
'f:pipelines.kubeflow.org/pipeline_spec': {}
'f:labels':
.: {}
'f:app.kubernetes.io/version': {}
'f:spec': {}
spec:
templates:
- name: hello-workflow
inputs:
parameters:
- name: message1
- name: message2
- name: message3
outputs: {}
metadata: {}
dag:
tasks:
- name: hello-workflow-step1
template: hello-workflow-step1
arguments:
parameters:
- name: message1
value: '{{inputs.parameters.message1}}'
- name: hello-workflow-step2
template: hello-workflow-step2
arguments:
parameters:
- name: message2
value: '{{inputs.parameters.message2}}'
- name: hello-workflow-step3
template: hello-workflow-step3
arguments:
parameters:
- name: message3
value: '{{inputs.parameters.message3}}'
dependencies:
- hello-workflow-step2
- name: hello-workflow-step1
inputs:
parameters:
- name: message1
outputs: {}
nodeSelector:
nodeType: preemptible-pool-2cpu-7gb
metadata: {}
container:
name: ''
image: docker/whalesay
command:
- cowsay
args:
- '--message={{inputs.parameters.message1}}'
envFrom:
- secretRef:
name: taranis-argo-dashboard-secrets
env:
- name: GOOGLE_APPLICATION_CREDENTIALS
value: /secret/gcp-credentials/user-gcp-sa.json
- name: CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE
value: /secret/gcp-credentials/user-gcp-sa.json
- name: profile
value: kubeflow
resources:
requests:
cpu: 700m
memory: 4Gi
volumeMounts:
- name: gcp-credentials-user-gcp-sa
mountPath: /secret/gcp-credentials
imagePullPolicy: IfNotPresent
volumes:
- name: gcp-credentials-user-gcp-sa
secret:
secretName: user-gcp-sa
retryStrategy:
limit: 3
retryPolicy: Always
backoff:
duration: '30'
factor: 2
maxDuration: '86400'
- name: hello-workflow-step2
inputs:
parameters:
- name: message2
outputs: {}
nodeSelector:
nodeType: preemptible-pool-2cpu-7gb
metadata: {}
container:
name: ''
image: docker/whalesay
command:
- cowsay
args:
- '--message={{inputs.parameters.message2}}'
envFrom:
- secretRef:
name: taranis-argo-dashboard-secrets
env:
- name: GOOGLE_APPLICATION_CREDENTIALS
value: /secret/gcp-credentials/user-gcp-sa.json
- name: CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE
value: /secret/gcp-credentials/user-gcp-sa.json
- name: profile
value: kubeflow
resources:
requests:
cpu: 700m
memory: 4Gi
volumeMounts:
- name: gcp-credentials-user-gcp-sa
mountPath: /secret/gcp-credentials
imagePullPolicy: IfNotPresent
volumes:
- name: gcp-credentials-user-gcp-sa
secret:
secretName: user-gcp-sa
retryStrategy:
limit: 3
retryPolicy: Always
backoff:
duration: '30'
factor: 2
maxDuration: '86400'
- name: hello-workflow-step3
inputs:
parameters:
- name: message3
outputs: {}
nodeSelector:
nodeType: preemptible-pool-2cpu-7gb
metadata: {}
container:
name: ''
image: docker/whalesay
command:
- cowsay
args:
- '--message={{inputs.parameters.message3}}'
envFrom:
- secretRef:
name: taranis-argo-dashboard-secrets
env:
- name: GOOGLE_APPLICATION_CREDENTIALS
value: /secret/gcp-credentials/user-gcp-sa.json
- name: CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE
value: /secret/gcp-credentials/user-gcp-sa.json
- name: profile
value: kubeflow
resources:
requests:
cpu: 700m
memory: 4Gi
volumeMounts:
- name: gcp-credentials-user-gcp-sa
mountPath: /secret/gcp-credentials
imagePullPolicy: IfNotPresent
volumes:
- name: gcp-credentials-user-gcp-sa
secret:
secretName: user-gcp-sa
retryStrategy:
limit: 3
retryPolicy: Always
backoff:
duration: '30'
factor: 2
maxDuration: '86400'
entrypoint: hello-workflow
arguments:
parameters:
- name: message1
value: message1
- name: message2
value: message2
- name: message3
value: message3
ttlStrategy:
secondsAfterCompletion: 86400
secondsAfterSuccess: 86400
secondsAfterFailure: 604800
activeDeadlineSeconds: 86400
podGC:
strategy: OnPodCompletion

[terrytangyuan]

I think this is expected behavior. The issue is that your step 2 is an individual step that belongs to a steps group hello-workflow. If your entrypoint is hello-workflow or if you change your step 2 to a steps group or a DAG then the following step 3 will run.

[Rachel-Rabinowitz]

How do I cause only step2 and step3 to run if entrypoint is hello-workflow?
or how to change step 2 to a steps group or a DAG according to dependencies inside hello-workflow?
thanks

[Rachel-Rabinowitz]

Okay
thank you