Deleting a workflow on the workflow details page doesn't do anything #11661

tico24 · 2023-08-23T15:40:13Z

Pre-requisites

I have double-checked my configuration
I can confirm the issues exists when I tested with :latest
I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

If you go to the workflows details page and hit delete (and then confirm the delete), you are taken back to the workflows list page... where your workflow is still sat there staring at you :)

We do not have workflows archiving enabled.

Version

v3.5.0-rc1

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

any workflow

Logs from the workflow controller

I couldn't find anything useful. I'll update if I find anything.

Logs from in your workflow's wait container

n/a

The text was updated successfully, but these errors were encountered:

terrytangyuan · 2023-08-23T15:54:12Z

cc @toyamagu-2021 This might be related to your fix

terrytangyuan · 2023-08-23T16:01:56Z

If you refresh, would it be gone?

toyamagu-2021 · 2023-08-24T14:25:37Z

I cannot reproduce this both on master branch and on 3.5.0-rc1 on k3d cluster... 🤔
（I don't refresh page）
Might be moving to workflows list page before workflow is completely deleted?

argo-workflows/ui/src/app/workflows/components/workflow-details/workflow-details.tsx

Line 203 in 7cb22a2

window.location.reload();

tico24 · 2023-08-24T14:28:18Z

Still very repeatable. Stays after refresh

My gif tool cut off the deletion popup but obviously I said 'ok' to it :)

terrytangyuan · 2023-08-24T14:28:48Z

Check developer console and server logs?

tico24 · 2023-08-24T14:31:55Z

some console shouting. means nothing to me

Server logs show nothing special, just that the list page is loading.

toyamagu-2021 · 2023-08-24T14:51:45Z

Could you give me your workflow metadata.labels?
Might be following line returns false.

argo-workflows/ui/src/app/workflows/components/workflow-details/workflow-details.tsx

Lines 194 to 195 in 7cb22a2

    
           if (isWorkflowInCluster(workflow)) { 
        
               services.workflows.delete(workflow.metadata.name, workflow.metadata.namespace).catch(setError);

Or please give me your workflow (might be meaningless...).

tico24 · 2023-08-24T14:53:27Z

metadata:
  name: renovate-application-definitions-h7k7x
  generateName: renovate-application-definitions-
  namespace: ci
  uid: 7d878237-241a-46d6-804d-fdda0d4a5d89
  resourceVersion: '139954745'
  generation: 4
  creationTimestamp: '2023-08-24T14:41:50Z'
  labels:
    cron: 'true'
    workflows.argoproj.io/completed: 'true'
    workflows.argoproj.io/creator: system-serviceaccount-argo-argo-server
    workflows.argoproj.io/cron-workflow: renovate-application-definitions
    workflows.argoproj.io/phase: Succeeded
  annotations:
    workflows.argoproj.io/pod-name-format: v2
    workflows.argoproj.io/scheduled-time: '2023-08-24T14:41:50Z'
  ownerReferences:
    - apiVersion: argoproj.io/v1alpha1
      kind: CronWorkflow
      name: renovate-application-definitions
      uid: 0d2bb088-d382-4684-997e-13b23263e7b2
      controller: true
      blockOwnerDeletion: true
  managedFields:
    - manager: argo
      operation: Update
      apiVersion: argoproj.io/v1alpha1
      time: '2023-08-24T14:41:50Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:workflows.argoproj.io/scheduled-time: {}
          f:generateName: {}
          f:labels:
            .: {}
            f:cron: {}
            f:workflows.argoproj.io/creator: {}
            f:workflows.argoproj.io/cron-workflow: {}
          f:ownerReferences:
            .: {}
            k:{"uid":"0d2bb088-d382-4684-997e-13b23263e7b2"}: {}
        f:spec: {}
    - manager: workflow-controller
      operation: Update
      apiVersion: argoproj.io/v1alpha1
      time: '2023-08-24T14:43:25Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            f:workflows.argoproj.io/pod-name-format: {}
          f:labels:
            f:workflows.argoproj.io/completed: {}
            f:workflows.argoproj.io/phase: {}
        f:status: {}
spec:
  arguments:
    parameters:
      - name: repository_json
        value: '"pipekit/application-definitions"'
  workflowTemplateRef:
    name: renovate
status:
  phase: Succeeded
  startedAt: '2023-08-24T14:41:50Z'
  finishedAt: '2023-08-24T14:43:25Z'
  progress: 1/1
  nodes:
    renovate-application-definitions-h7k7x:
      id: renovate-application-definitions-h7k7x
      name: renovate-application-definitions-h7k7x
      displayName: renovate-application-definitions-h7k7x
      type: DAG
      templateName: main
      templateScope: local/
      phase: Succeeded
      startedAt: '2023-08-24T14:41:50Z'
      finishedAt: '2023-08-24T14:43:25Z'
      progress: 1/1
      resourcesDuration:
        cpu: 163
        memory: 311
      children:
        - renovate-application-definitions-h7k7x-4272356993
      outboundNodes:
        - renovate-application-definitions-h7k7x-4272356993
    renovate-application-definitions-h7k7x-4272356993:
      id: renovate-application-definitions-h7k7x-4272356993
      name: renovate-application-definitions-h7k7x.renovate
      displayName: renovate
      type: Pod
      templateName: renovate
      templateScope: local/
      phase: Succeeded
      boundaryID: renovate-application-definitions-h7k7x
      startedAt: '2023-08-24T14:41:50Z'
      finishedAt: '2023-08-24T14:43:15Z'
      progress: 1/1
      resourcesDuration:
        cpu: 163
        memory: 311
      outputs:
        artifacts:
          - name: main-logs
            s3:
              key: >-
                my-artifacts/2023/08/24/renovate-application-definitions-h7k7x/renovate-application-definitions-h7k7x-renovate-4272356993/main.log
        exitCode: '0'
      hostNodeName: ip-10-102-158-69.us-east-2.compute.internal
  storedTemplates:
    namespaced/renovate/main:
      name: main
      inputs: {}
      outputs: {}
      metadata: {}
      dag:
        tasks:
          - name: renovate
            template: renovate
            arguments: {}
    namespaced/renovate/renovate:
      name: renovate
      inputs: {}
      outputs: {}
      nodeSelector:
        nodegroup: arm-spot
      metadata:
        annotations:
          vault.hashicorp.com/agent-inject: 'true'
          vault.hashicorp.com/agent-inject-secret-ci-token: infrastructure/data/renovate
          vault.hashicorp.com/agent-inject-secret-docker-token: infrastructure/data/docker/pipekitdev
          vault.hashicorp.com/agent-inject-template-ci-token: |
            {{ with secret "infrastructure/data/renovate" -}}
                export RENOVATE_TOKEN="{{ .Data.data.github_PAT }}"
            {{- end }}
          vault.hashicorp.com/agent-inject-template-docker-token: |
            {{ with secret "infrastructure/data/docker/pipekitdev" -}}
                export RENOVATE_HOST_RULES="[{\"matchHost\":\"docker.io\",\"username\":\"tico24\",\"password\":\"{{ .Data.data.PAT }}\"}]"
            {{- end }}
          vault.hashicorp.com/agent-pre-populate-only: 'true'
          vault.hashicorp.com/auth-path: auth/runner
          vault.hashicorp.com/role: argo
      container:
        name: ''
        image: renovate/renovate:36.57.2
        command:
          - /bin/bash
          - '-c'
          - |
            . /vault/secrets/ci-token
            . /vault/secrets/docker-token
            renovate
        env:
          - name: LOG_LEVEL
            value: debug
          - name: RENOVATE_PLATFORM
            value: github
          - name: RENOVATE_CONFIG
            value: '{ "repositories": [ {{workflow.parameters.repository_json}} ] }'
          - name: RENOVATE_PR_CONCURRENT_LIMIT
            value: '0'
          - name: RENOVATE_BRANCH_CONCURRENT_LIMIT
            value: '0'
          - name: RENOVATE_PR_HOURLY_LIMIT
            value: '0'
          - name: RENOVATE_X_HARD_EXIT
            value: 'true'
        resources:
          requests:
            cpu: '1'
            memory: 256Mi
        imagePullPolicy: Always
  conditions:
    - type: PodRunning
      status: 'False'
    - type: Completed
      status: 'True'
  resourcesDuration:
    cpu: 163
    memory: 311
  storedWorkflowTemplateSpec:
    templates:
      - name: main
        inputs: {}
        outputs: {}
        metadata: {}
        dag:
          tasks:
            - name: renovate
              template: renovate
              arguments: {}
      - name: renovate
        inputs: {}
        outputs: {}
        nodeSelector:
          nodegroup: arm-spot
        metadata:
          annotations:
            vault.hashicorp.com/agent-inject: 'true'
            vault.hashicorp.com/agent-inject-secret-ci-token: infrastructure/data/renovate
            vault.hashicorp.com/agent-inject-secret-docker-token: infrastructure/data/docker/pipekitdev
            vault.hashicorp.com/agent-inject-template-ci-token: |
              {{ with secret "infrastructure/data/renovate" -}}
                  export RENOVATE_TOKEN="{{ .Data.data.github_PAT }}"
              {{- end }}
            vault.hashicorp.com/agent-inject-template-docker-token: |
              {{ with secret "infrastructure/data/docker/pipekitdev" -}}
                  export RENOVATE_HOST_RULES="[{\"matchHost\":\"docker.io\",\"username\":\"tico24\",\"password\":\"{{ .Data.data.PAT }}\"}]"
              {{- end }}
            vault.hashicorp.com/agent-pre-populate-only: 'true'
            vault.hashicorp.com/auth-path: auth/runner
            vault.hashicorp.com/role: argo
        container:
          name: ''
          image: renovate/renovate:36.57.2
          command:
            - /bin/bash
            - '-c'
            - |
              . /vault/secrets/ci-token
              . /vault/secrets/docker-token
              renovate
          env:
            - name: LOG_LEVEL
              value: debug
            - name: RENOVATE_PLATFORM
              value: github
            - name: RENOVATE_CONFIG
              value: '{ "repositories": [ {{workflow.parameters.repository_json}} ] }'
            - name: RENOVATE_PR_CONCURRENT_LIMIT
              value: '0'
            - name: RENOVATE_BRANCH_CONCURRENT_LIMIT
              value: '0'
            - name: RENOVATE_PR_HOURLY_LIMIT
              value: '0'
            - name: RENOVATE_X_HARD_EXIT
              value: 'true'
          resources:
            requests:
              cpu: '1'
              memory: 256Mi
          imagePullPolicy: Always
    entrypoint: main
    arguments:
      parameters:
        - name: repository_json
          value: '"pipekit/application-definitions"'
    serviceAccountName: ci
    nodeSelector:
      nodegroup: spot
    imagePullSecrets:
      - name: regcred
    ttlStrategy:
      secondsAfterCompletion: 3600
    activeDeadlineSeconds: 3600
    workflowTemplateRef:
      name: renovate
    volumeClaimGC:
      strategy: OnWorkflowCompletion
  artifactRepositoryRef:
    default: true
    artifactRepository:
      archiveLogs: true
      s3:
        endpoint: s3.amazonaws.com
        bucket: pipekit-argo-workflows
        region: us-east-2
        insecure: false
        useSDKCreds: true
        encryptionOptions: {}
        keyFormat: >-
          my-artifacts/{{workflow.creationTimestamp.Y}}/{{workflow.creationTimestamp.m}}/{{workflow.creationTimestamp.d}}/{{workflow.name}}/{{pod.name}}
  artifactGCStatus:
    notSpecified: true

tico24 · 2023-08-24T14:54:41Z

The same issue happens with a basic hello world though:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-
  labels:
    workflows.argoproj.io/archive-strategy: "false"
  namespace: ci
  annotations:
    workflows.argoproj.io/description: |
      This is a simple hello world example.
spec:
  serviceAccountName: ci
  entrypoint: whalesay
  templates:
  - name: whalesay
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["hello world"]

terrytangyuan · 2023-08-24T16:50:47Z

Any logs from your workflow controller?

terrytangyuan · 2023-08-25T16:06:22Z

I am not able to reproduce this either. Please check controller logs to see if the workflow deletion request is handled successfully.

shmruin · 2023-08-25T17:42:30Z

Interesting. I see some small error on provided gif when entering to the workflow as below.

And refer to this specific line in useEffect, are there any chances that let archivedWf: Workflow just stay undefined and goes into setWorkflow? and catched by isWorkflowInCluster condition check ?

terrytangyuan · 2023-08-25T18:36:47Z

@shmruin Can you check if your controller/executor/server image versions match?

terrytangyuan · 2023-08-25T19:11:34Z

@shmruin That error occurs when you visit a live workflow details page for a workflow that's been deleted from the cluster. I am improving the error message in #11674.

terrytangyuan · 2023-08-25T20:12:58Z

Also sent #11676. I suspect there are some errors that appear in the UI for a moment before redirecting to the workflow list page.

shmruin · 2023-08-27T02:24:22Z

@terrytangyuan Sadly, that screenshot is @tico24 's, and I also fail to reproduce this error.
I think @agilgur5 's reviews in #11676 makes sense and can be potentially related to this issue, but cannot sure as I cannot make this situation.

terrytangyuan · 2023-08-28T15:25:45Z

@tico24 Can you try the latest image? So far looks like no one else has been able to reproduce your original issue.

tico24 · 2023-08-29T14:41:14Z

As expected, and as mentioned in the original issue, I can recreate the issue on latest too.

controller, server and executor were moved to latest. Here's a screenshot of the server to prove it

latest did not exist on this node, so was freshly pulled. So no caching here either.

terrytangyuan · 2023-08-29T16:05:07Z

@tico24 Can you check workflow controller and K8s API server logs? I'd be interested to see if the request has been sent successfully.

tico24 · 2023-08-29T16:05:31Z

Both server and controller logs say nothing special. What specifically do you want from them?

terrytangyuan · 2023-08-29T16:07:48Z

@tico24 They would indicate whether the deletion request has been received and processed. Just do a grep from the logs.

agilgur5 · 2023-08-29T16:09:57Z

The Network tab of the browser console will also show if the request was sent

tico24 · 2023-08-29T16:10:13Z

I see what I'd expect to see.

server:

time="2023-08-29T16:09:21.189Z" level=info duration=4.18495ms method=DELETE path=/api/v1/workflows/ci/renovate-application-definitions-dv565 size=39 status=408
time="2023-08-29T16:09:21.189Z" level=error msg="finished unary call with code Internal" error="rpc error: code = Internal desc = getting archived workflows not supported" grpc.code=Internal grpc.method=DeleteWorkflow grpc.service=workflow.WorkflowService grpc.start_time="2023-08-29T16:09:21Z" grpc.time_ms=4.023 s
pan.kind=server system=grpc

controller still has nothing.

tico24 · 2023-08-29T16:11:10Z

agilgur5 · 2023-08-29T16:15:32Z

status=408 in the Server logs. A 408 gives some credence to my hypothesis from #11676. Though not necessarily conclusive.

I'll work on a fix for that

terrytangyuan · 2023-08-29T16:22:46Z

Yep exactly what we needed to see. The request went through but was canceled in argo-server.

@agilgur5 Looking forward to your fix :-)

agilgur5 · 2023-08-29T23:54:11Z

So I too was unable to reproduce this locally. That being said, as I mentioned on Slack, if my hypothesis is correct, then this is a race condition. Specifically, higher latency (as with a remote server vs. local server) may make this more likely. The low latency of local dev may mean that the request completes shortly before or after the page reload and so is not affected by the connection drop.

I also did try using throttling in browser dev tools, but that did not reproduce either. That being said, I'm not entirely sure how the emulation works under-the-hood and a slow request/response is not entirely the same as network hops to a remote server. Server-side throttling (e.g. a sleep on the request) may be a bit closer to the real thing, but also a bit different. Forgot to try a server-side throttle though.

Wrote up #11711 which hopefully fixes this -- if my hypothesis is correct.

terrytangyuan · 2023-09-01T17:11:08Z

@tico24 Have you tried @agilgur5's PR yet?

tico24 added the type/bug label Aug 23, 2023

This was referenced Aug 25, 2023

fix(server): Surface underlying error when getting a workflow #11674

Merged

fix(ui): Only redirect/reload to wf list page when wf deletion succeeds #11676

Merged

agilgur5 added the type/regression Regression from previous behavior (a specific type of bug) label Aug 29, 2023

agilgur5 added the area/ui label Aug 29, 2023

agilgur5 mentioned this issue Aug 29, 2023

fix(ui): don't reload the page until _after_ deletion #11711

Merged

terrytangyuan closed this as completed in #11711 Sep 4, 2023

agilgur5 mentioned this issue Sep 22, 2023

REQUEST: Promotion to Reviewer for @agilgur5 argoproj/argoproj#242

Closed

8 tasks

This was referenced Apr 12, 2024

fix(ui): remove unnecessary hard reload after delete #12930

Merged

feat: Unified workflows list UI and API #11121

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deleting a workflow on the workflow details page doesn't do anything #11661

Deleting a workflow on the workflow details page doesn't do anything #11661

tico24 commented Aug 23, 2023

terrytangyuan commented Aug 23, 2023

terrytangyuan commented Aug 23, 2023

toyamagu-2021 commented Aug 24, 2023 •

edited

Loading

tico24 commented Aug 24, 2023 •

edited

Loading

terrytangyuan commented Aug 24, 2023

tico24 commented Aug 24, 2023

toyamagu-2021 commented Aug 24, 2023

tico24 commented Aug 24, 2023

tico24 commented Aug 24, 2023 •

edited

Loading

terrytangyuan commented Aug 24, 2023

terrytangyuan commented Aug 25, 2023

shmruin commented Aug 25, 2023 •

edited

Loading

terrytangyuan commented Aug 25, 2023

terrytangyuan commented Aug 25, 2023 •

edited

Loading

terrytangyuan commented Aug 25, 2023

shmruin commented Aug 27, 2023

terrytangyuan commented Aug 28, 2023 •

edited

Loading

tico24 commented Aug 29, 2023 •

edited

Loading

terrytangyuan commented Aug 29, 2023

tico24 commented Aug 29, 2023 •

edited

Loading

terrytangyuan commented Aug 29, 2023

agilgur5 commented Aug 29, 2023

tico24 commented Aug 29, 2023 •

edited

Loading

tico24 commented Aug 29, 2023

agilgur5 commented Aug 29, 2023 •

edited

Loading

terrytangyuan commented Aug 29, 2023 •

edited

Loading

agilgur5 commented Aug 29, 2023 •

edited

Loading

terrytangyuan commented Sep 1, 2023

Deleting a workflow on the workflow details page doesn't do anything #11661

Deleting a workflow on the workflow details page doesn't do anything #11661

Comments

tico24 commented Aug 23, 2023

Pre-requisites

What happened/what you expected to happen?

Version

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

Logs from the workflow controller

Logs from in your workflow's wait container

terrytangyuan commented Aug 23, 2023

terrytangyuan commented Aug 23, 2023

toyamagu-2021 commented Aug 24, 2023 • edited Loading

tico24 commented Aug 24, 2023 • edited Loading

terrytangyuan commented Aug 24, 2023

tico24 commented Aug 24, 2023

toyamagu-2021 commented Aug 24, 2023

tico24 commented Aug 24, 2023

tico24 commented Aug 24, 2023 • edited Loading

terrytangyuan commented Aug 24, 2023

terrytangyuan commented Aug 25, 2023

shmruin commented Aug 25, 2023 • edited Loading

terrytangyuan commented Aug 25, 2023

terrytangyuan commented Aug 25, 2023 • edited Loading

terrytangyuan commented Aug 25, 2023

shmruin commented Aug 27, 2023

terrytangyuan commented Aug 28, 2023 • edited Loading

tico24 commented Aug 29, 2023 • edited Loading

terrytangyuan commented Aug 29, 2023

tico24 commented Aug 29, 2023 • edited Loading

terrytangyuan commented Aug 29, 2023

agilgur5 commented Aug 29, 2023

tico24 commented Aug 29, 2023 • edited Loading

tico24 commented Aug 29, 2023

agilgur5 commented Aug 29, 2023 • edited Loading

terrytangyuan commented Aug 29, 2023 • edited Loading

agilgur5 commented Aug 29, 2023 • edited Loading

terrytangyuan commented Sep 1, 2023

toyamagu-2021 commented Aug 24, 2023 •

edited

Loading

tico24 commented Aug 24, 2023 •

edited

Loading

tico24 commented Aug 24, 2023 •

edited

Loading

shmruin commented Aug 25, 2023 •

edited

Loading

terrytangyuan commented Aug 25, 2023 •

edited

Loading

terrytangyuan commented Aug 28, 2023 •

edited

Loading

tico24 commented Aug 29, 2023 •

edited

Loading

tico24 commented Aug 29, 2023 •

edited

Loading

tico24 commented Aug 29, 2023 •

edited

Loading

agilgur5 commented Aug 29, 2023 •

edited

Loading

terrytangyuan commented Aug 29, 2023 •

edited

Loading

agilgur5 commented Aug 29, 2023 •

edited

Loading