Skip to content

Dynamic mutexes #14363

@Gitopolis

Description

@Gitopolis

Summary

Dynamically generate list of mutexes from an expression, something like:

spec:
  synchronization:
    mutexesFrom:
    - expression: "{{ workflow.inputs.modified | toJson }}"

Use Cases

A workflow processing modified files for every commit to some git repository. The input from the webhook might look something like this:

commits:
- modified:
  - file_a
  - file_b
- modified:
  - file_c
- modified:
  - file_d
- modified:
  - file_a

I want to run workflows/steps in parallel for processing files file_c and file_d, but processing file_a must not interfere with each other. So it would make sense to have mutexes equal to modified file names:

  • workflow1
    synchronization:
      mutexes:
      - name:  <repo-name>/file_a
      - name:  <repo-name>/file_b
  • workflow2
    synchronization:
      mutexes:
      - name:  <repo-name>/file_c
  • workflow3
    synchronization:
      mutexes:
      - name:  <repo-name>/file_d
  • workflow4
    synchronization:
      mutexes:
      - name:  <repo-name>/file_a

Currently there is no way to generate list of mutexes on runtime, so I use just <repo-name>, and that makes the processing unreasonably slow


Message from the maintainers:

Love this feature request? Give it a 👍. We prioritise the proposals with the most 👍.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions