Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

actions: segregate workflows based on their purpose #6167

Closed
gustavovalverde opened this issue Feb 15, 2023 · 1 comment · Fixed by #7616
Closed

actions: segregate workflows based on their purpose #6167

gustavovalverde opened this issue Feb 15, 2023 · 1 comment · Fixed by #7616
Assignees
Labels
A-devops Area: Pipelines, CI/CD and Dockerfiles A-infrastructure Area: Infrastructure changes C-design Category: Software design work C-enhancement Category: This is an improvement I-usability Zebra is hard to understand or use S-needs-triage Status: A bug report needs triage

Comments

@gustavovalverde
Copy link
Member

gustavovalverde commented Feb 15, 2023

Motivation

Some workflows have different requirements like resources, time and dependencies, and even have jobs with different objectives (in some cases, not related to other workflow jobs).

Specifications

If multiple workflows require the same setup, this should be split into a dependable workflow which sets all the information of the following workflows. And if this workflows are not dependable on each other or have different objectives, those should also live in their own workflow files.

Sometimes we require to restart a specific workflow, which also triggers a whole dependency tree which makes re-running time consuming and almost as similar as depending on all previous workflows if there's a false/positive.

Complex Code or Requirements

  • We can't make a dependable workflow depend on another dependable workflow
  • Reduce workflow nesting as much as possible
  • Be able to cancel specific time consuming jobs without affecting other jobs which ran successfully
  • Resource intensive workflows/jobs should be split apart if other workflows don't have this resource heavy requirements
@gustavovalverde gustavovalverde added C-design Category: Software design work A-infrastructure Area: Infrastructure changes A-devops Area: Pipelines, CI/CD and Dockerfiles C-enhancement Category: This is an improvement S-needs-triage Status: A bug report needs triage P-Medium ⚡ I-usability Zebra is hard to understand or use labels Feb 15, 2023
@gustavovalverde gustavovalverde self-assigned this Feb 15, 2023
@teor2345
Copy link
Contributor

  • Be able to cancel specific time consuming jobs without affecting other jobs which ran successfully

  • Resource intensive workflows/jobs should be split apart if other workflows don't have this resource heavy requirements

Yep, this is the main thing that's annoying for us as developers.

Another thing that might be helpful is disabling failing jobs until we can do a proper fix, which we can only do in the Actions web interface on a workflow level. Jobs with similar requirements that test similar areas usually fail together.

gustavovalverde added a commit that referenced this issue Sep 25, 2023
This also renames the workflows to make their naming more consistent and adding a naming convention

Fixes: #6166
Fixes: #6167
@mergify mergify bot closed this as completed in #7616 Oct 18, 2023
mergify bot pushed a commit that referenced this issue Oct 18, 2023
* ref(workflows): consolidate workflows based on their purpose

This also renames the workflows to make their naming more consistent and adding a naming convention

Fixes: #6166
Fixes: #6167

* fix(workflows): use correct name for patch

* fix(workflow): docker unit tests

* fix(release): validation error

Error:
```
Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}
```

Fixes: release-drafter/release-drafter#1125

* fix(workflows): reference correct name

* fix: remove extra workflow

* fix(workflows): use larger runners

* fix(workflow): remove code already in docker unit-test

* fix(unit-tests): start zebra the right way

* fix: typo in patch name

* chore: move job to logical order

* imp(workflows): use better name for gcp tests

* add: missing merge changes

* chore: use better name for find-disks

* fix(ci): use the `entrypoint.sh` to change the Network

* fix(ci): add missing `ZEBRA_CONF_PATH` variable

* fix(ci): allow to build the entrypoint file with testnet

* fix(entrypoint): allow to create a dir and file with a single variable

* refactor: test config file in CI and CD with a reusable workflow

* fix(ci): wrong name used

* fix(ci): use checkout

* fix(ci): improve docker config tests

* fix(ci): use better name for protection rules

* Fix changed workflow file name in docs patch file

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Arya <aryasolhi@gmail.com>

* fix(cd): depend on file tests

* fix(docs): adapt to new workflow name

* fix: revert test coverage on CD

* chore: reduce diff

* fix(ci): allow using variable images for reusable workflows

* fix(dockerfile): use variables or default for config path and file

* fix(entrypoint): if `$ZEBRA_CONF_PATH` is set, do not override it

* Fix patch job names and remove failure job testnet dependencies

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Arya <aryasolhi@gmail.com>
mergify bot pushed a commit that referenced this issue Oct 19, 2023
* ref(workflows): consolidate workflows based on their purpose

This also renames the workflows to make their naming more consistent and adding a naming convention

Fixes: #6166
Fixes: #6167

* fix(workflows): use correct name for patch

* fix(workflow): docker unit tests

* fix(release): validation error

Error:
```
Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}
```

Fixes: release-drafter/release-drafter#1125

* fix(workflows): reference correct name

* fix: remove extra workflow

* fix(workflows): use larger runners

* fix(workflow): remove code already in docker unit-test

* fix(unit-tests): start zebra the right way

* fix: typo in patch name

* chore: move job to logical order

* imp(workflows): use better name for gcp tests

* ref(workflow): use a single job to run GCP tests

* fix(ci): do not format existing cached states if required

* test: wait for the instance to be fully ready

* fix(ci): use correct logic before formating

* fix: use correct condition

* fix: get more information

* fix(ci): use better shell handling and upgrade OS

* fix(ci): use better approach

* fix: `$DISK_OPTION` is not being correctly passed

* fix typo

* fix: more typos

* fix: use busybox

* fix: mount Docker volume at run and not before

* fix: use correct condition and simpler while

* add: missing merge changes

* chore: use better name for find-disks

* fix(ci): use the `entrypoint.sh` to change the Network

* fix(ci): add missing `ZEBRA_CONF_PATH` variable

* fix(ci): considerate new updates to jobs

* fix(ci): allow to build the entrypoint file with testnet

* fix(entrypoint): allow to create a dir and file with a single variable

* fix(ci): add missing jobs to `failure-issue`

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* fix(ci): use better comment

* refactor: test config file in CI and CD with a reusable workflow

* fix(ci): wrong name used

* fix(ci): use checkout

* fix(ci): improve docker config tests

* fix(ci): use better name for protection rules

* chore: missing merge conflicts

---------

Co-authored-by: teor <teor@riseup.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-devops Area: Pipelines, CI/CD and Dockerfiles A-infrastructure Area: Infrastructure changes C-design Category: Software design work C-enhancement Category: This is an improvement I-usability Zebra is hard to understand or use S-needs-triage Status: A bug report needs triage
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants