Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions docs/docs/using-semaphore/jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -1089,6 +1089,8 @@ It's not possible to use job parallelism at the same time as [job matrices](#mat

## Job matrix {#matrix}

<VideoTutorial title="How to use a Job Matrix" src="https://www.youtube.com/embed/jRpj2Pu5eak" />

A job matrix is a more advanced form of [job parallelism](#job-parallelism) where you can define multiple variables with different values and run all the possible permutations.

For example, let's say we want to test our application using three Node.js versions using npm and yarn
Expand Down
138 changes: 133 additions & 5 deletions docs/docs/using-semaphore/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,30 +27,159 @@ Before you can run a workflow in Semaphore you need:
- A Semaphore project linked to the repository
- One or more pipelines

The [project page](./projects#view-projects) shows all the recent workflows for the project.
The [project page](./projects#view-projects) shows all the recent workflows for the project.

![Types of workflows](./img/workflow-types.jpg)

## Visual workflow editor {#workflow-editor}

<VideoTutorial title="How to use the workflow builder" src="https://www.youtube.com/embed/dg2jDQmYJ_4" />

You can define most aspects of your workflows using the visual editor.
You can define most aspects of your workflows using the visual editor.

To access the editor, open one of your projects on and press **Edit Workflow**. All your changes are stored as YAML pipeline definitions on your Git repository. You can make changes using the visual editor, or edit the YAML directly by clicking on the pipeline YAML file.

![Workflow editor button](./img/workflow-editor.jpg)

See the [jobs page](./jobs) to learn how to define jobs and blocks.

## Modeling Complex Workflows {#modeling-complex-workflows}

This section provides guides to model complex, non-linear CI/CD processes.

### Fan-out Fan-In {#fan-out-fan-in}

<VideoTutorial title="Fan Out - Fan In" src="https://www.youtube.com/embed/HKv-jMkC7T0" />

The Fan-Out Fan-In workflow provides consistency and maximum parallelization. It can be split into 3 stages:

1. **Build stage**: you build your project once
2. **Fan-Out stage**: all your tests fan out from the build stage and run in parallel
3. **Fan-In stage**: once tested, the workflow fans in to a release or deploy stage

![Fan-Out Fan-In](./img/fan-out-fan-in.jpg)

<Tabs groupId="editor-yaml">
<TabItem value="editor" label="Editor" default>

<Steps>

1. Create your build [job](./jobs). Depending on the nature of your project, you can save the built artifact to the [artifact store](./artifacts) or push it to a [Docker registry](./containers/docker)

![Build stage](./img/fan-out-stage1.jpg)

2. Add your test jobs. Set dependencies so all your tests depend on the build job created on Step 1

![Fan-out stage](./img/fan-out-stage2.jpg)

3. Add the release/deploy job. Use dependencies so the new job depends on all your tests. This will make the release/job run only if all tests have passed

![Fan-in stage](./img/fan-out-stage3.jpg)

</Steps>

</TabItem>
<TabItem value="yaml" label="YAML">

You can create a Fan-Out Fan-In workflow by setting the dependencies in your blocks. The Fan-Out stage is achieved by defining `dependencies`. For example:

```yaml
blocks:
- name: "Build"
dependencies: []
...

- name: "Unit tests"
dependencies: ["Build"]
...

- name: "Integration tests"
dependencies: ["Build"]
...

- name: "E2E tests"
dependencies: ["Build"]
...

- name: "Release candidate"
dependencies: ["Integration tests", "Unit tests", "E2E tests"]
...
```

Find below a complex example of a Fan-Out, Fan-In for a Node-based workflow:

```yaml
version: v1.0
name: Continuous Integration Pipeline
agent:
machine:
type: f1-standard-2
os_image: ubuntu2204
blocks:
- name: Build Stage
dependencies: []
task:
jobs:
- name: Build Project
commands:
- 'checkout'
- 'npm run build'
- 'artifact push workflow build/'
- name: End to end tests
dependencies:
- Build Stage
task:
jobs:
- name: Run E2E tests
commands:
- 'checkout'
- 'artifact pull workflow build/'
- 'npm run test:e2e'
- name: Integration Tests
dependencies:
- Build Stage
task:
jobs:
- name: Run integration tests
commands:
- 'checkout'
- 'artifact pull workflow build/'
- 'npm run test:integration'
- name: Unit tests
dependencies:
- Build Stage
task:
jobs:
- name: Run unit tests
commands:
- 'checkout'
- 'artifact pull workflow build/'
- 'npm run test:unit'
- name: Release candidate
dependencies:
- End to end tests
- Integration Tests
- Unit tests
task:
jobs:
- name: Generate RC
commands:
- 'checkout'
- 'artifact pull workflow build/'
- 'npm run package'
```

</TabItem>
</Tabs>

## Workflow triggers

The following events or actions trigger workflows by default:

- Pushing commits into any branch
- Pushing Git tags
- Changing any pipelines
- Manually re-running workflows
- Manually re-running workflows
- Running pipelines using [Tasks](./tasks)

Additionally, you can configure workflows to be triggered by:
Expand Down Expand Up @@ -117,12 +246,11 @@ Approving forked pull requests is limited to new comments only and does not work

## How to skip commits {#skip}

If you don't want to start a workflow, type one of the following strings in the commit message.
If you don't want to start a workflow, type one of the following strings in the commit message.

- `[ci skip]`
- `[skip ci]`


For example, this push does not trigger a Semaphore pipeline execution, it is completely ignored:

```shell title="Skipping a commit"
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 2 additions & 6 deletions docs/versioned_docs/version-CE/using-semaphore/jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,6 @@ description: Jobs and blocks are the basic unit of work

# Jobs







Jobs get stuff done. This page explains to create and configure jobs.

## Job lifecycle {#job-lifecycle}
Expand Down Expand Up @@ -1025,6 +1019,8 @@ It's not possible to use job parallelism at the same time as [job matrices](#mat

## Job matrix {#matrix}

<VideoTutorial title="How to use a Job Matrix" src="https://www.youtube.com/embed/jRpj2Pu5eak" />

A job matrix is a more advanced form of [job parallelism](#job-parallelism) where you can define multiple variables with different values and run all the possible permutations.

For example, let's say we want to test our application using three Node.js versions using npm and yarn
Expand Down
129 changes: 129 additions & 0 deletions docs/versioned_docs/version-CE/using-semaphore/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,135 @@ To access the editor, open one of your projects on and press **Edit Workflow**.

See the [jobs page](./jobs) to learn how to define jobs and blocks.

## Modeling Complex Workflows {#modeling-complex-workflows}

This section provides guides to model complex, non-linear CI/CD processes.

### Fan-out Fan-In {#fan-out-fan-in}

<VideoTutorial title="Fan Out - Fan In" src="https://www.youtube.com/embed/HKv-jMkC7T0" />

The Fan-Out Fan-In workflow provides consistency and maximum parallelization. It can be split into 3 stages:

1. **Build stage**: you build your project once
2. **Fan-Out stage**: all your tests fan out from the build stage and run in parallel
3. **Fan-In stage**: once tested, the workflow fans in to a release or deploy stage

![Fan-Out Fan-In](./img/fan-out-fan-in.jpg)

<Tabs groupId="editor-yaml">
<TabItem value="editor" label="Editor" default>

<Steps>

1. Create your build [job](./jobs). Depending on the nature of your project, you can save the built artifact to the [artifact store](./artifacts) or push it to a [Docker registry](./containers/docker)

![Build stage](./img/fan-out-stage1.jpg)

2. Add your test jobs. Set dependencies so all your tests depend on the build job created on Step 1

![Fan-out stage](./img/fan-out-stage2.jpg)

3. Add the release/deploy job. Use dependencies so the new job depends on all your tests. This will make the release/job run only if all tests have passed

![Fan-in stage](./img/fan-out-stage3.jpg)

</Steps>

</TabItem>
<TabItem value="yaml" label="YAML">

You can create a Fan-Out Fan-In workflow by setting the dependencies in your blocks. The Fan-Out stage is achieved by defining `dependencies`. For example:

```yaml
blocks:
- name: "Build"
dependencies: []
...

- name: "Unit tests"
dependencies: ["Build"]
...

- name: "Integration tests"
dependencies: ["Build"]
...

- name: "E2E tests"
dependencies: ["Build"]
...

- name: "Release candidate"
dependencies: ["Integration tests", "Unit tests", "E2E tests"]
...
```

Find below a complex example of a Fan-Out, Fan-In for a Node-based workflow:

```yaml
version: v1.0
name: Continuous Integration Pipeline
agent:
machine:
type: f1-standard-2
os_image: ubuntu2204
blocks:
- name: Build Stage
dependencies: []
task:
jobs:
- name: Build Project
commands:
- 'checkout'
- 'npm run build'
- 'artifact push workflow build/'
- name: End to end tests
dependencies:
- Build Stage
task:
jobs:
- name: Run E2E tests
commands:
- 'checkout'
- 'artifact pull workflow build/'
- 'npm run test:e2e'
- name: Integration Tests
dependencies:
- Build Stage
task:
jobs:
- name: Run integration tests
commands:
- 'checkout'
- 'artifact pull workflow build/'
- 'npm run test:integration'
- name: Unit tests
dependencies:
- Build Stage
task:
jobs:
- name: Run unit tests
commands:
- 'checkout'
- 'artifact pull workflow build/'
- 'npm run test:unit'
- name: Release candidate
dependencies:
- End to end tests
- Integration Tests
- Unit tests
task:
jobs:
- name: Generate RC
commands:
- 'checkout'
- 'artifact pull workflow build/'
- 'npm run package'
```

</TabItem>
</Tabs>

## Workflow triggers

The following events or actions trigger workflows by default:
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 2 additions & 6 deletions docs/versioned_docs/version-EE/using-semaphore/jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,6 @@ description: Jobs and blocks are the basic unit of work

# Jobs







Jobs get stuff done. This page explains to create and configure jobs.

## Job lifecycle {#job-lifecycle}
Expand Down Expand Up @@ -1025,6 +1019,8 @@ It's not possible to use job parallelism at the same time as [job matrices](#mat

## Job matrix {#matrix}

<VideoTutorial title="How to use a Job Matrix" src="https://www.youtube.com/embed/jRpj2Pu5eak" />

A job matrix is a more advanced form of [job parallelism](#job-parallelism) where you can define multiple variables with different values and run all the possible permutations.

For example, let's say we want to test our application using three Node.js versions using npm and yarn
Expand Down
Loading