Skip to content

Commit

Permalink
Introduce objects and subjects (cdevents#35)
Browse files Browse the repository at this point in the history 
Introduce objects and subjects

Add the type system including objects.
Define subjects in the core protocol stage and rework the spec
accordingly. Add the general cloudevents binding specification.

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
  • Loading branch information
@afrittoli
afrittoli committed Jul 7, 2022
1 parent 6559231 commit a5d0e69
Show file tree
Hide file tree
Showing 5 changed files with 369 additions and 28 deletions.
143 changes: 143 additions & 0 deletions cloudevents-binding.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,149 @@ The CloudEvents Binding for CDEvents defines how CDEvents are mapped to CloudEve
## Table Of Contents

<!-- toc -->
- [Context](#context)
- [specversion](#specversion)
- [id](#id)
- [source](#source)
- [type](#type)
- [subject](#subject)
- [time](#time)
- [datacontenttype](#datacontenttype)
- [dataschema](#dataschema)
- [Event Data](#event-data)
- [Example](#example)
<!-- /toc -->

## Context

The CloudEvents context is built by the event producer using some of the data
from the [CDEvents context](spec.md#context).

### specversion

The [CloudEvents `specversion`](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#specversion)
MUST be set to `1.0`.

### id

The [CloudEvents `id`](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#id)
MUST be set to the CDEvents [`id`](spec.md#id).

### source

The [CloudEvents `source`](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#source-1)
MUST be set to the CDEvents [`source`](spec.md#source).

### type

The [CloudEvents `type`](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#type)
MUST be set to the [`type`](spec.md#type) of the CDEvent.

### subject

The [CloudEvents `subject`](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#subject)
MUST be set to the [subject `id`](spec.md#subjectid) of the CDEvent.
__Note__: since the *subject* is mandatory in CDEvents, the `subject` in the
CloudEvents format will always be set - even if it's not mandated by the
CloudEvents specification.

### time

The [CloudEvents `time`](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#subject)
MUST be set to the [`timestamp`](spec.md#timestamp) of the CDEvent. The
CloudEvents specification allows for `time` to be set to either the current time
or the time of the occurrence, but it requires all producers to be chose the
same option. CDEvents requires all producers to use the `timestamp` from the
CDEvent to meet the CloudEvents specification.

### datacontenttype

The [CloudEvents `datacontenttype`](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#datacontenttype)
is optional, its use depends on the specific CloudEvents binding and mode in
use. See the [event data](#event-data) section for more details.

### dataschema

The [CloudEvents `dataschema`](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#dataschema)
is MAY be set to a URL that points to the event data schema included in this
specification.

## Events Data

The content and format of the event data depends on the specific CloudEvents
binding in use. All the example, unless otherwise stated, refer to the
[HTTP binding](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/bindings/http-protocol-binding.md)
in [binary content mode](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/bindings/http-protocol-binding.md#31-binary-content-mode). In this format, the
CloudEvents context is stored in HTTP headers.

### Content Modes

This specification defines two content modes for transferring events:
*structured* and *binary*. The *structured* mode can be used in all cases, the
*binary* mode may only be used in conjunction with the HTTP CloudEvent binding
in *binary* mode:

| CloudEvents / CDEvents | Structured | Binary |
|------------------------|------------|--------|
| HTTP Binary | V | V |
| HTTP Structured | V | X |
| HTTP Batch | V | X |
| Other Binding | V | X |

#### Structured Content Mode

In *structured* content mode, the [CloudEvents Event Data](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md#event-data)
MUST include the full CDEvents [`context`](spec.md#context) rendered as JSON
in the format specified by the [schema](./schemas/) for the event type.

In CloudEvents HTTP binary mode, the `Content-Type` HTTP header MUST be set to
`application/cdevents+json`. In CloudEvents HTTP structured mode, the same
information is carried in the CloudEvents context field `datacontenttype`.

##### Structured Mode Examples

Full example of a CDEvents in *structured* content mode, transported through a
CloudEvent in HTTP *binary* mode:

```json
POST /sink HTTP/1.1
Host: cdevents.example.com
ce-specversion: 1.0
ce-type: dev.cdevents.taskrun.started
ce-time: 2018-04-05T17:31:00Z
ce-id: A234-1234-1234
ce-source: /staging/tekton/
ce-subject: /namespace/taskrun-123
Content-Type: application/cdevents+json; charset=utf-8
Content-Length: nnnn

{
"context": {
"version" : "draft",
"id" : "A234-1234-1234",
"source" : "/staging/tekton/",
"type" : "dev.cdevents.taskrun.started",
"timestamp" : "2018-04-05T17:31:00Z",
}
"subject" : {
"id": "/namespace/taskrun-123",
"type": "taskRun",
"content": {
"task": "my-task",
"url": "/apis/tekton.dev/v1beta1/namespaces/default/taskruns/my-taskrun-123"
"pipelineRun": {
"id": "/somewherelse/pipelinerun-123",
"source": "/staging/jenkins/"
}
}
}
}
```

#### Binary Content Mode

TBD

##### Binary Mode Examples

TBD
142 changes: 123 additions & 19 deletions core.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,31 +10,135 @@ description: >

__Note:__ This is a work-in-progress draft version and is being worked on by members of the Events SIG. You are very welcome to join the work and the discussions!

Continuous Delivery related to activities and orchestration that needs to exist to be able to deterministically and continuously being able to delivery software to users.
Continuous Delivery Core Events include the subjects and predicates related to orchestration of execution required to deterministically and continuously delivery software to users.
Core events are at the lower level of abstraction in the dictionary: they describe the status of an execution, but they don't include any semantic about the function of the execution itself. These events are generated by [pipeline orchestration engines](https://landscape.cd.foundation/card-mode?category=ci-pipeline-orchestration&grouping=category).

A pipeline, in the context of Continuous Delivery, is the definition of a set of tasks that needs to be performed to build, test, package, release and deploy software artifacts.
A pipeline can be instantiated multiple times, for example to build different versions of the same artifact.
We are referring to this instance as PipelineRun. It will have a unique Id and it will help us to track the build and release progress on a particular software artifact.
## Subjects

Due the dynamic nature of Pipelines, most of actual work needs to be queued to happen in a distributed way, hence Queued events are added. Adopters can choose to ignore these events if they don't apply to their use cases.
In the context of Continuous Delivery, a *pipeline* is the definition of a set of *tasks* that needs to be performed to build, test, package, release and deploy software artifacts.
The definition of *pipelines* and *tasks* is an authoring process, and has no event associated to it. CDEvents identifies two [*subjects*](./spec/README.md), [`pipelineRun`](#pipelinerun) and [`taskRun`](#taskrun), which are the runtime counterparts of *pipelines* and *tasks*.

- __PipelineRun Queued__: a PipelineRun has been schedule to run
- __PipelineRun Started__: a PipelineRun has started and it is running
- __PipelineRun Finished__: a PipelineRun has finished it execution, the event will contain the finished status, success, error or failure
| Subject | Description | Predicates |
|---------|-------------|------------|
| [`pipelineRun`](#pipelinerun) | An instance of a *pipeline* | [`queued`](#pipelinerun-queued), [`started`](#pipelinerun-started), [`finished`](#pipelinerun-finished)|
| [`taskRun`](#taskrun) | An instance of a *task* | [`started`](#taskrun-started), [`finished`](#taskrun-finished)|

Each pipeline is defined as a set of Tasks to be performed in sequence, hence tracking this sequence might be important for some cases. A TaskRun is an instance of the Task defined inside the pipeline, as you can expect multiple execution of the pipelines (each a PipelineRun) you can also expect multiple execution of the Tasks, for that reason we use TaskRun to refer to one of these instances.
### `pipelineRun`

- __TaskRun Started__: a TaskRun inside a PipelineRun has started.
- __TaskRun Finished__: a TaskRun inside a PipelineRun has finished.
A pipeline can be instantiated multiple times, for example to build different
versions of the same artifact. We are referring to this instance as
[`pipelineRun`](#pipelinerun). It will have a unique id and it will help us to
track the build and release progress on a particular software artifact.

Pipeline Events MUST include the following attributes:
| Field | Type | Description | Examples |
|-------|------|-------------|----------|
| id | `String` | Uniquely identifies the subject within the source. | `tenant1/12345-abcde`, `namespace/pipelinerun-1234` |
| source | `URI-Reference` | [source](spec.md#source) from the context | |
| pipelineName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` |
| outcome | `Enum` | outcome of a finished `pipelineRun` | `success`, `error` or `failure`|
| url | `URI` | url to the `pipelineRun` | `https://dashboard.org/namespace/pipelinerun-1234`, `https://api.cdsystem.com/namespace/pipelinerun-1234` |
| errors | `String` | In case of error or failed pipeline, provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `pipelineRun cancelled by user`, `Unit tests failed`|

- __Event Type__: the type is restricted to include `dev.cdevents.__` prefix. For example `dev.cdevents.pipelinerun.queued` or `dev.cdevents.tests.started`
- __PipelineRun Id__: unique identifier for a pipeline execution
- __Pipeline Name__: unique identifier for the pipeline, not for the instance. A pipeline can have multiple instances/runs.
- __PipelineRun Status__: current status of the PipelineRun at the time when the event was emitted. If the pipeline is finished, this attribute should reflect if it finished successfully or if there was an error on the execution. Possible statuses: [Running, Finished, Error]
### `taskRun`

Optional attributes:
Pipelines are composed by as a set of Tasks to be performed, like performing a
build, running some tests, publishing an artifact. A taskRun is an instance
of a Task, typically part of a pipeline. If a Pipeline is composed by a single
Task, the execution of pipelineRun MUST generate both the pipelineRun and
taskRun events. Some systems may support execution of Tasks without a Pipeline
associated, in which case it is acceptable to generate only taskRun events.

- __PipelineRun URL__: URL to locate where pipeline instances are running
- __PipelineRun Errors__: error field to indicate possible compilation, test, build and package errors.
| Field | Type | Description | Examples |
|-------|------|-------------|----------|
| id | `String` | Uniquely identifies the subject within the source. | `tenant1/12345-abcde`, `namespace/taskrun-1234` |
| source | `URI-Reference` | [source](spec.md#source) from the context | |
| taskName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` |
| pipelineRun | `Object` ([`pipelineRun`](#pipelinerun)) | The `pipelineRun` that this `taskRun` belongs to. | `{"id": "namespace/pipelinerun-1234"}`|
| outcome | `Enum` | outcome of a finished `taskRun` | `success`, `error` or `failure`|
| url | `URI` | url to the `taskRun` | `https://dashboard.org/namespace/taskrun-1234`, `https://api.cdsystem.com/namespace/taskrun-1234` |
| errors | `String` | In case of error or failed pipeline, provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `taskRun cancelled by user`, `Unit tests failed`|

## Events

### `pipelineRun queued`

Due the dynamic nature of Pipelines, most of actual work needs to be queued to
happen in a distributed way, hence queued events are added. Adopters can choose
to ignore these events if they don't apply to their use cases.

- Event Type: __`dev.cdevents.pipelinerun.queued`__
- Predicate: queued
- Subject: [`pipelineRun`](#pipelinerun)

| Field | Type | Description | Examples | Mandatory ✅ \| Optional ⚪ |
|-------|------|-------------|----------|----------------------------|
| id | `String` | Uniquely identifies the subject within the source. | `tenant1/12345-abcde`, `namespace/pipelinerun-1234` | ✅ |
| source | `URI-Reference` | [source](spec.md#source) from the context | | ⚪ |
| pipelineName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` | ⚪ |
| url | `URI` | url to the `pipelineRun` | `https://dashboard.org/namespace/pipelinerun-1234`, `https://api.cdsystem.com/namespace/pipelinerun-1234` | ⚪ |

### `pipelineRun Started`

A pipelineRun has started and it is running.

- Event Type: __`dev.cdevents.pipelinerun.started`__
- Predicate: started
- Subject: [`pipelineRun`](#pipelinerun)

| Field | Type | Description | Examples | Mandatory ✅ \| Optional ⚪ |
|-------|------|-------------|----------|----------------------------|
| id | `String` | Uniquely identifies the subject within the source. | `tenant1/12345-abcde`, `namespace/pipelinerun-1234` | ✅ |
| source | `URI-Reference` | [source](spec.md#source) from the context | | ⚪ |
| pipelineName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` | ⚪ |
| url | `URI` | url to the `pipelineRun` | `https://dashboard.org/namespace/pipelinerun-1234`, `https://api.cdsystem.com/namespace/pipelinerun-1234` | ⚪ |

### `pipelineRun Finished`

A pipelineRun has finished, successfully or not.

- Event Type: __`dev.cdevents.pipelinerun.finished`__
- Predicate: finished
- Subject: [`pipelineRun`](#pipelinerun)

| Field | Type | Description | Examples | Mandatory ✅ \| Optional ⚪ |
|-------|------|-------------|----------|----------------------------|
| id | `String` | Uniquely identifies the subject within the source. | `tenant1/12345-abcde`, `namespace/pipelinerun-1234` | ✅ |
| source | `URI-Reference` | [source](spec.md#source) from the context | | ⚪ |
| pipelineName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` | ⚪ |
| url | `URI` | url to the `pipelineRun` | `https://dashboard.org/namespace/pipelinerun-1234`, `https://api.cdsystem.com/namespace/pipelinerun-1234` | ⚪ |
| outcome | `Enum` | outcome of a finished `pipelineRun` | `success`, `error` or `failure`| ⚪ |
| errors | `String` | In case of error or failed pipeline, provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `pipelineRun cancelled by user`, `Unit tests failed`| ⚪ |

### `taskRun Started`

A taskRun has started and it is running.

- Event Type: __`dev.cdevents.taskrun.started`__
- Predicate: started
- Subject: [`taskRun`](#taskrun)

| Field | Type | Description | Examples | Mandatory ✅ \| Optional ⚪ |
|-------|------|-------------|----------|----------------------------|
| id | `String` | Uniquely identifies the subject within the source. | `tenant1/12345-abcde`, `namespace/taskrun-1234` | ✅ |
| source | `URI-Reference` | [source](spec.md#source) from the context | | ⚪ |
| taskName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` | ⚪ |
| pipelineRun | `Object` ([`pipelineRun`](#pipelinerun)) | The `pipelineRun` that this `taskRun` belongs to. | `{"id": "namespace/pipelinerun-1234"}`| ⚪ |
| url | `URI` | url to the `taskRun` | `https://dashboard.org/namespace/taskrun-1234`, `https://api.cdsystem.com/namespace/taskrun-1234` | ⚪ |

### `taskRun Finished`

A taskRun has finished, successfully or not.

- Event Type: __`dev.cdevents.taskrun.finished`__
- Predicate: finished
- Subject: [`taskRun`](#taskrun)

| Field | Type | Description | Examples | Mandatory ✅ \| Optional ⚪ |
|-------|------|-------------|----------|----------------------------|
| id | `String` | Uniquely identifies the subject within the source. | `tenant1/12345-abcde`, `namespace/taskrun-1234` | ✅ |
| source | `URI-Reference` | [source](spec.md#source) from the context | | ⚪ |
| taskName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` | ⚪ |
| pipelineRun | `Object` ([`pipelineRun`](#pipelinerun)) | The `pipelineRun` that this `taskRun` belongs to. | `{"id": "namespace/pipelinerun-1234"}`| ⚪ |
| url | `URI` | url to the `taskRun` | `https://dashboard.org/namespace/taskrun-1234`, `https://api.cdsystem.com/namespace/taskrun-1234` | ⚪ |
| outcome | `Enum` | outcome of a finished `taskRun` | `success`, `error` or `failure`| ⚪ |
| errors | `String` | In case of error or failed pipeline, provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `taskRun cancelled by user`, `Unit tests failed`| ⚪ |
13 changes: 13 additions & 0 deletions primer.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -116,3 +116,16 @@ There are two root use cases that we are considering:
The same events can be used to track different metrics over time, to be visualized through a dashboard.

The use cases are work in progress - the list is being drafted in a [separate document](https://hackmd.io/ZCS2KYKZTpKBqhU9PMuCew).

## Design Decisions

### Keys, Values and Types

The CDEvents specification defines event types, keys and, for ENUM types, values.

Event types are defined as all lowercase, separated by dots. The first part of
each type is always "dev.cdevents" which is the reverse DNS domain of the
CDEvents project.

Keys and ENUM values are always written in [lowerCamelCase](https://en.wikipedia.org/wiki/Camel_case)
for readability purposes.
Empty file added schemas/.keep
View file
Empty file.
Loading

0 comments on commit a5d0e69

Please sign in to comment.