Link produced events to consumed events #10
Comments
afrittoli
changed the title
|
I like this initiative as it I think we need this in the protocol. Eiffel also have links but there it stores the information as an object see the links object. Eiffel would call this link a CAUSE link and would define it as shown from an example of a ActivityTriggeredEvent.
and the json for a event would look like this: "links": [
{
"type": "CAUSE",
"target": "aaaaaaaa-bbbb-5ccc-8ddd-eeeeeeeeeee1",
"domainId": "com.example"
}
]From Eiffels perspective I would like to also have the link typed if/when we want to add more links to the protocol. Today I opened #19 to add a note we made on a vocabulary meeting about how we want to handle extensions. Do we want this as an extension or as part of the data field? |
Oh, interesting, I will go read more about cause vs. context in Eiffel
I would not add this to #19 for now. Hopefully we can merge #19 soon and adding more to it might slow the process down. What do you think about the open question on |
|
Really nice! Being able to have links to previous event will really help with things like visualizing chains without knowing beforehand which events to expect. The "cdevents_source-event" is a bit unclear to me. |
In such cases it would make sense to me to refer back to the corresponding "XStarted" event. The direct "cause" for an activity ending is that it was started in the first place. |
"Source + Subject" Identifies the activity The "source" part is needed to make the subject / id reference globally unique.
The |
This was my initial thought, but ultimately I decided against it in my proposal. The If we added the For pipelinerun specifically, the initial event is actually "queued" and that would be the one to include the source event. |
|
So again from an Eiffel perspective if we have links with "types" I think we could solve the problem. For example Eiffels EiffelActivityFinishedEvent links back to the EiffelActivityTriggeredEvent via the ACTIVITY_EXECUTION link. In CDEvents we could link our @afrittoli Does this give any clue on the |
@afrittoli Sorry for my fussy language. I wanted to ask if this initiative described by your issue be a extension or should it be moved to the |
Heh, that was me not reading properly, sorry about that. "Dig into the body" out of context sounds really bad 🧟 |
I do see the benefit for having the links outside of the body but that would limit us to strings (if I understand the CloudEvents spec correct). If we would do as in Eiffel and type them it would make more sense to have the link as an object but then we are forced inside the |
We could have both. The string is a serialised version, with limited information, used for routing. |
Hmm, that could be interesting, worth discussing. Could you then have something like I am pasting this picture from my presentation on link I held on SIG event in May last year showing how Eiffel events are linked together.
|
Why do you need to have type embedded in the source event?
|
If we want to have more links types it would be easier to read for a human and also could also be used when validating events. It could look like |
|
Also when querying a database for event chains having the link type in the top level (and not just in the body) enables restricting the query results for the needed link types. E.g. in the picture above if you want to follow the link chain from |
|
Hello, @e-backmark-ericsson pointed me to this GitHub issue when I asked about a DAG structure between events at the recent CDEvents Community Summit, and this page has some interesting ideas. With tracing for example, there is a concept of an all-encompassing “Trace” that contains all the “Spans” inside it. Each span can have a parent (the “link” between nodes), but the code declaring a span does not necessarily need to know its parent ID. This is because the context attached to the trace itself includes the concept of a “current span”, under which new spans can be attached. The parent IDs are propagated in order to maintain these links between nodes. This model fits pretty well with tracing since a trace has this waterfall-like structure where new spans are often added under one that was previously started. But how would this work with CDEvents? If my full CI pipeline was something as simple as this:
Then at each step the events that are generated would need to “point” to a parent event to gradually build the DAG. Which means… what? That the first event in the CI pipeline would have to be given the event ID for the commit event? and then each one after that would need to remember the last ID? Given the variety of systems involved, doing this without a standardized way of transmitting and receiving this parent information would get messy very quickly. When you start having complex graphs with multiple branches and systems triggering subsystems, it becomes difficult to always know the parent ID that a new event should be pointing to. How was this done with Eiffel events? |
|
The links between events are a very core part of the Eiffel event structure, and I wouldn't say it is particularly messy at all to use them. This might require more hands-on examples and clarifications, but the bottom line is that it is not really complicated. |
I think Eiffel definition of links is very neat and clear, and it provides great value when all system involved speak Eiffel natively and there is a global database of events available.
This is possible, but it requires embedding the event ID deeply in the core of the the CI system. In some cases the link between platforms may happen through interfaces that are already standardised and it might be non-trivial to introduce extra context there. When an image is pushed to an OCI registry, we may wish to send a CDEvent about it, produced by the registry. The registry however will not have context about what happened before that. Something tools that work with container images take the approach of storing things like the image signature and attestation into a separate artifact that is linked to the original through the artifact sha. We could design a cdevents artifact for the purpose of providing enough context to the container registry. Alternatively an even consumer could use the artifact sha or the attestation (when available) to infer the relation with the initial part of the workflow that lead to the image push.
My feeling is that the further we move towards the end of the workflow (deployments, incidents, rollbacks etc), the harder it becomes for the event producer to have a source event ID in the context. My current take on links is that we should specify them as optional, and use case by use case provide recommendations about how a system may provide such link. Alternatively we could state that links may be used only when a database of links is available, and somehow build in the SDK the logic to pick the correct link. |
That's a good way to put it, and what I thought made it more strongly coupled with the system being monitored than it needs to be.
This is another good example, which shows how lightly-connected these systems can be, and how passing IDs around makes it more difficult to think of them as various building blocks that can be arranged in different ways. It's also not a given that all protocols will support the injection of this additional metadata: when you push an image with The mention of the events database where IDs could be looked up would help with making these events a lighter addition to the existing process than a core piece of state that is passed around. If your CI pipeline already an identifier like a Jenkins build ID or its own "pipeline ID", then it would be possible for step
I tend to agree, but it seems to me that without such links the only structure left to arrange events in a coherent sequence is to use their timestamps. Putting aside the very real issue of synchronizing clocks across space, it feels like this ends up limiting their use to a linear sequence of events - that is, without branches. Even for a basic pipeline with two branches where events have monotonically increasing timestamps in each branch, it becomes impossible to distinguish the events emitted by one branch from those emitted by the other. Doing so requires additional metadata. I suppose you could also tag the events with |
|
It's hard to explain this in text only so I drew a picture to exemplify how the linking could be achieved. The event abbreviations stand for "Change Merged", "Artifact Packaged", "Test Suite Finished", "Artifact Published".
Alt 1. Each step is controlled by a tool that is fully supporting CDEvents so it can listen to CDEvents events and aggregate data from multiple events if needed. Each step is directly triggered on an incoming event. No external event storage needed. The steps triggered by events will see the event id in the event it is triggered by and can easily include that in the event sent from the step. Alt 2. Each step is controlled by a tool that is aware of CDEvents but cannot directly trigger on such events. Instead it is explicitly called through a REST call or other RPC like call. The tool uses data recieved from the caller (e.g. SCM change/commit id, artifact identity and version, test suite identity, or similar) and uses that to perform lookups towards an event storage. The id of the event returned by that lookup can be included as a link in the new event to be sent. Alt 3. Each step is controlled by a CI/CD orchestrator, for example Tekton, Argo Workflows, Keptn, or similar. The steps themselves has no knowledge about CDEvents, and not the SCM system either. The orchestrator that sends all events will be aware of the ids of them and can easily add them to the events sent as links as needed. And of course any combination of the above could be used as a CI/CD setup. With this I believe I have showed that is shouldn't be very hard to accomplish the needed linking between the events. Of course we could leave the links as optional, at least to begin with, to prove them out with PoCs and examples, but I do believe they would satisfy several needs of measurability, visibility, traceability etc of end-to-end CI/CD pipelines. They could even span between SW projects, if there are means to propagate events between them and provide means to store events and query for them as needed. |
Thanks @e-backmark-ericsson for drawing the diagrams. I think links would be undoubtedly beneficials; my only concern is that adding links into the events requires an amount of knowledge or context that may not be easily available to the sender, at least not right now, which is why I think they should be in the spec but optional for now. If not, I think we would very much inhibit the adoption, or perhaps even worst foster an adaption that is not compliant to the spec.
I think it's fair to assume that most tools won't receive events directly. They may produce them. When they do receive events, the receiver is likely to be a dedicated microservice, that accepts the event(s), filters them based on some logic, and runs the tool if needed. So, for the tool to produce an event which links back to the source event(s), it must have some place it its own input model where to get the source event IDs from. Wrapping a tool with your pipeline orchestrator of choice is a common strategy to add an event receiver and producer to your tool, but even so the link may not be obvious. In Tekton, the triggers component receives the events and runs the pipeline that wraps the tool, but there is no Tekton native place where to carry source event information so that is sent in the outgoing event. Today in existing PoCs we use k8s annotations with specific names to achieve that. We could devise and document recommended strategies to achieve this.
The caveat here is the lookup. Events are unequivocally identified by an (event ID, source) tuple, but no other that in the event is expected to unequivocally identify the event.
This approach works when events are used by an "external" observer. In this case the orchestrator can also embed a context ID of some kind (build ID, pipeline ID) in the events to give more context to the observer.
|
|
I agree that we should have the links as optional parameters, at least to begin with. There is, based on my experience and also on discussions in several CDEvents meetings, a strong need to relate events to each other. And the need is the relate the events end-to-end, from source change to customer deployment (and beyond?), to be able to visualize, trace and provide metrics end-to-end. There are at least three different ways to do that. Two of those require some logic on the producer side - adding links to the events, or adding a well-known context in all "related" events. The third option would be to let the consumers of the events implement that logic themselves, without any relational info in the events. Implementing that logic on the consumer side would be problematic, as many consumers will not have a required knowledge about the producers and their domain within themselves to figure out how the events relate to each other, so I think that is a dead end. Remains the two options for the producer side. Including a dedicated context in each event that are related to each other could be an option, but it is very limited. It would be singe-dimensional and would be depending on event timestamps to find out the order of occurrences. Being single-dimensional makes it virtually impossible to describe parallel (fan-out/fan-in) activities in the pipelines. And the dedicated context object needs to be propagated to each event-sending service to be included in the events to sent. To be able to use links between events, the source event ids needs to be propagated in a similar way to if a dedicated context was used, and that could be an issue. Not least in the case described by @afrittoli above where a dedicated CDEvents aware wrapper microservice triggers on a CDEvent and the calls a tool internally as there might not be simple ways to propagate that data to an event sender within the tool. We'd need to look at each such tool case by case. Most tools have some kind of environment/workflow parameters that could be used for such needs. So, to answer some comments above:
I agree, the links should be optional
Yes, but the same goes for any other relational info needed to be included on the producer side. Without it the relations need to be created on the consumer side which will be almost impossible in many cases.
Yes, there needs to be tool specific implementations for this kind of relational data, being it "links" or "dedicated context" data
Yes
Without unequivocally identifiable event data it will not be possible to relate events deterministically. References such as links (event ID, source), dedicated contexts, or some other event specific unique data to be evaluated by the consumer, needs to be available to be able to relate the events.
This is not true. We've used one single plugin for Eiffel events for years, and it is responsible both for triggering builds on incoming events and for sending events linking to those incoming events. |
|
Related PR: #139 |
|
After carefully reading this issue, most have been solved by #139. What's left is propagation, which is mentioned in the spec already via headers, but needs actual implementation. This issue can be closed, and I will create a follow up issue in the appropriate place (still looking to where that is) for implementing propagation in the SDKs and document how services can propagate these headers, etc. |
Feature Description
The CDEvents specification should allow propagating identifiers from consumed events into produced events.
When an activity is directly started as a consequence of a consumed CDEvents, an identifier of the consumed event must be included in the produced events related to the activity.
When an activity is directly started as a consequence of a consumed event (non CDEvent), an identifier of the consumed event may be included in the produced events related to the activity.
Use Cases
Children Pipelines
When a pipeline runs, it generates events as different tasks are started and finished. Children pipelines may be triggered by these events (whether in the same CD system or a different one), and they will in turn send events.
As a DevOps engineer I want to be able to discover children pipelines from the events they generated.
Proposal
These are the spec fields that would be required for CDEvents.
We already use id and source in our PoC, however we do not specify them in our spec, but we should add them.
The
source-eventfield is a globally unique identified of another event, which was the trigger to the activity identified by "source + subject".CloudEvents have a Source and ID attached to them. Each instance of each platform is responsible for generating unique Source + IDs for its events, but it's not required for one platform to guarantee global uniqueness.
If we enforce global uniqueness, it would responsibility of the administrator of the overall system to ensure each source is configured with a globally unique value (e.g. a DNS name) that can be used to generated the various sources.
If we do not enforce global uniqueness, we could introduce the cdevents_platform extension or the cdevents_source extension to provide a globally unique value. Again it would be responsibility of the overall administrator to configure a globally unique seed. Alternatively platform could use some strategy to pick a name that is likely to be globally unique. Examples:
{ "specversion" : "1.0", "type" : "cd.events.taskrun.started", "source" : "/apis/tekton.dev/v1beta1/namespaces/default/taskruns/curl-run-6gplk", "subject" : "my-taskrun-123", "id" : "A234-1234-1234", "time" : "2018-04-05T17:31:00Z", "cdeventssource" : "tekton_ci_prod/apis/tekton.dev/v1beta1/namespaces/default/taskruns/curl-run-6gplk", "cdeventsplatform" : "tekton_node123_clusterABC", "cdeventssourceevent": "A123-5678-9012@jenkins_ci_prod/pipeline/my-source-pipeline", "datacontenttype" : "text/json", "data" : "{}" }Open Questions
It may not be possible for a platform to implement the source-event extension in all outgoing events, because of a few reason:
cdeventsare implemented via an adaptation layer, the information about the source event may not be there at allThis means that a consumer of events that would like to visualise the end to end workflow triggered by an initial event will have to use a combination of source-event, subject and event type to discover all relevant events.
The text was updated successfully, but these errors were encountered: