re-design events resource and mechanisms for registering new events #408
Comments
|
Current Event resource schema:
Creation conditions: Creation of Events are implemented server side.
Condition for deleting events:
|
|
Suggestion reuse existing event resource in Nuvla
When we should register an event:
To be completed |
Type of eventsFrom my current understanding, we would like to have events to represent both:
Below I analyze only the first kind of events (internal to the api-server). As-is analysis of event processing in api-serverCategoriesOne of: SeveritiesOne of: Event creationCurrent way to add an event is by calling Example of event created: {:resource-type "event",
:content {:resource {:href "deployment/847a79a2-993a-443a-bc4e-506d7424e1e9"},
:state "deployment/847a79a2-993a-443a-bc4e-506d7424e1e9 created"},
:severity "medium",
:category "action",
:timestamp "2023-07-24T13:37:55.464Z",
:acl {:owners ["user/jane"]}}which is created from this creation payload: {:params {:resource-name "event"},
:body {:resource-type "event",
:content {:resource {:href "deployment/847a79a2-993a-443a-bc4e-506d7424e1e9"},
:state "deployment/847a79a2-993a-443a-bc4e-506d7424e1e9 created"},
:severity "medium",
:category "action",
:timestamp "2023-07-24T13:37:55.464Z",
:acl {:owners ["user/jane"]}},
:nuvla/authn {:user-id "internal",
:active-claim "group/nuvla-admin",
:claims #{"group/nuvla-admin" "group/nuvla-anon" "group/nuvla-user"}}}`sixsq.nuvla.server.resources.event.utils/create-event` is currently called in:
ActionsSometimes actions on a resource trigger crud operations on other resources, Events retentionEvents are retained in ES for 1 year, then they are deleted. Summary
Goals
Non-goals
ProposalEvent spec
[UPDATE 02/08/2023] In the first iteration I tried to get rid of the Resulting top-level spec would be something like: (s/def ::resource
(-> (st/spec (su/only-keys :req-un [::href]
:opt-un [::common/resource-type]))))
(s/def ::schema
(su/only-keys-maps common/common-attrs
{:req-un [::timestamp
::event-type
::message
::content {:req-un [::resourcce {:req-un [::type, ::href]}
::payload]}
::category
::severity
::user-id
::active-claim]
:opt-un [::session-id]}))AclSet acl in such a way that the event is visible for the user/group that created it. Something like this ? {:owners ["group/nuvla-admin"]
:view-meta [(auth/current-active-claim request)]}Is this enough? Not sure yet.. Event config registryHave a global registry of events configurations, keyed by resource type.
Event creation utility fnAn event creation utility fn should be added; use
CRUD operationsMake the generation of events more systematic, by plugging-in the event generation as part of the crud operations. ActionsIt would be good to make eventing more systematic also for actions other then crud. Proposal is to:
Bulk actionsSingle or multiple (one per resource) events or both for bulk actions ?
Async actionsWhen an async action is triggered via the api-server, an event can be generated to signal that the processing Direct event creationRegardless of how generic we make the above mechanism, there will be cases where it does not fit. Events searching and filteringGiven the above, it should be possible for a user with the proper permissions to:
NotificationsTODO: clarify how notifications work in Nuvla. Remove all present calls to
|
This is an interesting proposition to be able to trace events. But I would suggest to add it in a second phase. |
I advise to make it a defmulti to get the config event for each resource. Because we load some resources dynamically from other projects jar. If the config of the resource is defined in each resource namespace, it will be loaded also dynamically at runtime. I think it’s better than having to configure resources that will maybe be loaded later. The other solution is to have function that each resource call at initialization that add event config to the atom registry. But I personally I prefer the defmulti. |
what is the advantage of nesting instead keeping a flat document |
User need at least view-data rights to be able to see the content of the event. (view-meta right give access only to name description common attributes) |
• how to migrate ? We can create a migration script from old schema to the new one. But we can do it at the very end. Do not bother with that at this step. |
|
What is the spec definition for this field? Because ES doesn't play well with open maps. |
Makes sense. Will change it to defmulti, thanks. |
Ok, then given all these impacted systems, I guess it is better if the event spec is backward-compatible ? At least until all other components have migrated. |
I took the spec of The idea would be that this field is a dump of whatever might be useful, but should not be indexed by ES. I didn't see issues with it in my limited tests,do you foresee some problems with the spec above ? |
Interesting suggestion. We will have to check that ES doesn't count the keys in case we set the field as not indexed. |
Ok, for some reason I thought that the keyword |
What do you mean ? Sorry,my understanding of how ES works is pretty basic.. |
Ok, in this scenario I would also create an event when the action succeeds. Otherwise if you are listening to events live, you have no way of distinguishing if an action started but is still in progress vs an action started and succeeded. |
I tested that we can fit a big number of keys in payload and seems to be working when not indexed in ES. 👏 |
I mean that ES limit indexed fields to 1000 keys per index by default. Here is an example of an error if we index the attribute But this is no more relevant, since when the payload attribute is not indexed ES doesn't account payload nested keys as part of that limit. |
Interesting, thanks. |
[Update 11/08/2023]After several code iterations I settled on some design decisions that I try to detail below. For the implementation and latest changes, see the linked PR. Event typesAll events must have an event type:
Event types must belong to a Several public implementation of event systems use event types.Sometimes it is called event name, sometimes it is called event type. StripeSee https://stripe.com/docs/api/events/types Examples
AWSSee https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html Examples{
"version": "0",
"id": "cd4d811e-ab12-322b-8255-872ce65b1bc8",
"detail-type": "event type",
"source": "aws.config",
"account": "111122223333",
"time": "2018-03-22T00:38:11Z",
"region": "us-east-1",
"resources": [
resources
],
"detail": {
specific message type
}
}{
"version":"0",
"id":"2f8147ab-8c48-47c6-b0b6-3ee23ec8d300",
"detail-type":"EMR Auto Scaling Policy State Change",
"source":"aws.emr",
"account":"123456789012",
"time":"2016-12-16T20:42:44Z",
"region":"us-east-1",
"resources":[],
"detail":{
"resourceId":"ig-X2LBMHTGPCBU",
"clusterId":"j-1YONHTCP3YZKC",
"state":"PENDING",
"message":"AutoScaling policy modified by user request",
"scalingResourceType":"INSTANCE_GROUP"
}
}{
"version": "0",
"id": "ffd8a6fe-32f8-ef66-c85c-111111111111",
"detail-type": "Tag Change on Resource",
"source": "aws.tag",
"account": "123456789012",
"time": "2018-09-18T20:41:06Z",
"region": "us-east-1",
"resources": [
"arn:aws:ec2:us-east-1:123456789012:instance/i-0000000aaaaaaaaaa"
],
"detail": {
"changed-tag-keys": [
"key2",
"key3"
],
"service": "ec2",
"resource-type": "instance",
"version": 5,
"tags": {
"key4": "value4",
"key1": "value1",
"key2": "value2"
}
}
}GCPSee for example: https://cloud.google.com/eventarc/docs/event-types Examples
In api-server, for every crud operation or action we define the following event types:
For example, the successful update of a
If the operation above fails for some reason, it will generate the following events instead:
Event detailsBased on the event type (or its category), an event can contain additional details, which are grouped under the key For example, events of category Event parentIn the example above the 2 events generated from the creation of a To make the link between the events explicit we use the property In the examples above the events Some actions trigger other actions or operations internally, which in turn generate other events: this situation ACLThe acl on events is set with the following algorithm:
Implementation detailsEvent managerCRUD operations on events are redirected to an event manager. The event manager has a generic interface and the actual implementation can be switched This makes it possible to easily switch from an implementation to another. The event manager also offers some wrapper functions for crud operations. The idea is CRUD with events facadeNew facade functions to invoke crud operations or actions with events generation enabled:
Per-resource type enabling/disablingEvents can also be enabled or disabled per resource type by redefining the multimethod
Other multi methods
Failure modeAfter further thinking, I am still a bit reluctant about letting an operation proceed when there is an error storing an event. If later on we want to build other functionalities on top of the event system, we must be able to rely I would do the following:
In any case, it is very important for the event creation code to be made simple and free of logical errors as much as possible: to be clear, it should only fail if the events are malformed, or if ES or the network etc. fails. Event testingSince operations and actions on any resource will generate events by default, most of the lifecycle tests can be integrated with checks on the generated events. I started doing this work for the Plan for next week
Plan for later
|
Some enhancement ideas:
|
|
Reason for Pending: work on Projects is impeding the progress. |
the events resource has a very simple schema and it is being underused.
Starting by the NuvlaBox, all relating actions (i.e. activate, commission, decommission, + custom actions , etc.) should generate an event, to be tracked in the NB detail page
The text was updated successfully, but these errors were encountered: