Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement context backends for the contextualizer #3684

Merged
merged 47 commits into from Dec 1, 2023
Merged

Implement context backends for the contextualizer #3684

merged 47 commits into from Dec 1, 2023

Conversation

Dakostu
Copy link
Member

@Dakostu Dakostu commented Nov 28, 2023

This PR implements context backends for the contextualizer.

TODO: Tests.

Dakostu and others added 28 commits November 28, 2023 17:05
@Dakostu
Implement the first lookup_table_plugin
6fdd2c8
@Dakostu
Implement the lookup-tables aspect
d8ce2cb
@Dakostu
Implement adding indicators to the lookup result
84c7e83
@tobim @Dakostu
Factor typed_array into a first class type
69e7b3e
@tobim @Dakostu
Add a function to parse data to the parser interface
80a1dea
@tobim @Dakostu
Add a context plugin type
5966dfa
@tobim @Dakostu
Update tenzir-plugins
14d372d
@tobim @Dakostu
Bump plugins
3535d45
@tobim @Dakostu
Add a constants context plugin
80f5140 
This context can be used to enrich events with constant fields that
are defined with the context create operator:

```
context create myconstants --type constants foo=bar
```
@tobim @Dakostu
Document context create and remove
b1d8ef4
@Dakostu
Change update() in the context API to return a record
3537561
@Dakostu
Implement a basic apply() workflow for the hashtable context
47678e0
@tobim @Dakostu
Finalize the context plugin interface
78f6dae
@Dakostu
Extend the contextualizer update()
f7d9f6f 
wip
@Dakostu
Make the context application use only one field with a vector<typed_a…
58d1f93 
…rray> return type
@Dakostu
Replace offset tables with a context table in the hashtable context
7956f8e
@dominiklohmann @Dakostu
Implement show contexts for the hashtable context
72a0d55
@dominiklohmann @Dakostu
Consider discard, export, and import as internal operators
f660719 
We do not want to count egress or ingress for discard, export, and
import. This makes it so their metrics are specifically annotated to be
internal, as they are technically not ingress or egress.
@dominiklohmann @Dakostu
Update integration test references for the pipeline manager
2fd760c
@dominiklohmann @Dakostu
Add the enrich operator
dd0fd20
@dominiklohmann @Dakostu
Make progress towards serialization for hashtable contexts
8d1bfe0
@dominiklohmann @Dakostu
Rename leftover mention of lookup tables
f11a8b4
@dominiklohmann @Dakostu
Adjust constants context to new enrich API
2fb12ae
@dominiklohmann @Dakostu
Add read json --arrays-of-objects
cd14a83
@Dakostu
Rename hashtable context to lookup-table context
d3d3489
@Dakostu
Remove the old lookup-tables plugin
968da0d
@Dakostu
Unify the update, creation and aspect output of the contextualizer
6642b64
@Dakostu
Implement serialization and deserialization of contextualizer contexts
d5f06f7
@Dakostu Dakostu added the feature New functionality label Nov 28, 2023
@Dakostu Dakostu requested a review from a team November 28, 2023 16:06
@Dakostu Dakostu marked this pull request as ready for review November 29, 2023 15:57
dominiklohmann
dominiklohmann approved these changes Nov 30, 2023
View reviewed changes
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one last question, then I think this is ready to go.

libtenzir/builtins/aspects/lookup-tables.cpp Outdated Show resolved Hide resolved
@Dakostu Dakostu merged commit 4fb4bc1 into main Dec 1, 2023
44 checks passed
@Dakostu Dakostu deleted the topic/contextulization branch December 1, 2023 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tobim tobim tobim left review comments

@jachris jachris jachris left review comments

@dominiklohmann dominiklohmann dominiklohmann approved these changes

Assignees
No one assigned
Labels
feature New functionality
Projects
None yet
Milestone
No milestone
4 participants
@Dakostu @tobim @dominiklohmann @jachris