Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for metadata and types in expressions #1998

Closed
dlvenable opened this issue Nov 2, 2022 · 2 comments
Closed

Support for metadata and types in expressions #1998

dlvenable opened this issue Nov 2, 2022 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@dlvenable
Copy link
Member

Is your feature request related to a problem? Please describe.

The Data Prepper event model has support for metadata which includes type information.

Data Prepper expression syntax allows for accessing data using JSON Pointer syntax. For example, /status_code. However, there is no mechanism to access Event Metadata.

Some pipeline authors would like to perform pipeline routing based on event type. Some others might want to filter based on tags (#629).

Describe the solution you'd like

Design and implement a syntax that allows for accessing any EventMetadata.

Additional context

The discussion around supporting Event tags offers a proposal for getTags() in this comment.

@kkondaka
Copy link
Collaborator

Working on adding getMetadata() function to DataPrepper Expressions which looks up a key in the event's metadata attributes. For event type, getEventType() function can be added in future.
Examples - getMetadata("key1") getMetadata("key1/key2"), getMetadata("key1/key2/key3")when "/" present, the lookup is done nested way. This is similar to the way key lookup is done in events. The keys could start with "/" (as ingetMetadata("/key1") getMetadata("/key1/key2"), getMetadata("/key1/key2/key3") ) and they are treated same as the keys without preceding "/"

@kkondaka
Copy link
Collaborator

Fixed by PR #2690

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Archived in project
Development

No branches or pull requests

2 participants