-
-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement a first basic pipeline string parser for the currently existing pipeline operators #2877
Conversation
Thanks for the good test coverage. I think the only code related issue is Dominik's suggestion about the return value from the option_set parse method. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly minor things for the docs and some inline notes in the code. I didn't review the parsers in-depth. This is a great piece of work, and I'm really looking forward to use this every day.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a minor comment with regards to initialization. Great work!
@Dakostu would you mind squashing your commits into a smaller number of logical units? The PR consists of 78 commits now and many of them are small fixups. I understand it is already approved, but doing such a final pass will make it easier to navigate history should it become necessary. |
Implement string parsing for pseudonymize pipeline operator Implement string parsing for extend pipeline operator Implement string parsing for replace pipeline operator Implement string parsing for rename pipeline operator Allow aggregate pipelines from VAST nodes (quick hacky way to allow summarize operator) Implement parsing for summarize pipeline operator Implement aggregation function name assignment in summarize operator
043243d
to
52b42a7
Compare
Add pipeline parsing unit tests Add first batch of pipeline string parsing unit tests Add unit test 'pipeline string parsing - options - operator with wrong short form option' Implement unit test 'pipeline string parsing - options - operator with wrong long form option' Implement simple pipeline parsers in new file parseable/vast/pipeline.hpp Add unit test 'option set - missing option value'
Add integration test 'Pipeline operator parsing only summarize' Add 'identity' to pipeline integration test Add integration test 'Pipeline operator parsing after expression' Add integration test 'Pipeline operator summarize after expression' Add -b option to integration tests that need blocking import
…arsing validation
…tion set parsing unit tests
Remove unnecessary inclusion
Specify that pipelines can come after expressions in pipeline Add empty lines around example headline in docs Elaborate on pipeline operator syntax and examples in docs Add "for" to changelog entry Co-authored-by: Dominik Lohmann <mail@dominiklohmann.de> Specify that the syntax is subject to change in changelog entry Co-authored-by: Dominik Lohmann <mail@dominiklohmann.de>
52b42a7
to
7d5b455
Compare
I squashed the number down to 38. |
…nd plugins use it for parsing input strings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I only looked at the last commit fixing the handling of the query language plugin. All comments I've left are trivial to address, so feel free to just resolve them after.
965d1e4
to
8fa5cf5
Compare
8fa5cf5
to
1ca24e6
Compare
This PR adds the ability to parse & execute the currently implemented pipeline operators in the command line, via
vast export <pipeline>
.Fixes tenzir/issues#6
TODO:
- [ ] if slated for 3.1 release: changelog item & docu