New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[apache_tomcat.access] Restructure Grok pattern and ingest pipeline to improve performance #8615
Conversation
🌐 Coverage report
|
/test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some nits. Otherwise the change looks good. Thanks
packages/apache_tomcat/data_stream/access/elasticsearch/ingest_pipeline/default.yml
Show resolved
Hide resolved
packages/apache_tomcat/data_stream/access/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
packages/apache_tomcat/data_stream/access/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
packages/apache_tomcat/data_stream/access/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
Co-authored-by: Ishleen Kaur <102962586+ishleenk17@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed/comment all suggestions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Package apache_tomcat - 1.1.0 containing this change is available at https://epr.elastic.co/search?package=apache_tomcat |
The customer is using this Integration on an ESS and had ~10-14ms per doc average processing Time.
After taking a look at the pipeline definition I recognised the "inefficient" way how the grok processing is done.
First test of restructuring the grok process (without changing what it is doing, just how it is done) I had an average of 0-5-0.-7ms, so a huge performance improvement with just a little restructuring.