Skip to content

fix(auditd): reduce grok backtracking in pipeline#17648

Merged
andrewkroh merged 2 commits intoelastic:mainfrom
andrewkroh:auditd/bug/grok-backtracking-clean
Mar 16, 2026
Merged

fix(auditd): reduce grok backtracking in pipeline#17648
andrewkroh merged 2 commits intoelastic:mainfrom
andrewkroh:auditd/bug/grok-backtracking-clean

Conversation

@andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Mar 4, 2026

Proposed commit message

Address three parser hot spots in auditd ingest processing.

- Replace one monolithic grok with conditional grok paths to avoid
  failing expensive patterns on unrelated events.
- Remove unbounded quoted payload matching from the msg branch by
  using quote-bounded classes.
- Replace the fallback key-value capture shape that relied on
  GREEDYDATA-style scanning with bounded pre-captures and safer
  fallback matching for non-kv records.

On benchmark pipeline runs, processing_time dropped from 0.42s
(source_doc_count=70, doc_count=1000) to 0.08s in this branch.
The previous single grok hot spot at 75% total time and 315us/doc
is no longer dominant after splitting and bounding captures.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

@andrewkroh andrewkroh added Integration:auditd Auditd Logs bugfix Pull request that fixes a bug issue Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform] labels Mar 4, 2026
Address three parser hot spots in auditd ingest processing.

- Replace one monolithic grok with conditional grok paths to avoid
  failing expensive patterns on unrelated events.
- Remove unbounded quoted payload matching from the msg branch by
  using quote-bounded classes.
- Replace the fallback key-value capture shape that relied on
  GREEDYDATA-style scanning with bounded pre-captures and safer
  fallback matching for non-kv records.

On benchmark pipeline runs, processing_time dropped from 0.42s
(source_doc_count=70, doc_count=1000) to 0.08s in this branch.
The previous single grok hot spot at 75% total time and 315us/doc
is no longer dominant after splitting and bounding captures.

Also bump auditd package version to 3.22.2 and add a changelog
bugfix entry for the backtracking fix.
@andrewkroh andrewkroh force-pushed the auditd/bug/grok-backtracking-clean branch from a09c074 to 198dd09 Compare March 4, 2026 02:20
@andrewkroh andrewkroh marked this pull request as ready for review March 4, 2026 02:21
@andrewkroh andrewkroh requested a review from a team as a code owner March 4, 2026 02:21
@elasticmachine
Copy link

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

History

@andrewkroh andrewkroh merged commit fea9cb6 into elastic:main Mar 16, 2026
8 of 9 checks passed
@elastic-vault-github-plugin-prod

Package auditd - 3.22.2 containing this change is available at https://epr.elastic.co/package/auditd/3.22.2/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:auditd Auditd Logs Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants