Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.14](backport #39544) [winlogbeat] performance improvment; avoid rendering event message twice #39573

Merged
merged 1 commit into from
May 16, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented May 15, 2024

Proposed commit message

The change increases events-per-second throughput by about 30%.

I have improved the throughput by using a fixed buffer size at first attempt of message parsing.

I thought about adding a config parameter to control the size of the initial buffer but after some research I believe it's not needed. I use the size 16KB already existing in our code base. It's relatively small, but the majority of windows event log messages are even much smaller. Historically the event log was designed to store only a message templates to be filled in with actual strings or values from external resources when viewing.

This PR does not pose a regression risk, related to #35437 The former code was prone to error, on certain Windows releases, by using the out parameter BufferUsed instead of relying only on C-style string format when handling the output. The documentation of this parameter is a bit vague, in practice it's only needed when ERROR_INSUFFICIENT_BUFFER is returned.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Author's Checklist

  • [ ]

How to test this PR locally

Setup a reasonably large VM, for example 8 CPU, 32 GB RAM. Prepare an event log with thousands of entries (or just create empty event log and prepare a tool to quickly produce events).
Prepare winlogbeat.yaml config pointing to the event log, use file output. Configure http statistics endpoint to periodically check the progress. Run winlogbeat.exe with the same config, on the same machine, before and after the change or run them side-by-side. Observe the increased events-per-second throughput.

Related issues

Use cases

Screenshots

Logs


This is an automatic backport of pull request #39544 done by [Mergify](https://mergify.com).

…ice (#39544)

* wineventlog performance improvment; avoid rendering message twice

* ignore missing or mismatched parameter values

* add comment

* changelog

* actually increase the buffer

(cherry picked from commit d2ebffe)
@mergify mergify bot requested a review from a team as a code owner May 15, 2024 11:49
@mergify mergify bot added the backport label May 15, 2024
@mergify mergify bot assigned intxgo May 15, 2024
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 15, 2024
@botelastic
Copy link

botelastic bot commented May 15, 2024

This pull request doesn't have a Team:<team> label.

@intxgo intxgo merged commit c92d942 into 8.14 May 16, 2024
16 of 17 checks passed
@intxgo intxgo deleted the mergify/bp/8.14/pr-39544 branch May 16, 2024 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant