Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

filebeat/config: set filebeat.registry.flush default to 1s #31473

Merged
merged 9 commits into from
May 18, 2022
Merged

filebeat/config: set filebeat.registry.flush default to 1s #31473

merged 9 commits into from
May 18, 2022

Conversation

belimawr
Copy link
Contributor

@belimawr belimawr commented Apr 29, 2022 •
edited
Loading

Currently this PR is rebased onto #31588. Once it's merged I'll update this PR and make it ready for review.

What does this PR do?

Set filebeat.registry.flush default to 1s

Why is it important?

This will reduce the CPU usage and disk I/O from Filebeat, while
keeping the changes of data duplication in the event of a crash to a
minimum.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

## Author's Checklist
## How to test this PR locally

Related issues

## Use cases
## Screenshots
## Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 29, 2022
@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 29, 2022 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-05-17T14:42:03.979+0000

  • Duration: 78 min 41 sec

Test stats 🧪

Test Results
Failed 0
Passed 6236
Skipped 732
Total 6968

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@belimawr belimawr force-pushed the filebeat-registry-flush-1s branch from 2866c43 to b432190 Compare May 10, 2022 16:22
@jlind23 jlind23 added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label May 11, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 11, 2022
@jlind23
Copy link
Collaborator

jlind23 commented May 11, 2022

/test

@belimawr belimawr force-pushed the filebeat-registry-flush-1s branch from 4b81a96 to 6b5a780 Compare May 12, 2022 09:28
@mergify
Copy link
Contributor

mergify bot commented May 12, 2022

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b filebeat-registry-flush-1s upstream/filebeat-registry-flush-1s
git merge upstream/main
git push upstream filebeat-registry-flush-1s

@belimawr belimawr force-pushed the filebeat-registry-flush-1s branch 2 times, most recently from bed6dbf to 9aba566 Compare May 12, 2022 14:20
@belimawr belimawr marked this pull request as ready for review May 12, 2022 15:19
@belimawr belimawr requested a review from a team as a code owner May 12, 2022 15:19
@belimawr belimawr requested review from rdner and cmacknz and removed request for a team May 12, 2022 15:19
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@belimawr
filebeat/config: set filebeat.registry.flush default to 1s
66377c4 
This will reduce the CPU usage and disk I/O from Filebeat, while
keeping the changes of data duplication in the event of a crash to a
minimum.

Linter issues are also fixed
@belimawr
Fixing tests
260960e
@belimawr
Making linter happy
28d3ccc
@belimawr belimawr force-pushed the filebeat-registry-flush-1s branch from fe05f6d to 28d3ccc Compare May 12, 2022 15:24
cmacknz
cmacknz reviewed May 12, 2022
View reviewed changes
CHANGELOG.next.asciidoc Outdated Show resolved Hide resolved
cmacknz
cmacknz requested changes May 12, 2022
View reviewed changes
Copy link
Member

@cmacknz cmacknz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes to the CHANGELOG.

@mergify
Copy link
Contributor

mergify bot commented May 15, 2022

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b filebeat-registry-flush-1s upstream/filebeat-registry-flush-1s
git merge upstream/main
git push upstream filebeat-registry-flush-1s

@rdner rdner removed their request for review May 16, 2022 09:12
cmacknz
cmacknz reviewed May 16, 2022
View reviewed changes
CHANGELOG.next.asciidoc Outdated Show resolved Hide resolved
belimawr and others added 4 commits May 17, 2022 16:28
@belimawr @cmacknz
Update CHANGELOG.next.asciidoc
126a60e 
Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co>
@belimawr @kvch
Update filebeat/_meta/config/filebeat.global.reference.yml.tmpl
356e1d6 
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
@belimawr
fix test timeouts
d123c82
@belimawr
mage check/fmt
8501f24
@belimawr belimawr merged commit 8c36359 into elastic:main May 18, 2022
@belimawr belimawr deleted the filebeat-registry-flush-1s branch May 18, 2022 09:31
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
@belimawr @cmacknz
@kvch @chrisberkhout
filebeat/config: set filebeat.registry.flush default to 1s (#31473)
1d6997d 
This will reduce the CPU usage and disk I/O from Filebeat, while
keeping the chances of data duplication in the event of a crash to a
minimum.

Linter issues are also fixed.

Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kvch kvch kvch left review comments

@cmacknz cmacknz cmacknz approved these changes

Assignees

@belimawr belimawr

Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Filebeat] Set filebeat.registry.flush default to 1s
5 participants
@belimawr @elasticmachine @jlind23 @cmacknz @kvch