Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Osquerybeat: Improve osquery client connect code. Update config_refresh to 60 seconds. #28848

Merged
merged 1 commit into from
Nov 6, 2021
Merged

Osquerybeat: Improve osquery client connect code. Update config_refresh to 60 seconds. #28848

merged 1 commit into from
Nov 6, 2021

Conversation

aleksmaus
Copy link
Member

What does this PR do?

Fixes code logic in osquery connect with retries.
The error not correctly propagated from this line:
https://github.com/elastic/beats/blame/20d637e1afc7104e7a1f24532cd491e59ad6b245/x-pack/osquerybeat/internal/osqdcli/client.go#L138
when the context was cancelled while waiting next retry.

This doesn't cause an issue, but is not the correct behavior and could be potentially be misleading when logging

osquery client is connected

when the connection attempt was cancelled via context.

This PR also updates the config refresh interval from 10 seconds to 60 seconds.
This is the interval how often osqueryd polls our configuration plugin (implemented in osquerybeat) over local RPC (osquery domain socket or pipe). The reasoning for that is explained in the added comment:

// Osquery does not expect that frequent policy/configuration changes
// and can tolerate non real-time configuration change application.

This follows the similar change in the kolide/launcher:
kolide/launcher@5d31869#diff-4fb691d7c9dc698ab635973bdc357a76d02d94583bc6e0c3e8f5483b2f55996aR150

Summary of changes:

  • Refactored/extracted retry logic for osquery reconnect, made it better testable
  • Added unit tests covering retry.go
  • Updated config_refresh from 10 to 60 seconds

Why is it important?

Improves osquery client connect code quality, and unit test coverage, fixes a minor defect there.
Updates for less frequent RPC calls to configuration plugin.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas

Screenshots

Debug log changes
Screen Shot 2021-11-05 at 10 36 28 AM

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Nov 5, 2021
@mergify
Copy link
Contributor

mergify bot commented Nov 5, 2021

This pull request does not have a backport label. Could you fix it @aleksmaus? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Nov 5, 2021
@aleksmaus aleksmaus added backport-v7.16.0 Automated backport with mergify v7.16.0 labels Nov 5, 2021
@mergify mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Nov 5, 2021
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-11-05T15:04:47.009+0000

  • Duration: 77 min 59 sec

  • Commit: fd070fb

Test stats 🧪

Test Results
Failed 0
Passed 4083
Skipped 0
Total 4083

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

@aleksmaus aleksmaus merged commit d2e3b99 into elastic:master Nov 6, 2021
@aleksmaus aleksmaus deleted the improve/osquery_client_reconnect branch November 6, 2021 13:59
mergify bot pushed a commit that referenced this pull request Nov 6, 2021
@aleksmaus
Osquerybeat: Improve osquery client connect code (#28848)
843b668 
(cherry picked from commit d2e3b99)
@mergify mergify bot mentioned this pull request Nov 6, 2021
Merged
aleksmaus added a commit that referenced this pull request Nov 6, 2021
@mergify @aleksmaus
Osquerybeat: Improve osquery client connect code (#28848) (#28854)
dafb447 
(cherry picked from commit d2e3b99)

Co-authored-by: Aleksandr Maus <aleksandr.maus@elastic.co>
v1v added a commit to v1v/beats that referenced this pull request Nov 8, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into feature/use-arch-…
3afea6c 
…in-the-package-binareis

* upstream/master:
  allows disable pod events enrichment with deployment name (elastic#28521)
  Remove Docker input from Filebeat (elastic#28817)
  [breaking] Make default_field: false the default for all fields (elastic#28596)
  Osquerybeat: Improve osquery client connect code (elastic#28848)
  Add crawler  metrics into the stats metricset for Enterprise Search (elastic#28790)
  Remove the now deprecated appsearch module from metricbeat (elastic#28850)
  Remove Beat generators (elastic#28816)
  chore: upload files to Google Storage when they exist (elastic#28836)
  Revert "chore(ci): disable E2E tests in Beats (elastic#28715)" (elastic#28812)
  Deprecate generating custom Beats (elastic#28814)
  [Metricbeat] upgrade flatbuffers to 1.12.1 (elastic#28094)
  Osquerybeat: Fix restart flags after previously bad config (elastic#28827)
  Force ECS and JSON logging for libbeat/logp (elastic#28573)
  Filebeat: Error on startup for unconfigured module (elastic#28818)
  Deprecate log input in favour of filestream (elastic#28623)
  Fix some spelling mistakes (elastic#28080)
@aleksmaus aleksmaus added the backport-v8.0.0 Automated backport with mergify label Nov 10, 2021
mergify bot pushed a commit that referenced this pull request Nov 10, 2021
@aleksmaus
Osquerybeat: Improve osquery client connect code (#28848)
0aa92f5 
(cherry picked from commit d2e3b99)
@mergify mergify bot mentioned this pull request Nov 10, 2021
Merged
aleksmaus added a commit that referenced this pull request Nov 11, 2021
@mergify @aleksmaus
Osquerybeat: Improve osquery client connect code (#28848) (#28912)
d87b2e1 
(cherry picked from commit d2e3b99)

Co-authored-by: Aleksandr Maus <aleksandr.maus@elastic.co>
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@v1v
Merge remote-tracking branch 'upstream/master' into feature/use-arch-…
061692b 
…in-the-package-binareis

* upstream/master:
  allows disable pod events enrichment with deployment name (elastic#28521)
  Remove Docker input from Filebeat (elastic#28817)
  [breaking] Make default_field: false the default for all fields (elastic#28596)
  Osquerybeat: Improve osquery client connect code (elastic#28848)
  Add crawler  metrics into the stats metricset for Enterprise Search (elastic#28790)
  Remove the now deprecated appsearch module from metricbeat (elastic#28850)
  Remove Beat generators (elastic#28816)
  chore: upload files to Google Storage when they exist (elastic#28836)
  Revert "chore(ci): disable E2E tests in Beats (elastic#28715)" (elastic#28812)
  Deprecate generating custom Beats (elastic#28814)
  [Metricbeat] upgrade flatbuffers to 1.12.1 (elastic#28094)
  Osquerybeat: Fix restart flags after previously bad config (elastic#28827)
  Force ECS and JSON logging for libbeat/logp (elastic#28573)
  Filebeat: Error on startup for unconfigured module (elastic#28818)
  Deprecate log input in favour of filestream (elastic#28623)
  Fix some spelling mistakes (elastic#28080)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@james-elastic james-elastic james-elastic approved these changes

@scunningham scunningham Awaiting requested review from scunningham

@lykkin lykkin Awaiting requested review from lykkin

Assignees
No one assigned
Labels
backport-v7.16.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify bug cleanup Team:Asset Mgt v7.16.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@aleksmaus @elasticmachine @james-elastic