Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

name all k8s workqueue #28085

Merged
merged 6 commits into from Nov 3, 2021
Merged

Conversation

newly12
Copy link
Contributor

@newly12 newly12 commented Sep 23, 2021

Enhancement

What does this PR do?

name all k8s workqueue accordingly

Why is it important?

we have metricbeat running with autodiscover enabled to collect metrics from all pods in the entire cluster, we noticed that in some large clusters(in our case tens of thousands of pods) autodiscover is "slow", for example a new pod could take very long time or forever for metricbeat to start scrapping, however it can be seen after restarti metricbeat for several minutes. we add done these code changes and then leverage the default metrics instrumentation done by controller runtime and identified that there were many items pending on processing in the "pod" workqueue.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • queue name is essential for its metrics instrumentation

How to test this PR locally

Related issues

Use cases

get insights into beats' k8s workqueue

Screenshots

Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 23, 2021
@mergify
Copy link
Contributor

mergify bot commented Sep 23, 2021

This pull request does not have a backport label. Could you fix it @newly12? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Sep 23, 2021
@newly12 newly12 changed the title name name all k8s workqueue Sep 23, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 23, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-11-03T01:37:11.841+0000

  • Duration: 124 min 5 sec

  • Commit: b6ff463

Test stats 🧪

Test Results
Failed 0
Passed 53550
Skipped 5225
Total 58775

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

@MichaelKatsoulis MichaelKatsoulis added the Team:Integrations Label for the Integrations team label Oct 20, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Oct 20, 2021
@ChrsMark
Copy link
Member

/test

Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, left one minor

libbeat/common/kubernetes/watcher.go Show resolved Hide resolved
@MichaelKatsoulis
Copy link
Contributor

NewWatcher is also used by elastic-agent Kubernetes provider. If we continue with the named watcher approach which is a useful addition we should consider updating agent too so that we won't need both functions. This could be part of a separate PR

@MichaelKatsoulis
Copy link
Contributor

/test

Copy link
Contributor

@MichaelKatsoulis MichaelKatsoulis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks a lot @newly12. Let's wait CI to be green and then we will merge it!

@mergify
Copy link
Contributor

mergify bot commented Nov 2, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b k8s_work_queue upstream/k8s_work_queue
git merge upstream/master
git push upstream k8s_work_queue

@MichaelKatsoulis
Copy link
Contributor

@newly12 Can you resolve the conflicts in the CHANGELOG.next.asciidoc ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-skip Skip notification from the automated backport with mergify Team:Integrations Label for the Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants