Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes #24051 #24052

Merged
merged 3 commits into from Feb 17, 2021

Conversation

jeffspahr
Copy link
Contributor

@jeffspahr jeffspahr commented Feb 16, 2021

What does this PR do?

Adds nodes to the fileabeat ClusterRole.
Fixes #24051

Why is it important?

Filebeat will throw an error and be unable to list nodes without this.

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
    - [ ] I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

Deploy Filebeat from the latest manifest in the docs:
https://www.elastic.co/guide/en/beats/filebeat/7.11/running-on-kubernetes.html
curl -L -O https://raw.githubusercontent.com/elastic/beats/7.11/deploy/kubernetes/filebeat-kubernetes.yaml

Use the autodiscover config by doing the following:
# To enable hints based autodiscover, remove filebeat.inputs configuration and uncomment this:

Filebeat starts up and is unable to list nodes:

E0215 03:45:32.109053 7 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes "k3s-01a.spahr.dev" is forbidden: User "system:serviceaccount:bourbontracker:filebeat" cannot list resource "nodes" in API group "" at the cluster scope
E0215 03:45:33.243209 7 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes "k3s-01a.spahr.dev" is forbidden: User "system:serviceaccount:bourbontracker:filebeat" cannot list resource "nodes" in API group "" at the cluster scope

This error goes away after adding nodes to the ClusterRole.

Related issues

Use cases

Screenshots

Logs

Logs without the change:

E0215 03:45:32.109053 7 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes "k3s-01a.spahr.dev" is forbidden: User "system:serviceaccount:bourbontracker:filebeat" cannot list resource "nodes" in API group "" at the cluster scope
E0215 03:45:33.243209 7 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes "k3s-01a.spahr.dev" is forbidden: User "system:serviceaccount:bourbontracker:filebeat" cannot list resource "nodes" in API group "" at the cluster scope

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 16, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Feb 16, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: jsoriano commented: /test

  • Start Time: 2021-02-16T19:18:04.139+0000

  • Duration: 60 min 40 sec

  • Commit: 6e57685

Trends 🧪

Image of Build Times

❕ Flaky test report

No test was executed to be analysed.

@andresrc andresrc added the Team:Integrations Label for the Integrations team label Feb 16, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 16, 2021
@ChrsMark ChrsMark self-assigned this Feb 16, 2021
@ChrsMark ChrsMark self-requested a review February 16, 2021 09:00
@ChrsMark
Copy link
Member

Hey! Thanks for working on this! You will need to update https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat/filebeat-role.yaml too in order to have a happy/green CI.

@jeffspahr
Copy link
Contributor Author

Thanks for catching that @ChrsMark! I just added that in a new commit.

@jsoriano
Copy link
Member

/test

Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! Thanks for fixing this @jeffspahr!

@ChrsMark ChrsMark merged commit 410d900 into elastic:master Feb 17, 2021
ChrsMark pushed a commit to ChrsMark/beats that referenced this pull request Feb 17, 2021
ChrsMark pushed a commit to ChrsMark/beats that referenced this pull request Feb 17, 2021
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
…-arm

* upstream/master:
  [CI] install docker-compose with retry (elastic#24069)
  Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052)
  updating manifest files for filebeat threatintel module (elastic#24074)
  Add Zeek Signatures (elastic#23772)
  Update Beats to ECS 1.8.0 (elastic#23465)
  Support running Docker logging plugin on ARM64 (elastic#24034)
  Fix ec2 metricset fields.yml and add integration test (elastic#23726)
  Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060)
  [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773)
  [Elastic Agent] Enroll with Fleet Server (elastic#23865)
  [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944)
  [Ingest Management] Fix reloading of log level for services (elastic#24055)
  Add Agent standalone k8s manifest (elastic#23679)
ChrsMark added a commit that referenced this pull request Feb 17, 2021
) (#24080)

(cherry picked from commit 410d900)

Co-authored-by: Jeff Spahr <spahrj@gmail.com>
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
…dows-7

* upstream/master: (332 commits)
  Use ECS v1.8.0 (elastic#24086)
  Add support for postgresql csv logs (elastic#23334)
  [Heartbeat] Refactor config system (elastic#23467)
  [CI] install docker-compose with retry (elastic#24069)
  Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052)
  updating manifest files for filebeat threatintel module (elastic#24074)
  Add Zeek Signatures (elastic#23772)
  Update Beats to ECS 1.8.0 (elastic#23465)
  Support running Docker logging plugin on ARM64 (elastic#24034)
  Fix ec2 metricset fields.yml and add integration test (elastic#23726)
  Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060)
  [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773)
  [Elastic Agent] Enroll with Fleet Server (elastic#23865)
  [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944)
  [Ingest Management] Fix reloading of log level for services (elastic#24055)
  Add Agent standalone k8s manifest (elastic#23679)
  [Metricbeat][Kubernetes] Extend state_node with more conditions (elastic#23905)
  [CI] googleStorageUploadExt step (elastic#24048)
  Check fields are documented for aws metricsets (elastic#23887)
  Update go-concert to 0.1.0 (elastic#23770)
  ...
ChrsMark added a commit that referenced this pull request Feb 18, 2021
@maggieghamry
Copy link

@ChrsMark @jeffspahr is there a similar PR to fix this same issue in 7.11 for Metricbeat as well? (Same issue is occurring there)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Integrations Label for the Integrations team v7.11.1 v7.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Filebeat kubernetes manifest is missing nodes in the ClusterRole
6 participants