Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat] Remove checking region/zone and service name from googlecloud #18398

Merged
merged 4 commits into from
May 11, 2020
Merged

[Metricbeat] Remove checking region/zone and service name from googlecloud #18398

merged 4 commits into from
May 11, 2020

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented May 8, 2020

What does this PR do?

This PR is to remove validations for region/zone/service name in order for stackdriver metricset to work for wider services.

Why is it important?

service is only used to check when adding metadata and this is only available for compute metricset right now.

region/zone is not required for making ListTimeSeriesRequest API.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

  1. Enable googlecloud in Metricbeat
    ./metricbeat modules enable googlecloud

  2. Modify modules.d/googlecloud.yml to use stackdriver metricset to collect logging metrics:

- module: googlecloud
  metricsets:
    - stackdriver
  project_id: elastic-observability
  credentials_file_path: "/Users/kaiyansheng/Downloads/elastic-observability-d17781618202.json"
  exclude_labels: false
  period: 1m
  stackdriver:
    service: billing
    metrics:
      - aligner: ALIGN_NONE
        metric_types:
          - "logging.googleapis.com/billing/bytes_ingested"
          - "logging.googleapis.com/byte_count"
  1. Start metricbeat:
./metricbeat -e
  1. You should see metrics in Kibana like:
    googlecloud.stackdriver.byte_count.value and googlecloud.stackdriver.billing.bytes_ingested.value

  2. Change modules.d/googlecloud.yml to collect compute metrics and also should work:

- module: googlecloud
  metricsets:
    - stackdriver
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m
  period: 1m
  stackdriver:
    service: compute
    metrics:
      - aligner: ALIGN_NONE
        metric_types:
          - "compute.googleapis.com/instance/cpu/reserved_cores"
          - "compute.googleapis.com/instance/cpu/usage_time"
          - "compute.googleapis.com/instance/cpu/utilization"
          - "compute.googleapis.com/instance/uptime"

Related issues

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 8, 2020
@kaiyan-sheng kaiyan-sheng self-assigned this May 8, 2020
@kaiyan-sheng kaiyan-sheng added needs_backport PR is waiting to be backported to other branches. review Team:Platforms Label for the Integrations - Platforms team test-plan Add this PR to be manual test plan labels May 8, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@elasticmachine
Copy link
Collaborator

elasticmachine commented May 8, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview stats

Expand to view the summary

Build stats

Test stats 🧪

Test Results
Failed 0
Passed 3022
Skipped 756
Total 3778

@andresrc andresrc added [zube]: Inbox [zube]: In Review and removed needs_team Indicates that the issue/PR needs a Team:* label [zube]: Inbox labels May 9, 2020
Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Once comment about the nil check you added.

@kaiyan-sheng kaiyan-sheng merged commit 2d7c9d5 into elastic:master May 11, 2020
@kaiyan-sheng kaiyan-sheng deleted the stackdriver_region branch May 11, 2020 22:57
@kaiyan-sheng kaiyan-sheng added v7.9.0 and removed needs_backport PR is waiting to be backported to other branches. labels May 11, 2020
kaiyan-sheng added a commit that referenced this pull request May 12, 2020
…nd service name from googlecloud (#18434)

* [Metricbeat] Remove checking region/zone and service name from googlecloud (#18398)

* remove validate region/zone

* add stackdriver into googlecloud default config

(cherry picked from commit 2d7c9d5)

* add changelog
kaiyan-sheng added a commit that referenced this pull request May 12, 2020
…nd service name from googlecloud (#18433)

* [Metricbeat] Remove checking region/zone and service name from googlecloud (#18398)

* remove validate region/zone

* add stackdriver into googlecloud default config

(cherry picked from commit 2d7c9d5)

* add changelog
@andresrc andresrc added the test-plan-added This PR has been added to the test plan label Jul 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
review Team:Platforms Label for the Integrations - Platforms team test-plan Add this PR to be manual test plan test-plan-added This PR has been added to the test plan v7.8.0 v7.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Metricbeat] Add stackdriver metricset for googlecloud module
4 participants