Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat] Change sqs metricset to use Average statistic method #16438

Merged
merged 4 commits into from
Feb 20, 2020
Merged

[Metricbeat] Change sqs metricset to use Average statistic method #16438

merged 4 commits into from
Feb 20, 2020

Conversation

kaiyan-sheng
Copy link
Contributor

What does this PR do?

This PR is to change sqs metricset to use Average statistic method instead of Sum when querying CloudWatch metrics.
Related Discuss Forum Issue: https://discuss.elastic.co/t/metricbeat-aws-module-dashboard-not-working-and-aws-sqs-messages-visible-multiplied-by-5

This PR also fixed SQS dashboard to show last bucket with "drop_last_bucket":0. This case, when Metricbeat first started, dashboard will not be empty.

Why is it important?

Most of the metrics collected by CloudWatch regarding to SQS is more meaningful with average as statistic method. For example ApproximateAgeOfOldestMessage for the approximate age of the oldest non-deleted message in the queue, and ApproximateNumberOfMessagesVisible for the number of messages available for retrieval from the queue.

Screenshots

This is what I changed for each visualization in SQS dashboard to not drop the last bucket.
Screen Shot 2020-02-19 at 1 06 33 PM

@kaiyan-sheng kaiyan-sheng requested a review from a team as a code owner February 19, 2020 20:54
@kaiyan-sheng kaiyan-sheng self-assigned this Feb 19, 2020
@kaiyan-sheng kaiyan-sheng added bug needs_backport PR is waiting to be backported to other branches. review Team:Platforms Label for the Integrations - Platforms team labels Feb 19, 2020
Copy link
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I admit that I had to go over the discuss thread to fully understand the issue. Ship it!

nit: fix is correct, but please re-export JSON files:

Error: failed to parse dashboard from module/aws/_meta/kibana/7/dashboard/Metricbeat-aws-sqs-overview.json: json: cannot unmarshal string into Go struct field .objects.attributes.kibanaSavedObjectMeta.searchSourceJSON of type struct { Index string "json:\"index\"" }

@kaiyan-sheng kaiyan-sheng merged commit 6076de6 into elastic:master Feb 20, 2020
@kaiyan-sheng kaiyan-sheng deleted the sqs_stats branch February 20, 2020 22:34
@kaiyan-sheng kaiyan-sheng added v7.7.0 and removed needs_backport PR is waiting to be backported to other branches. labels Feb 20, 2020
kaiyan-sheng added a commit that referenced this pull request Feb 21, 2020
…6438) (#16472)

* Change sqs metricset to use Average statistic method

* re-export dashboard for sqs

* update changelog

* Add terms_field back into sqs visualizations

(cherry picked from commit 6076de6)
kaiyan-sheng added a commit that referenced this pull request Feb 24, 2020
…verage statistic method (#16473)

* [Metricbeat] Change sqs metricset to use Average statistic method (#16438)
(cherry picked from commit 6076de6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug review Team:Platforms Label for the Integrations - Platforms team v7.6.1 v7.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants