packages/aws/data_stream/lambda: Add Async metrics

[agithomas]

Proposed commit message

• Add async event age and async event drops metrics.
• Add sum aggregation metrics for Throttle, Errors, DeadLetterErrors, DestinationDeliveryFailures metrics.
• Update Error dashboard with sum aggregation metrics
• Update dashboard with async event age and async event drop metrics

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• elastic-package build
• elastic-package stack up -v -d --services package-registry
• Dashboard data verification
• Package upgrade verification
• Index mapping verification

How to test this PR locally

• elastic-package build
• elastic-package stack up -v -d --services package-registry

Screenshots

[elasticmachine]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[efd6]

nits only

[aliabbas-elastic]

Some minor nits and dashboard changes requested. Dashboard changes are not specific with the changes in this PR but they are kind of minor improvements which can be done alongside

[shmsr]

Please check all avg. Error average fields will have "per minute" in the definition as the aggregates are done on one-minute intervals.

Please check unchanged avg fields if they have the average mentioned in their description.

[shmsr]

Suggested change

	description: The number of events that Lambda successfully queues for processing.
	description: The number of asynchronous events that Lambda successfully adds to the function event queue.

[aliabbas-elastic]

One more thing came to my mind seeing the updated screenshot. For visualizations Async Event Wait and Async Events Received their @timestamp interval is set to Auto which is correct and the timestamp per x mins/secs/hours would change dynamically as per the duration selected above. However for Top Errors and Lambda Function Duration same is not reflected. Can we check there if something else is set in there?

[agithomas]

Please check all avg. Error average fields will have "per minute" in the definition as the aggregates are done on one-minute intervals.

This is not true. The blog was merely mentioning on how the in-built cloud watch dashboard is built on.

In elastic integration, the user has the flexibility to choose the Period and aggregations occur based on the configured Period value.

[shmsr]

Please check all avg. Error average fields will have "per minute" in the definition as the aggregates are done on one-minute intervals.

This is not true. The blog was merely mentioning on how the in-built cloud watch dashboard is built on.

In elastic integration, the user has the flexibility to choose the Period and aggregations occur based on the configured Period value.

Got it. Then let's not add the per minute change I suggested. Or we can replace "per minute" to "per configured period". Is it by default 1m?

[agithomas]

Got it. Then let's not add the per minute change I suggested. Or we can replace "per minute" to "per configured period". Is it by default 1m?

The period settings is applicable for all the fields in this integration. Specifically calling out for this field alone may not be needed, i think

[aliabbas-elastic]

LGTM

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 31acd62

History

• 💚 Build #11128 succeeded a4f9f49
• 💚 Build #11108 succeeded 6a6c71a
• 💚 Build #11104 succeeded a104ed4

cc @agithomas

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

Package aws - 2.15.0 containing this change is available at https://epr.elastic.co/search?package=aws