[AWS] Add API Gateway metrics data stream #6188

lucian-ioan · 2023-05-12T14:54:07Z

What does this PR do?

Adds API Gateway metrics datastream which covers HTTP, REST and Websocket APIs.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author checklist

Add unit metric types
Add docs

Related issues

Concerns #6058
Closes #6270

elasticmachine · 2023-05-12T15:05:34Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-06-05T14:21:24.786+0000
Duration: 54 min 57 sec

Test stats 🧪

Test	Results
Failed	0
Passed	190
Skipped	4
Total	194

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2023-05-14T23:01:54Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`15/15`)	💚
Files	93.75% (`15/16`)	👍
Classes	93.75% (`15/16`)	👍
Methods	85.921% (`238/277`)	👎 -0.211
Lines	85.925% (`7387/8597`)	👍
Conditionals	100.0% (`0/0`)	💚

…metrics

shmsr

Left some comments.

packages/aws/data_stream/apigateway_metrics/agent/stream/stream.yml.hbs

packages/aws/data_stream/apigateway_metrics/fields/fields.yml

packages/aws/data_stream/apigateway_metrics/fields/package-fields.yml

packages/aws/data_stream/apigateway_metrics/sample_event.json

packages/aws/manifest.yml

lucian-ioan · 2023-05-16T08:38:43Z

@shmsr I was also confused by this for the longest time, not sure why AWS chose this naming scheme.

Under https://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html, there are three "Working with..." for the supported APIs (HTTP, REST, WebSockets), each with variations in metrics/dimensions under Monitor/Metrics tab (ex: 4XXError for REST vs 4xx for HTTP).

There are also some mentions of 4XXError metrics:
https://github.com/aws/aws-cdk/blob/b885ece7285ebe8219dbe4d62ae80554bd657632/packages/aws-cdk-lib/aws-apigateway/test/method.test.ts#L947

Snippet of event I got in elasticsearch from REST:

          "aws": {
            "apigateway": {
              "metrics": {
                "4XXError": {
                  "sum": 0
                },
                "Count": {
                  "sum": 6
                },
                "5XXError": {
                  "sum": 0
                },
                "IntegrationLatency": {
                  "avg": 2.6
                },
                "Latency": {
                  "avg": 4.5
                }
              }
            },
            "cloudwatch": {
              "namespace": "AWS/ApiGateway"
            },
            "dimensions": {
              "ApiName": "PetStore",
              "Stage": "prod"
            }
          }

shmsr · 2023-05-16T08:55:36Z

Oh, okay. Thanks for sharing!

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

shmsr

Looks good to me.

But as I'm still not much experienced with integrations as of now, I'd like to request you get another pair of eyes to review this PR. I might have missed something.

packages/aws/_dev/build/docs/apigateway.md

packages/aws/docs/apigateway.md

packages/aws/_dev/build/docs/apigateway.md

packages/aws/docs/apigateway.md

packages/aws/manifest.yml

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

gpop63 · 2023-05-31T20:59:07Z

LGTM, I'll take a look at the documentation.

zmoog

Please considers adding the latency doc from the other PR.

LGTM.

packages/aws/data_stream/apigateway_metrics/fields/fields.yml

packages/aws/data_stream/apigateway_metrics/manifest.yml

Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co>

…metrics

elasticmachine · 2023-06-05T15:49:25Z

Package aws - 1.39.0 containing this change is available at https://epr.elastic.co/search?package=aws

* Add sample event * Add changelog entry * Add stream * Add manifest * Add fields * Add manifest entry * Fix fields * Add icon * Update descriptions * Update package-fields description Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com> * Add unit and metric types * Add docs * Update packages/aws/_dev/build/docs/apigateway.md Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com> * Update packages/aws/docs/apigateway.md Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com> * Update packages/aws/manifest.yml Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com> * Update packages/aws/docs/apigateway.md Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com> * Update packages/aws/manifest.yml Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com> * Update packages/aws/_dev/build/docs/apigateway.md Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com> * Update packages/aws/data_stream/apigateway_metrics/manifest.yml Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co> * fix apigateway docs --------- Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com> Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co>

Add sample event

c134653

lucian-ioan changed the title ~~Add API Gateway~~ [AWS] Add API Gateway metrics data stream May 12, 2023

Lucian added 5 commits May 15, 2023 00:18

Add changelog entry

77400c6

Add stream

6f2115e

Add manifest

42cf949

Add fields

585d39d

Add manifest entry

b23aa9d

lucian-ioan self-assigned this May 15, 2023

Lucian added 2 commits May 15, 2023 16:54

Fix fields

8bf1148

Merge remote-tracking branch 'upstream/main' into add_aws-apigateway_…

fefed56

…metrics

lucian-ioan marked this pull request as ready for review May 15, 2023 14:05

lucian-ioan requested a review from a team as a code owner May 15, 2023 14:05

Add icon

ca29ca1

shmsr requested changes May 16, 2023

View reviewed changes

Update descriptions

baca46b

Update package-fields description

41bddcc

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

lucian-ioan added the 8.9 candidate label May 21, 2023

Add unit and metric types

51571d1

lucian-ioan force-pushed the add_aws-apigateway_metrics branch from ecbca8a to 51571d1 Compare May 29, 2023 23:33

lucian-ioan mentioned this pull request May 30, 2023

AWS API Gateway Integration #6058

Closed

8 tasks

Add docs

5600cb1

lucian-ioan requested a review from shmsr May 31, 2023 04:56

shmsr requested changes May 31, 2023

View reviewed changes

packages/aws/_dev/build/docs/apigateway.md Outdated Show resolved Hide resolved

packages/aws/docs/apigateway.md Outdated Show resolved Hide resolved

packages/aws/_dev/build/docs/apigateway.md Outdated Show resolved Hide resolved

shmsr reviewed May 31, 2023

View reviewed changes

packages/aws/docs/apigateway.md Show resolved Hide resolved

packages/aws/manifest.yml Outdated Show resolved Hide resolved

packages/aws/manifest.yml Outdated Show resolved Hide resolved

lucian-ioan and others added 2 commits May 31, 2023 09:29

Update packages/aws/_dev/build/docs/apigateway.md

552cf26

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

Update packages/aws/docs/apigateway.md

ecc40af

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

lucian-ioan and others added 4 commits May 31, 2023 09:30

Update packages/aws/manifest.yml

0f90421

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

Update packages/aws/docs/apigateway.md

7d03282

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

Update packages/aws/manifest.yml

b867a42

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

Update packages/aws/_dev/build/docs/apigateway.md

18855ca

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

lucian-ioan requested a review from gpop63 May 31, 2023 06:31

zmoog approved these changes Jun 5, 2023

View reviewed changes

packages/aws/data_stream/apigateway_metrics/fields/fields.yml Show resolved Hide resolved

packages/aws/data_stream/apigateway_metrics/manifest.yml Show resolved Hide resolved

Update packages/aws/data_stream/apigateway_metrics/manifest.yml

9bd25c0

Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co>

gpop63 approved these changes Jun 5, 2023

View reviewed changes

shmsr approved these changes Jun 5, 2023

View reviewed changes

lucianpy and others added 4 commits June 5, 2023 14:36

Merge remote-tracking branch 'upstream/main' into add_aws-apigateway_…

587c085

…metrics

fix apigateway docs

4bf2620

Merge branch 'main' into add_aws-apigateway_metrics

321d0aa

Merge branch 'main' into add_aws-apigateway_metrics

2793d6d

lucian-ioan merged commit e465e79 into elastic:main Jun 5, 2023
3 checks passed

lucian-ioan linked an issue Jul 24, 2023 that may be closed by this pull request

AWS API Gateway Integration #6058

Closed

8 tasks

lucian-ioan deleted the add_aws-apigateway_metrics branch September 6, 2023 11:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AWS] Add API Gateway metrics data stream #6188

[AWS] Add API Gateway metrics data stream #6188

lucian-ioan commented May 12, 2023 •

edited

Loading

elasticmachine commented May 12, 2023 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented May 14, 2023 •

edited

Loading

shmsr left a comment

lucian-ioan commented May 16, 2023 •

edited

Loading

shmsr commented May 16, 2023 •

edited

Loading

shmsr left a comment

gpop63 commented May 31, 2023

zmoog left a comment

elasticmachine commented Jun 5, 2023

[AWS] Add API Gateway metrics data stream #6188

[AWS] Add API Gateway metrics data stream #6188

Conversation

lucian-ioan commented May 12, 2023 • edited Loading

What does this PR do?

Checklist

Author checklist

Related issues

elasticmachine commented May 12, 2023 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

elasticmachine commented May 14, 2023 • edited Loading

🌐 Coverage report

shmsr left a comment

Choose a reason for hiding this comment

lucian-ioan commented May 16, 2023 • edited Loading

shmsr commented May 16, 2023 • edited Loading

shmsr left a comment

Choose a reason for hiding this comment

gpop63 commented May 31, 2023

zmoog left a comment

Choose a reason for hiding this comment

elasticmachine commented Jun 5, 2023

lucian-ioan commented May 12, 2023 •

edited

Loading

elasticmachine commented May 12, 2023 •

edited

Loading

elasticmachine commented May 14, 2023 •

edited

Loading

lucian-ioan commented May 16, 2023 •

edited

Loading

shmsr commented May 16, 2023 •

edited

Loading