Enabled CloudWatch Embedded Metric Format Logging

[AhmedSedek]

Overview

I needed to have some custom metrics to monitor my Resource Handlers behvaiour to get a better understanding of the development experience with the resources.

Fortunately, Cloudwatch has released an embedded-metrics-format logging feature where you publish logs using a specific format that automatically get exported to metrics and then can be consumed for Dashboards/Alarms too.

How to support it?

By including a new header in the request to PutLogEvents endpoint => ['x-amzn-logs-format'] = 'json/emf'
This triggers CW to extract metrics from the logs that matches the emf

This PR

This PR is a one-liner that adds the support for the embedded-metrics-format simply by adding the request header mentioned above ^

Who will be affected?

This wouldn't break any existing plugin consumer because:

• The header is specific for this feature, so it's backward compatible.
• The format is not trivial to have, it's very specific to CloudWatch metrics.
• Even if someone published the logs somehow on this format, nothing would break as they would just see the metrics in their AWS account and they can totally ignore them, similar to what happens with almost all AWS services metrics.
• It doesn't make sense to log something while not wanting to have metrics.

What value does this change add?

• Custom Monitoring Experience including CW Dashboards and Alarming
• Decreases the number or feature requests you would have for a specific metric as everyone could now implement their own desired metrics :D

Testing

Tested the package manually with my Resource Providers Packages.
Here's a proof it works:

• A paste link to the emitted logs from my providers.
• A screenshot to the generated metrics in my account.

[tobywf]

and they can totally ignore them, similar to what happens with almost all AWS services metrics.

😄

thanks for the PR. i must admit, i don't 100% understand the impact of this change. e.g. from Ingesting High-Cardinality Logs and Generating Metrics with CloudWatch Embedded Metric Format :

Charges are incurred for logs ingestion and archival, and custom metrics that are generated. For more information, see Amazon CloudWatch Pricing

let me dig deeper and follow up on this - it sounds really useful!

[arnulfojr]

@tobywf any updates in this topic?

[tobywf]

I've been communicating with Ahmed offline. I think this is a great change, but as a managed service, we have to be a bit mindful of the pricing implications of this change. Unfortunately, I'm still working on some of these details, as it goes beyond a simple code change.

[tobywf]

Sorry for the delay, works great! Nice first PR 👏

[AhmedSedek]

@tobywf That's great news! Thank you for approving the PR.
Could you please clarify how we can proceed with this PR? It's mentioned that it requires two approvals, do I need to do anything else or you are going to take care of the second approval and(or) merging it?

[tobywf]

Please keep an eye out for a new release. Thanks for getting involved, this feature was really interesting to know about!

[AhmedSedek]

Thank you for accepting the contribution and being on top of it!
I really enjoyed engaging in conversations about the impact and pricing implications.