feat(metrics): introduce Metrics.flushMetrics #2154

humanzz · 2025-09-29T13:01:44Z

Summary

Changes

Please provide a summary of what's being changed

Please add the issue number below, if no issue is present the PR might get blocked and not be reviewed

introduce Metrics.flushMetrics as a more powerful version of flushSingleMetrics to allow
- using defaults by inheriting state e.g. namespace, dimensions and properties
- emitting multiple metrics in one metrics context
update EmfMetricsLogger.addMetadata to use emf's putProperty
refactor flushSingleMetrics and captureColdStartMetric to use flushMetrics
consolidate request id and trace id setting into a newly introduced internal MetricsUtils

Issue number: #2153

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

- introduce `Metrics.flushMetrics` as a more powerful version of `flushSingleMetrics` to allow - using defaults by inheriting state e.g. namespace, dimensions and metadata - emitting multiple metrics in one metrics context - refactor `flushSingleMetrics` to use `flushMetrics` - move namespace/service setting from `MetricsFactory` to `EmfMetricsLogger`

…-powertools-java into flush-metrics

phipag

Thanks @humanzz for sending this PR. Overall, it looks good and I left some comments that we need to address before moving forward.

Primarily, can you go back to the issue #2153 and let us know your use-case? We still need to decide whether we want to move forward with inheriting the default configuration of the metrics logger for flushMetrics and flushSingleMetric.

powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/MetricsFactory.java

...trics/src/main/java/software/amazon/lambda/powertools/metrics/internal/EmfMetricsLogger.java

spotbugs-exclude.xml

humanzz · 2025-09-30T10:08:13Z

I've addressed the metricsContext comments in 4730be4, and will leave details about the use case on the feature request.

phipag

Thanks for the quick turnaround @humanzz. Just one more thing that I discovered.

...trics/src/main/java/software/amazon/lambda/powertools/metrics/internal/EmfMetricsLogger.java

...s/src/test/java/software/amazon/lambda/powertools/metrics/internal/EmfMetricsLoggerTest.java

humanzz · 2025-09-30T12:37:06Z

@phipag another observation/inconsistency is the usage of putProperty vs. putMetadata. See

phipag · 2025-09-30T13:29:51Z

@phipag another observation/inconsistency is the usage of putProperty vs. putMetadata. See

* https://github.com/aws-powertools/powertools-lambda-java/blob/main/powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/internal/EmfMetricsLogger.java#L212

* https://github.com/aws-powertools/powertools-lambda-java/blob/main/powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/internal/LambdaMetricsAspect.java#L95

Good catch! It seems that a "property" is simply a key added to the log line while "metadata" becomes part of the _aws object. These are logs from the official example:

Cold start metric (putProperty)

{
    "_aws": {
        "Timestamp": 1751977621635,
        "CloudWatchMetrics": [
            {
                "Namespace": "ServerlessAirline",
                "Metrics": [
                    {
                        "Name": "ColdStart",
                        "Unit": "Count"
                    }
                ],
                "Dimensions": [
                    [
                        "Service",
                        "FunctionName"
                    ]
                ]
            }
        ]
    },
    "function_request_id": "b8628984-b745-44bb-8025-247c2da76da1",
    "xray_trace_id": "1-686d0e92-396456d519ba8c28174120a4",
    "ColdStart": 1,
    "Service": "payment",
}

Regular metric (putMetadata)

{
    "_aws": {
        "Timestamp": 1751977946689,
        "CloudWatchMetrics": [
            {
                "Namespace": "ServerlessAirline",
                "Metrics": [
                    {
                        "Name": "CustomMetric1",
                        "Unit": "Count"
                    },
                    {
                        "Name": "CustomMetric3",
                        "Unit": "Count",
                        "StorageResolution": 1
                    }
                ],
                "Dimensions": [
                    [
                        "Service"
                    ]
                ]
            }
        ],
        "function_request_id": "feccb848-47a9-4b32-a16b-73d45d7ad308",
        "xray_trace_id": "1-686d0fdb-651f4fc05b57221e726890c0"
    },
    "CustomMetric1": 1,
    "Service": "payment",
    "CustomMetric3": 1
}

I need to find out the semantic difference – those nuances don't exist in the other runtimes. None of it is searchable in cloudwatch metrics. Besides the location in the json output I don't see any difference.

Update:

It appears that putMetadata will add key-value pairs to the _aws (metadata) object. Adding things there will not have an effect. Only Timestamp which is the officially documented metadata key in addition to CloudWatchMetrics (https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html#CloudWatch_Embedded_Metric_Format_Specification_structure_metadata%22%3ECloudWatch%20%20%20%20%20*%20%20%20%20%20Metadata). We shouldn't use this since it is nowhere documented and favor putProperty: https://github.com/awslabs/aws-embedded-metrics-java

We should update the addMetadata method to call putProperty on the EMF logger since this is the officially documented method (putMetadata is not documented). This is consistent with e.g. Python (see https://docs.powertools.aws.dev/lambda/python/latest/core/metrics/#add_metadata_outputjson)

...trics/src/main/java/software/amazon/lambda/powertools/metrics/internal/EmfMetricsLogger.java

sonarqubecloud · 2025-09-30T22:55:12Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

phipag

Thanks @humanzz for your continued engagement. Everything looks good so far besides these minor comments. 🚀

The final step after this will be to update the documentation with the new flushMetrics method including code examples showing developers how to use it. Let me know if you would like to do this – otherwise I am happy to commit the documentation into this PR.

The docs file uses mkdocs and is located here docs/core/metrics.md.

phipag · 2025-10-01T08:35:07Z

...trics/src/main/java/software/amazon/lambda/powertools/metrics/internal/EmfMetricsLogger.java

-            singleMetricLogger.setNamespace(namespace);
-        } catch (Exception e) {
-            LOGGER.error("Namespace cannot be set for single metric due to an error in EMF", e);
+        // Create a new instance, inheriting namespace/dimensions state


Suggested change

// Create a new instance, inheriting namespace/dimensions state

// Create a new instance, inheriting namespace, default dimensions, and metadata

phipag · 2025-10-01T08:35:35Z

...cs/src/main/java/software/amazon/lambda/powertools/metrics/internal/LambdaMetricsAspect.java

-            LambdaHandlerProcessor.getXrayTraceId()
-                    .ifPresent(traceId -> metricsInstance.addMetadata(TRACE_ID_PROPERTY, traceId));
+            Context extractedContext = extractContext(pjp);
+            MetricsUtils.addRequestIdAndXrayTraceIdIfAvailable(extractedContext, metricsInstance);


Good catch, the request id was missing here.

phipag · 2025-10-01T08:40:01Z

...s-metrics/src/main/java/software/amazon/lambda/powertools/metrics/internal/MetricsUtils.java

+
+import static software.amazon.lambda.powertools.common.internal.LambdaHandlerProcessor.getXrayTraceId;
+
+class MetricsUtils {


Can you make this class final and add a private constructor such as:

private MetricsUtils { // Utility class }

?

See Sonarlint finding and pmd analysis.

For classes that only have static methods, consider making them utility classes. Note that this doesn't apply to abstract classes, since their subclasses may well include non-static methods. Also, if you want this class to be a utility class, remember to add a private constructor to prevent instantiation. (Note, that this use was known before PMD 5.1.0 as UseSingleton). UseUtilityClass (Priority: 1, Ruleset: Design) https://docs.pmd-code.org/snapshot/pmd_rules_java_design.html#useutilityclass

phipag · 2025-10-01T08:41:40Z

powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/Metrics.java

+     * Flush a separate metrics context that inherits the namespace, dimensions and metadata. This creates a separate metrics context
+     * that doesn't affect the default metrics context.


Suggested change

* Flush a separate metrics context that inherits the namespace, dimensions and metadata. This creates a separate metrics context

* that doesn't affect the default metrics context.

* Flush a separate metrics context that inherits the namespace, default dimensions, and metadata. This creates a separate metrics context

* that doesn't affect the default metrics context.

phipag · 2025-10-01T08:42:21Z

powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/Metrics.java

+     * Flush a separate metrics context that inherits the namespace, dimensions and metadata. This creates a separate metrics context
+     * that doesn't affect the default metrics context.
+     *
+     * @param metricsConsumer the consumer to use to edit the metrics instance (e.g. add metrics, override namespace) before flushing


Suggested change

* @param metricsConsumer the consumer to use to edit the metrics instance (e.g. add metrics, override namespace) before flushing

* @param metricsConsumer the consumer to use to edit the metrics instance (e.g. add metrics, override namespace, add custom dimensions) before flushing

phipag · 2025-10-01T08:45:00Z

powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/Metrics.java

+            metrics.setNamespace(namespace);
+            metrics.addMetric(name, value, unit);
+        });
    }


Following the discussion in the issue: All runtimes inherit the namespace by default if it is not overwritten. Let's add the method overload you suggested in your initial feature request here (without namespace param). We should also add a unit test for this.

default void flushSingleMetric(String name, double value, MetricUnit unit) { flushSingleMetric(name, value, unit, namespace, null); flushMetrics(metrics -> { metrics.addMetric(name, value, unit); });

While this might make sense, a couple of thoughts

Not sure what value this would provide over Metrics.addMetric (I understand it's a different flush, but end result is that the same metric is emitted, reusing namespace, dimensions, metadata)

I sincerely believe that all flushSingleMetric should be marked as deprecated, to help reduce the api surface area as flushMetrics would already enable all that's supported by these overloads

pull-request-size bot added the size/L label Sep 29, 2025

humanzz mentioned this pull request Sep 29, 2025

Feature request: Reuse namespace in Metrics.flushSingleMetric and other limitations #2153

Open

2 tasks

humanzz force-pushed the flush-metrics branch from 6587c5a to 13eb9d1 Compare September 29, 2025 15:20

humanzz force-pushed the flush-metrics branch from 13eb9d1 to 537dc90 Compare September 29, 2025 22:15

Merge branch 'flush-metrics' of https://github.com/humanzz/aws-lambda…

240b761

…-powertools-java into flush-metrics

phipag requested changes Sep 30, 2025

View reviewed changes

address metrics context issue

4730be4

phipag requested changes Sep 30, 2025

View reviewed changes

fix addDimension wrongly adding to defaultDimensions

a97906c

phipag reviewed Sep 30, 2025

View reviewed changes

...trics/src/main/java/software/amazon/lambda/powertools/metrics/internal/EmfMetricsLogger.java Outdated Show resolved Hide resolved

...trics/src/main/java/software/amazon/lambda/powertools/metrics/internal/EmfMetricsLogger.java Outdated Show resolved Hide resolved

humanzz added 4 commits September 30, 2025 15:36

introduce Metrics.addProperty and use it instead of addMetadata

3738d62

use flushMetrics in captureColdStartMetric

f1f3e70

Merge branch 'main' into flush-metrics

616a46e

use putProperty for addMetadata and consolidate duplicate logic

13eff8f

phipag reviewed Oct 1, 2025

View reviewed changes

	// Create a new instance, inheriting namespace/dimensions state
	// Create a new instance, inheriting namespace, default dimensions, and metadata


		import static software.amazon.lambda.powertools.common.internal.LambdaHandlerProcessor.getXrayTraceId;

		class MetricsUtils {

		* Flush a separate metrics context that inherits the namespace, dimensions and metadata. This creates a separate metrics context
		* that doesn't affect the default metrics context.

	* @param metricsConsumer the consumer to use to edit the metrics instance (e.g. add metrics, override namespace) before flushing
	* @param metricsConsumer the consumer to use to edit the metrics instance (e.g. add metrics, override namespace, add custom dimensions) before flushing

feat(metrics): introduce Metrics.flushMetrics #2154

Are you sure you want to change the base?

feat(metrics): introduce Metrics.flushMetrics #2154

Conversation

humanzz commented Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Uh oh!

phipag left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

humanzz commented Sep 30, 2025

Uh oh!

phipag left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

humanzz commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phipag commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud bot commented Sep 30, 2025

Quality Gate passed

Uh oh!

phipag left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

humanzz commented Sep 29, 2025 •

edited

Loading

humanzz commented Sep 30, 2025 •

edited

Loading

phipag commented Sep 30, 2025 •

edited

Loading