You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently PluginMetrics publishes metrics with prefix <pipeline-name>.<plugin-name>.<defined-by-plugins> which adds a restriction to publish any metrics from Data Prepper core peer forwarding or is some cases conditional routing router.
Describe the solution you'd like
Make the pipeline-name component more generic, it can be pipeline name / component-scope. Something like <generic-name>.<component-id>.<defined-metric-name>. So for metrics from core package we could prefix it by <component-scope> instead of a <pipeline-name>.
Examples:
log-pipeline.buffer.numberOfRecords
log-pipeline.grok.numberOfRecords
log-pipeline.opensearch.numberOfRecords
log-pipeline.router.defined-metric-name
core.peer-forwarder.defined-metric-name
Pros:
No need to change any existing code in PluginMetrics class.
Cons:
If pipeline name is core all the metrics will be prefixed with core.
Describe alternatives you've considered (Optional)
An alternative could be <pipeline-name>.<component-scope>.<component-id>.<defined-metric-name>.
Examples:
What are all the metrics for core peer forwarding we plan on supporting/providing to users? Will there be client side metrics we will want to emit for different aggregate processors? This may help with trying to define the syntax as this changes the requirements.
Can you elaborate on the solution analysis? What are the cons to the first solution and pros to the alternative approach?
I like the solution you proposed. One con is that we could have metric name conflicts if users name their pipeline core. I'd suggest that we disallow pipelines with the names core or data-prepper. In this way, we reserve this metric prefix.
An alternative is to prefix all pipeline metrics with pipeline.<pipeline-name>. I'm not in favor of this because most metrics are this way and it becomes very verbose. Thus, I like the reservation approach personally.
Is your feature request related to a problem? Please describe.
Currently
PluginMetrics
publishes metrics with prefix<pipeline-name>.<plugin-name>.<defined-by-plugins>
which adds a restriction to publish any metrics from Data Prepper core peer forwarding or is some cases conditional routing router.Describe the solution you'd like
Make the pipeline-name component more generic, it can be pipeline name / component-scope. Something like
<generic-name>.<component-id>.<defined-metric-name>
. So for metrics from core package we could prefix it by<component-scope>
instead of a<pipeline-name>
.Examples:
Pros:
PluginMetrics
class.Cons:
core
all the metrics will be prefixed withcore
.Describe alternatives you've considered (Optional)
An alternative could be
<pipeline-name>.<component-scope>.<component-id>.<defined-metric-name>
.Examples:
Cons for alternative approach:
Additional context
data-prepper/data-prepper-api/src/main/java/com/amazon/dataprepper/metrics/PluginMetrics.java
Lines 32 to 35 in e3b8a82
The text was updated successfully, but these errors were encountered: