Adding per-worker metrics reported to JMX in applicationinsights.json for App Insights

[MonikaReddy-MSFT]

Customer opened a question on Microsoft Q&A, but opening this issue in Github as suggested by PG.

I have a Flink pipeline and if possible I would like to report the metrics to JMX so that I can use the applicationinsights.json to report them to Application Insights. However, when Flink metrics are reported to JMX they have object names with generated IDs like below:

org.apache.flink.taskmanager.job.task.operator.currentOutputWatermark:host=,job_id=,job_name=Flink_metrics_test,operator_id=,operator_name=Sink-_logger-sink,subtask_index=0,task_attempt_id=,task_attempt_num=0,task_id=,task_name=Source-Custom_Source-time-now-stream-counter-map-_Sink-_logger-sink,tm_id=
Is it still possible to configure them as JMX metrics? Related documentation: https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-jmx-metrics-configuration#configuration-example

[jeanbisutti]

@MonikaReddy-MSFT In the JMX metrics configuration, you can use * to represent a generated ID.

It should be something like that:

{
  "jmxMetrics": [
    {
      "name": "name",
      "objectName": "org.apache.flink.taskmanager.job.task.operator.currentOutputWatermark:host=*,job_id=*,job_name=Flink_metrics_test,operator_id=*,operator_name=Sink-_logger-sink,subtask_index=0,task_attempt_id=<ID>,task_attempt_num=0,task_id=*,task_name=Source-_Custom_Source_-_time-now-stream_-_counter-map_-_Sink-_logger-sink,tm_id=*",
      "attribute": "attribute"
    }
}

[MonikaReddy-MSFT]

@jeanbisutti - Thanks for the response.

Customer responded back saying - The documentation needs updated because it made no mention or example of that. Additionally, are these able to replace values in general (i.e. does it work like Regex pattern matching, so I could use a pattern like "objectName": "*.currentOutputWatermark",). T

[jeanbisutti]

@MonikaReddy-MSFT

See https://docs.oracle.com/javase/8/docs/api/javax/management/ObjectName.html: "The asterisk matches any sequence of zero or more characters".

Microsoft documentation with wildcard: https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-config#java-management-extensions-metrics

[ivanthewebber]

@jeanbisutti that page has some great information; I think a great solution would be to add links in the documentation so that it can be found easier. Here is a PR: MicrosoftDocs/azure-docs#121864 to fix https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-jmx-metrics-configuration#configuration-example