Export evaluated DMN decisions/elements to Prometheus/Grafana

[saig0]

Description

With Zeebe 1.4.0, we support the evaluation of DMNs from business rule tasks. We want to track the usage of DMN by exporting the evaluated DMN decisions/elements as metrics to Prometheus/Grafana. The metric should be similar to the process instances metric zeebe.executed_instances_total.

There is related issue for zeebe-analytics: https://github.com/camunda-cloud/zeebe-analytics/issues/22

• create a new metric: zeebe.evaluate_dmn_elements_total
  • with labels: organizationId, action, partition
  • with action as one of: evaluated_successfully, evaluated_failed
  • as metric type counter
• increase the metric by one for each DMN element that is evaluated
  • if the called decision has no required decisions then the metric is increased by one
  • if the called decision has required decisions then the metric is increased by one and for each required decision recursively
  • the metric is increased by one independent of the type of decision (i.e. decision table = +1, decision literal expression = +1)
• increase the metric with the label action: evaluated_successfully if the evaluation of the called DMN was successful, otherwise use the label action: evaluated_failed

blocked by #8115

[menski]

let's only count the decision instances that were evaluated successfully and not the ones that are failed. this way we have it aligned with c7. does this work for you?

From a product perspective we only need to count successful evaluations for the decision value metrics, but I think it's nice to still have both counted as described in the issue. Console will then only use the successful executions for the value metric.