Prometheus metrics like http_request_duration_milliseconds should use path variables

[checketts]

Prometheus metrics currently aren't showing the path variables:

For example I have many metrics like:

http_request_duration_milliseconds{quantile="0.1",path="/api/admin/features/other-feature/toggle/on",method="POST",status="200"} 11.892953
http_request_duration_milliseconds{quantile="0.5",path="/api/admin/features/other-feature/toggle/on",method="POST",status="200"} 11.892953
http_request_duration_milliseconds{quantile="0.9",path="/api/admin/features/other-feature/toggle/on",method="POST",status="200"} 11.892953
http_request_duration_milliseconds{quantile="0.99",path="/api/admin/features/other-feature/toggle/on",method="POST",status="200"} 11.892953
http_request_duration_milliseconds_sum{path="/api/admin/features/other-feature/toggle/on",method="POST",status="200"} 11.892953
http_request_duration_milliseconds_count{path="/api/admin/features/other-feature/toggle/on",method="POST",status="200"} 1

Note the part of the path 'other-feature' should be :featureName so multiple toggles doesn't explode the number of metrics and so that metrics are aggreggated for the same endpoint.

Perhaps using the express-prometheus-middleware lib might be a good idea. (Or we could mimic their approach for path variables (https://github.com/joao-fontenele/express-prometheus-middleware/blob/master/src/index.js#L53)

[ivarconr]

I agree that we should migrate towards path variables to avoid explosion in time-series.

I am not i favor of depending directly on express-prometheus-middleware for two reasons:

1. we might need to support multiple metrics formats in the future
2. express-prometheus-middleware depends directly on "prom-client" and given it being a global you soon end up in version problems (e.g.: fix: upgrade prom-client from 12.0.0 to 13.1.0 #709)

I think however we mimic they way express-prometheus-middleware resolves paths.