requirement to have consistent tag sets causes problems for multi-app app server implementations

[donbourne]

Problem
mp_app, used at app level, causes one of (server's) /metrics and (app's) servlet stats to not register

For example, an HTTP endpoint could apply to both an app's endpoints and to server-level endpoints such as /metrics or /health. If the app uses the mp_app tag on its metrics then, because we require all tag sets to be consistent for a metric name, the server-level metrics must similarly have an mp_app tag. Since the mp_app value is not generally set at a server level, it is not possible to use it for server-level endpoints.

Potential solution:

• provide config setting so users can set a default mp_app to use for all server metrics that overlap with metric names apps will use (eg. REST metrics)
• would want this to be part of the spec (5.0.1), because this problem is due to spec requirement that tag sets for a given metric name be consistent (which was necessary due to prometheus client requirement for same)
• currently we have mp.metrics.appName
• propose we add mp.metrics.defaultAppName
• In cases where a metric name is likely to be used by both applications and the runtime/app server, the value of mp.metrics.defaultAppName will be automatically added to those runtime/app server metrics and will be automatically used for any app that doesn’t specify mp.metrics.appName.