Description
When exposing metrics in the OpenMetrics format using content negotiation (via Accept: application/openmetrics-text; version=1.0.0), metrics containing colons (:) have their names escaped in the sample lines but remain unescaped in the # HELP and # TYPE lines.
This produces mismatched exposition output that violates the OpenMetrics standard, causing strict standard parsers (including promtool check metrics and downstream ingesters) to fail or treat them as completely separate metrics (an orphaned metadata set and an untyped metric).
Steps to Reproduce
- Create a simple Python server using
prometheus_client:
from http.server import BaseHTTPRequestHandler, HTTPServer
from prometheus_client import CollectorRegistry, Gauge
from prometheus_client.exposition import choose_encoder
registry = CollectorRegistry()
token_usage = Gauge(
name="sglang:token_usage",
documentation="Total token usage.",
registry=registry,
)
token_usage.set(42.0)
class MetricsHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path == "/metrics":
accept_header = self.headers.get("Accept", "")
encoder, content_type = choose_encoder(accept_header)
data = encoder(registry)
self.send_response(200)
self.send_header("Content-Type", content_type)
self.end_headers()
self.wfile.write(data)
if __name__ == "__main__":
server = HTTPServer(("0.0.0.0", 8000), MetricsHandler)
server.serve_forever()
- Query the endpoint requesting OpenMetrics formatting:
curl -i -H "Accept: application/openmetrics-text; version=1.0.0" http://localhost:8000/metrics
Actual Output
Notice that the Content-Type is negotiated with escaping=underscores, but the comment lines mismatch the sample line:
HTTP/1.0 200 OK
Content-Type: application/openmetrics-text; version=1.0.0; charset=utf-8; escaping=underscores
# HELP sglang:token_usage Total token usage.
# TYPE sglang:token_usage gauge
sglang_token_usage 42.0
# EOF
Expected Output
The metric name inside the # HELP and # TYPE comment blocks should match the name used in the sample line:
# HELP sglang_token_usage Total token usage.
# TYPE sglang_token_usage gauge
sglang_token_usage 42.0
# EOF
Root Cause
In prometheus_client/openmetrics/exposition.py:
- The comment generator uses the unescaped
metric.name directly (since a colon is treated as legacy-valid under default validation checks).
- The sample line generator, however, strictly sanitizes the sample's name against the narrower OpenMetrics name requirements, replacing the colon with an underscore under
escaping=underscores.
Description
When exposing metrics in the OpenMetrics format using content negotiation (via
Accept: application/openmetrics-text; version=1.0.0), metrics containing colons (:) have their names escaped in the sample lines but remain unescaped in the# HELPand# TYPElines.This produces mismatched exposition output that violates the OpenMetrics standard, causing strict standard parsers (including
promtoolcheck metrics and downstream ingesters) to fail or treat them as completely separate metrics (an orphaned metadata set and an untyped metric).Steps to Reproduce
prometheus_client:curl -i -H "Accept: application/openmetrics-text; version=1.0.0" http://localhost:8000/metricsActual Output
Notice that the
Content-Typeis negotiated withescaping=underscores, but the comment lines mismatch the sample line:Expected Output
The metric name inside the
# HELPand# TYPEcomment blocks should match the name used in the sample line:Root Cause
In
prometheus_client/openmetrics/exposition.py:metric.namedirectly (since a colon is treated as legacy-valid under default validation checks).escaping=underscores.