Extend /status to better serve frontends

[joe-elliott]

Is your feature request related to a problem? Please describe.
A frontend, like say Grafana, may interact with many different versions of Tempo. It would be nice to extend the /status endpoint so that a frontend could dynamically detect the feature set of Tempo and react accordingly.

The status endpoint is "ok" at doing this. You can currently ask for the version:

$ curl http://localhost:3100/status/version
GET /status/version
tempo, version weekly-r93-d460a218 (branch: weekly-r93, revision: d460a218)
  build user:       
  build date:       
  go version:       go1.20.3
  platform:         linux/amd64
  tags:             netgo

And for the config to see what feature flags are enabled:

$ curl http://localhost:3100/status/config
GET /status/config
---
target: querier
multitenancy_enabled: true
http_api_prefix: /tempo
server:
    http_listen_network: tcp
    http_listen_address: ""
    http_listen_port: 3100
    http_listen_conn_limit: 0
...

Neither of these are great, but you can technically use them to determine what feature set Tempo has. Also note that the config is only true for the component that was polled. So if a feature flag were enabled only on an ingester then requesting config from the query frontend would not show this.

Inviting @grafana/observability-traces-and-profiling to comment on what they would want from an extension to the /status endpoint to better detect the feature set of Tempo.

[kvrhdn]

I was thinking of something similar, just two thoughts:

1. Should we bundle public APIs under /api/?

All user-accessible endpoints of Tempo are currently under /api/. We have several endpoints at the root level (/ready, /config, /memberlist,...) but you could argue these are all internal endpoints intended to be used by the operator only.

This /status endpoint will be accessed by Grafana and should be whitelisted if you are running a gateway. As such I'd suggest making it a public endpoint on /api/status.

2. Can we also expose tenant configuration?

This might be slightly tangential but... apps built on top of Tempo will be interested in tenant-specific configuration of Tempo as well. So besides "does Tempo support feature X?", it might also want to know "is feature Y enabled for my user?".

Can we expose this kind of information through a /api/status/ or /api/config endpoint? This would require the request to be sent with authorization headers.
A specific example would be an app consuming span metrics from Tempo, it would be useful for them to know how the metrics-generator processor is configured.

Drawback:

• if we want to expose how the metrics-generator is configured we would need to query the metrics-generator -> each component can have a separate configmap with slightly different settings. So the implementation will be a bit more complicated.

[joey-grafana]

One of the main use cases I can think of right now is for checking if the current version of Tempo supports a particular API. For example, being able to ping the status endpoint to see if the current version of Tempo supports v1 of the tags API /api/search/tags or v2 /api/v2/search/tags.

We could then cache the result in the front end and use it to decide which version of the API to call on subsequent requests. There are other features of course that exist on the front end which are enabled contingent on which version of Tempo the user is running.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days.
The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity.
Please apply keepalive label to exempt this Issue.

[joe-elliott]

We have discussed internally and would like to add a new endpoint at :

/api/status/buildinfo

This endpoint should return the same json object that Loki does:

{"version":"","revision":"","branch":"","buildUser":"","buildDate":"","goVersion":""}

The values returned should be retrieved from here:
https://github.com/grafana/tempo/blob/main/cmd/tempo/build/build.go#L28

Adding "good first issue" and "help wanted" tags to see if there are any takers.

[fabrizio-grafana]

I can have a look at it, since it is needed by the Traces and Profiling squad!

[joe-elliott]

All you @fabrizio-grafana. Please let me know if you have any questions.