RFC: Optional per-model response usage-footer for chat UI visibility

[walcz-de]

Problem

usage.prompt_tokens / completion_tokens already exist in responses.
But most chat UIs (LibreChat, AnythingLLM, Open-WebUI) don't surface
them — either at all, or only in tooltips. For an operator running
multi-model setups where compression, routing or fallback may kick in,
the user seeing the response doesn't know which model answered,
whether compression fired, or how big the request was.

Our production UX at walcz.de appends a tiny one-line footer under
each response:

Modell: qwen3.6-35b-a3b-apex · Route: tool_passthrough · Tokens: 465 → 465 (−0%, Ratio 1.00×) · Antwort: 39 Tokens

Users like it. It's observability built into the answer, not hidden
in /metrics.

Proposal

Opt-in, per-model middleware that appends a compact footer to the
final assistant content, delimited by an HTML comment so it's
client-strippable:

<original content>

<!-- localai-usage-footer -->
Modell: X · Tokens: 123 → 45 · Antwort: 45 Tokens

Config

usage_footer:
  enabled: true
  fields:                          # default: all
    - model
    - compression_ratio            # only meaningful when compression fired
    - prompt_tokens
    - completion_tokens
  format: "de"                     # "de" | "en" label localization

Flow

1. After response is produced (or stream complete), compute footer
   string from usage.* already in hand.
2. Append marker + footer to final message content.
3. Streaming: footer is an additional single chunk at the end of the
   SSE stream.

Why content-embedded instead of just usage.* visibility

• UIs already render content — no UI changes needed.
• Marker makes the footer programmatically strippable client-side if
  desired: re.sub(r'<!-- localai-usage-footer -->.*$', '', content, flags=re.DOTALL).
• Alternative (get every chat UI to add a footer component) is N×M
  work we don't control.

Non-goals

• Not a billing/observability replacement — complements Traces and
  Prometheus, doesn't replace them.
• No multi-line markdown-table footer — one compact line.
• No mid-response footer — once at end.

Backward compatibility

Opt-in. Default off. No change when absent.

Open questions

1. Marker format — HTML comment is markdown-safe in most renderers.
   Alternative: a ZWSP-delimited line. Preference?

2. Label localization — embedded de/en or operator-supplied
   template string? Template is more flexible but widens the surface.

3. Metric emission — do we also emit localai_usage_footer_rendered_total
   or is that over-instrumentation for a UX feature?

4. Interaction with compression (RFC RFC: Optional server-side context-compression middleware for /v1/chat/completions #9534) — when compression
   fires, footer should show the ratio. Coordination: footer middleware
   runs after compression middleware in the chain, reads
   usage.compression_meta to populate compression_ratio field.

Prior art

walcz.de production (prompt-optimizer service) renders this via
post-processing. Moving it into LocalAI removes one layer of our proxy
chain and benefits any LocalAI operator with end-user-facing chat UIs.

Implementation outline

1. UsageFooterConfig in core/config/model_config.go:32.
2. Middleware core/http/middleware/usage_footer.go — Echo
   response-writer wrapper.
3. Streaming: append one final chunk with the footer.
4. pkg/usage_footer/i18n.go — tiny label map.
5. Ginkgo tests; docs at docs/content/features/usage_footer.md.

Approx 1-2 days.

Assisted-by: Claude:claude-opus-4-7

[walcz-de]

Closing — we'll keep this as a client-side feature in our sidecar rather than asking LocalAI to own presentation UX. Thanks.