tok/s missing with external llama.cpp api connection

[rozsazoltan]

Hi,

I noticed that Unsloth Studio shows tok/s when using the built-in llama.cpp integration, but not when I connect to my own llama.cpp / ik_llama.cpp server through the Connection API.

Streaming works, and the server is started with --metrics. The /metrics endpoint also works.

Is this expected? Does the built-in integration use extra llama.cpp timing/usage data that the external Connection API currently does not request or forward?

It would be nice if externally connected llama.cpp servers could show the same tok/s stats too.

Default	Connection API

[tk1475]

This is expected, and it's separate from --metrics. There are three different timing surfaces in llama.cpp:

• /metrics is a Prometheus scrape of server-wide counters. It isn't per-request, so a client can't turn it into the tok/s for the response you just streamed.
• The native /completion endpoint returns a timings object per response (predicted_per_second, predicted_ms, …) — that's what a built-in integration usually reads.
• The OpenAI-compatible /v1/chat/completions route (what a generic Connection API targets) omits those timings by default. On recent llama.cpp you can request them per call with "timings_per_token": true, which adds timing to the streamed chunks; without it the client has nothing to show.

So the built-in one shows tok/s because it talks to the native endpoint (or asks for timings), while the external connection likely hits the OpenAI route without them. Two ways to get it: send timings_per_token: true and read timings.predicted_per_second, or compute it client-side (tokens streamed ÷ wall-clock). Worth filing as a feature request with that framing.