A rate limit says when it stops being true; we were not listening

[devdownin]

Description

Four things about the OpenRouter integration, found by reading it end to end. Split out of #275, which is back to its own subject.

A 429 was handled like a 5xx. sendWithRetry retried both on the same backoff — 500 ms, then 1 s, then give up: all three attempts inside a second and a half, which is shorter than any rate limit worth the name. So the caller was told "call failed with status 429" about a request that would have been accepted a few seconds later, having been refused three times to get there. The two are not alike: a 5xx may well pass half a second later, while a rate limit states when it stops being true. Retry-After is now read first and in both its legal forms (delta-seconds and an HTTP date), then OpenRouter's X-RateLimit-Reset, which is an instant rather than a delay and is taken as seconds or milliseconds on the same 10^10 threshold setup-demo.sh uses. Bounded twice — 30 s, and never past the call's own request timeout — and a delay beyond that is a refusal naming it rather than three more attempts arriving at the same answer, since nothing here can shorten it. Everything unusable (an unparsable header, a negative delay, a reset already past) answers -1, which selects the ordinary backoff: a gateway that says nothing behaves exactly as before.

The model shortlist led with the models the default policy cannot route. It is sorted cheapest-first, so free models arrive at the top — and under the shipped openrouter-data-collection: DENY those are usually the unroutable ones, the free endpoints being the ones that train. The first rows offered were therefore the ones that answer 404, qualified only afterwards by explainRoutingRefusal. They now carry the caveat. It warns rather than deciding — which providers serve a model today is the routing's business — and it keys on the published price being zero, a real measurement, never on the :free suffix, which is a naming convention; an absent price stays null and produces nothing. The flag comes from the policy that is running, never from the form.

The key's remaining credit was consulted nowhere. A key running out answers 402, which LlmHttpSupport.remedyFor already reads correctly — but only after an analysis has failed. It is published, so the press that already asks what the model can do asks this too, under the catalogue's four rules (OpenRouter's own host, best-effort, no retry, never on the analysis path). It is also the number claude.session-cost-limit-usd had been missing: that cap ships disabled because "any figure chosen here would be arbitrary", and arbitrary is what a cap is while nothing on screen says what the budget is. Three states are kept apart, and the third is the one to get right: a limited key with credit left, a limited key with none (a 402 waiting to happen), and a key with no limit at all, which has a known usage and no remainder — "unlimited minus what you spent" is not a number, and rendering it as 0 would announce an exhausted key to somebody who has an unlimited one. error is a fourth thing entirely and renders nothing: it says the question could not be asked.

claude.openrouter-max-price-usd-per-million is the enforceable half of session-cost-limit-usd. That one is an accumulator: it adds up what has already been spent and stops the session afterwards, which is the right shape for a running total and the wrong one for a surprise — a model routed to an expensive provider has already been paid for by the time the cap notices. A price ceiling is refused at the routing layer, before the tokens exist, the same way data_collection turns a privacy warning into a property. One value against both published prices, in the unit the catalogue publishes and the picker renders. 0 and off by default, on the rule its two siblings follow: a model every provider prices above the ceiling becomes unroutable, arriving as the same "no endpoints found" a restrictive data policy produces.

Two smaller ones from the same reading. A 404 now names all three things it can mean on this gateway — a slug that does not exist, one that no longer does, a key not entitled to it, plus the routing policy when one is set — and points at the Test button, which is the one place that asks the catalogue. Deliberately not a lookup from the analysis path: that read is a considered gesture, and a live session that had begun failing would otherwise make one request per window. And OpenRouterModelCatalog now builds its HTTP client through LlmHttpSupport.newClient instead of restating the connect-timeout rule — one definition, the argument that produced SecureXml and v1Url.

docs/verify-openrouter-contract.py reaches the completion request, behind --chat because those are real billed calls. It covered the catalogue only, while the body of a completion is where provider and usage ride on the assumption that an unrecognised field is refused rather than ignored — which is the wrong way round: an ignored field costs nothing and says nothing, so a privacy restriction that never applied and an accounting field that never came back both look exactly like success. Its new assertions: whether usage.cost needs usage: {include: true}; that provider.max_price binds rather than being tolerated (a ceiling below every published price must refuse the route — a 200 there would mean the setting promises a bound it does not deliver); and what /models/user is relative to /models.

That last one is why the shortlist does not mark the rows a key cannot reach, though it easily could: nothing has established whether that endpoint lists what a key is entitled to or something narrower, and labelling a usable model unusable is a worse lie than the silence it would replace.

Related Issue

None — found by reading the OpenRouter integration end to end, at the user's request.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

LlmTestResponse gains a component and keeps its previous constructor; describeOption gains an optional second argument defaulting to today's behaviour. One new setting, claude.openrouter-max-price-usd-per-million, defaults to 0 = off, so nothing changes for an existing deployment. CLAUDE.md, CHANGELOG.md, docs/DOCKERHUB.md and docs/LLM-PROVIDERS.md are updated.

How Has This Been Tested?

packages.confluent.io answers 403 through this environment's proxy — the case verify-offline.sh exists for — so the backend was compiled and run through that harness rather than mvn verify. CI is the authority.

• Full Java suite offline — 830 tests, 0 failures (KafkaClusterIntegrationTest excluded: it needs Testcontainers).
• LlmRateLimitTest (new): both header forms, the seconds/milliseconds threshold, Retry-After winning over the reset instant, and every unusable form falling back to the ordinary backoff — all pure, with the clock as a parameter, so none of it waits. Then, against a real loopback HTTP server: a short delay waited out in one retry, a long one refused at once without a second call, a gateway sending no header behaving as before, and the cap following the call's own timeout when that is the smaller of the two.
• OpenRouterModelCatalogTest: the key's three states, including an unlimited key that must not read as an exhausted one, the remainder derived only as a fallback and only from two real numbers, and a body carrying nothing about the key reported as unavailable rather than as an empty key.
• Frontend — 1319 tests across 54 files, including the free-model caveat appearing only under the policy that refuses data collection, never on a priced model, and never on one whose price was simply not published.
• npx tsc --noEmit clean, npm run lint clean (--max-warnings 0).
• All docs/check-*.py pass — including check-config-table.py, which resolves the new setting's environment-variable form against the code.
• docs/verify-openrouter-contract.py --chat has NOT been run. openrouter.ai is unreachable from this environment (403 on the CONNECT tunnel), which is the very reason that script exists. Its usage.cost assertion decides whether the client should start sending usage: {include: true}, and its max_price assertion decides whether the new ceiling actually binds. Somebody with a key and a network should run it before either is trusted — nothing in the client was changed on the strength of an unverified answer, which is why the cost field is read exactly as before.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

🤖 Generated with Claude Code

https://claude.ai/code/session_0119y8rLek8kEbt6Ym1xZ8N1

---

Generated by Claude Code