You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
[v1.9.0] - 2026-11-12
Added
Add Server.ConditionEngineMode (LegacyFirst default / ExprCompatible / ExprOnly): the expr condition engine with three modes — LegacyFirst keeps the generated parser as baseline with expr fallback, ExprCompatible allows mixing legacy primitives with expr syntax, ExprOnly is pure expressions; an illegal mode or illegal condition fails fast at load time
Add AIConf.PriceMissAction three-state control for a missing (model, mode) price entry (issues #1408/#1479): fallback_chat (default, the previous behaviour), bill_zero (bill at 0 and count PriceMissZeroBilled), and reject (refuse before forwarding with 503 + error.code=PRICE_NOT_CONFIGURED, no upstream call, no billing, count PriceRejected; a missing ModelTable as a whole rejects the same way). Price-miss signals come in four layers: a structured throttled PRICE_MISS log, the counters PriceMissZeroBilled / PriceFallbackUsed / PriceRejected / PriceFallbackSubItemMiss, the access-log fields ai_price_miss (763, now also written on non-2xx reject rows) and ai_price_warn (766), and ready-made reconciliation queries
Add a configurable key-penalty TTL (issue #1448): the AI upstream key penalty (circuit breaker, issue #1439) duration is configurable instead of hard-coded, so a 429 punishment window can be tuned per deployment
Add reasoning-content normalization (2026-11-03): normalize reasoning_content / thinking-block variants across providers so downstream clients see a stable shape; the gateway skips rewriting when the request body exceeds Server.AccessibleBodySize and counts ReasoningSkip instead of touching a truncated body
Add AIConf.EnsureReasoningContent ([]string, default empty = disabled): the model access protocols for which the gateway fills an empty reasoning_content on the assistant messages that sit after the last user message. Some thinking-mode upstreams require that field to be echoed back yet have turns that produce no reasoning text, so a client mirroring the upstream response gets a 400 on the next turn, and the first request after a route switch always does (the assistant turn came from the previous backend). openai adds "reasoning_content": ""; anthropic prepends an empty thinking block. Only openai and anthropic are accepted — gemini has no such field in its contents/parts body, so it is rejected at load time rather than silently doing nothing; each entry must also be declared by the cluster's ModelProtocols. The rewrite is in place (sjson): field order and every other byte are preserved, existing non-empty values are never touched, and any parse/structure failure (including a body that is not valid JSON, or one larger than Server.AccessibleBodySize) passes the request through unchanged. New counters ReasoningPatched / ReasoningSkip / ReasoningFail / ReasoningStill400 — the last one is the exit signal: it counts requests that were patched and still got a 400, i.e. the upstream changed the contract and the switch must be cleared
Add Server.AIUpstreamAcceptEncoding (auto/identity/off, default auto): how Accept-Encoding is rewritten on an AI upstream request, so content negotiation stays inside the coding set the gateway can decode (gzip, deflate). auto keeps only the decodable codings of the client's list (q values preserved, an empty intersection is left untouched, identity;q=0 is honoured), identity strips compression for diagnostics, off keeps the previous behaviour. The rewrite only touches the outbound header: the inbound request, the access log, mod_ai_cache and mod_compress still see the client value (issue #1468)
Add bfe_util.NewContentDecoder / CanDecodeContentEncoding / SupportedContentEncodings, a streaming Content-Encoding decoder covering gzip/x-gzip and deflate (zlib header sniffed, raw DEFLATE accepted). DecodeContentEncoding now delegates to it, so the non-streaming parse-only copy also understands deflate (issue #1468)
Add the ResStreamCompressed / ResStreamDecoded / ResStreamDecodeSkipped / ResStreamRawOnlymod_body_process counters: how many responses carried a Content-Encoding, how many got the decompress-for-parsing tap, how many could not be decoded and how many degraded mid-stream. The last two mean "this request is billed by estimation", i.e. the only signal that an upstream ignored the narrowed Accept-Encoding (issue #1468)
Changed
Downgrade the normative client-close semantics of the "non-streaming boundary" clause 3 (issue #1474, product ruling 2026-11-12): when a non-streaming 200 response body is truncated mid-transfer, the connection now closes gracefully with FIN (consistent with the general proxy path and mainstream reverse proxies such as nginx), and the gateway is no longer required to abort the connection / send RST; truncation detectability relies on the response framing (a short read on a fixed-length response or a missing terminating chunk on a chunked response is detected by a conformant HTTP stack per RFC 9112 §7.1), and for close-delimited shapes (Content-Length stripped by transport-level gzip decompression, HTTP/1.0 clients) a clean EOF is indistinguishable from a complete response — an accepted residual limitation. The previously implemented abortive-close fix (closeDirectly + RST) has been reverted; the system-test acceptance criterion SC3201-TC035 A-06 is relaxed accordingly, while the err_code=BK_READ_RESP_BODY and billing-attribution criteria are unchanged
Finalize the non-2xx usage counter semantics as a dual-counter scheme (issues #1409/#1480/#1493, driven by the QA re-check SC2201-TC005): suppressed estimate residue counts Non2xxEstimateDropped, while a non-2xx response body that carries a real usage object is reported as an observation with ai_usage_source=parsed (never billed) and counted by Non2xxRealUsageKept — the non-streaming parse gate is opened from this release (the previous StatusOK gate made that branch unreachable), and the billing completion gate and estimate seeding stay 2xx-only. The issue #1480 renaming to Non2xxUsageReported and the failed-source reporting were rejected and never shipped
An AI upstream request whose client Accept-Encoding contains codings the gateway cannot decode no longer receives them (Server.AIUpstreamAcceptEncoding = auto, the default): the upstream is offered gzip/deflate only, so a client that asked for zstd may now receive gzip instead (still inside its own Accept-Encoding, <1% size difference). Set Server.AIUpstreamAcceptEncoding = off to keep the previous negotiation (issue #1468)
Fixed
Fix a data race on the forwarded request body size that -race reports intermittently in bfe_server (e.g. TestForwardExpectContinue) and therefore reddens CI: bfe_http.(*Request).write stores req.State.BodySize on the transport's writeLoop goroutine after the body is written, while clusterInvoke reads it from the serving goroutine as soon as the response header arrives. The two have no happens-before edge — an upstream that answers before reading the body, or a client that sends its body slowly (Expect: 100-continue is the typical shape), lets the response land first — so the pair was undefined behaviour; the store and the load are now atomic.StoreUint32/atomic.LoadUint32. No behavioural change: when the response wins the race, req_body_len still reads 0, which is what the (previously undefined) plain read observed anyway. The race has been present since the initial commit and is unrelated to any recent feature work
Stop recording a fully delivered AI streaming response as a client write error when the client hangs up right after the protocol's terminal event (issue #1473): the cancelOnClientClose watcher closes the upstream body, the collateral read error was labelled CLIENT_WRITE and counted in ErrClientWrite, and its err_msg blamed the upstream (BodyProcessError: read tcp ...) — so 52% of production CLIENT_WRITE rows were normal hangs, real faults drowned in the noise, and the metric could not be alerted on. The AI send path now attributes the copy error in three states: an upstream drop stays BK_READ_RESP_BODY (issue #1474), a close observed after IsResponseCompleted() sets no err_code and no ErrClientWrite but the new access-log field ai_client_closed_after_complete (816) plus the ReqAiClientClosedAfterComplete counter, and everything else (abort before the terminal event, failed write) keeps CLIENT_WRITE with a direction-correct err_msg (client closed connection / client write failed). Also wire the last missing ai_stream_truncated (815) path: a clean EOF without the protocol's terminal event now marks truncation even when NormalizeUpstreamError is unconfigured (the injector is then the only observer), with the event parser authoritative over the byte-level markers so data:-only Anthropic-compatible streams are not misreported, and a cross-read overlap window so a marker split across reads is not missed. Requires bfe-access-pb v0.3.18
Stop reporting an affinity binding miss as a Redis failure in the AI upstream key session affinity (issue #1472): RedisClient.Get reports an absent key as redis.ErrNil, and redisGetBinding passed it on as a real error, so the first request of every session (and every request after the binding TTL expires or is dropped by a 429/401) logged aiKeyAffinity: redis get binding error[redigo: nil returned], counted ReqAiKeyAffinityRedisErr and returned early — leaving ReqAiKeyAffinityMiss (and ReqAiKeyAffinityPenaltySkip) at zero in production and the new binding unwritten until a successful forward, which also kept a session whose first request failed from ever binding. The miss is now normalized with redis_client.IsKeyNotFound at the read point (redisGetBinding, and redisGetBatchBinding for the batch namespace), so a plain miss takes the documented path (weighted random, write the binding, count the miss) while a real Redis failure still warns and counts
Parse the usage of a compressed streaming (SSE) response instead of estimating it: mod_body_process bypassed any Content-Encoding body to a byte-wise pass-through, so cache_read_tokens stayed 0, prompt_tokens fell back to the request-body estimate and completion_tokens to the compressed-bytes estimate, and the whole prompt was billed at the input price — measured on the field at a 96% cache-hit rate that overcharges by roughly 46x and writes ai_usage_source=estimated. Compressed SSE now goes through CompressedStreamDecoder, which tees the source into a decompressor used only for parsing while the original compressed bytes are forwarded verbatim together with the Content-Encoding header; a decompression failure or an unsupported coding degrades to a byte-exact pass-through plus the estimate, visibly (issue #1468)
Report a skipped stream_options.include_usage injection when the request body exceeds Server.AccessibleBodySize: the injection used to fail silently, which raised the probability of an upstream returning no usage at all (issue #1468)
Fix a nil pointer panic that killed the request before its access log was written when a non-streaming 200 response body was truncated mid-transfer (issue #1501, PR #1504): bfe_http.(*response).GetBodyAccessor nil-ed res.Body on a failed buffered read and mod_body_process.(*JsonDecoder).Decode dereferenced the nil reader, so the connection-level panic was recovered without a RequestLog row and BK_READ_RESP_BODY/usage/quota attribution was lost entirely. The body now survives a failed accessor read (the forwarding path reports BK_READ_RESP_BODY / upstreamDropped and bills only forwarded content), and mod_body_process degrades safely on a nil/closed body instead of panicking. The non-stream truncation acceptance criteria (the "non-streaming boundary" clause of 2026-10-30-issue-1474) were re-verified after deploy, which drove the client-close semantic downgrade recorded under Changed above
Fix per-count (image/video generation) billing when the response omits the usage object (issue #1498): the generated count parsed from the response data.# array is adopted as recognized usage, so billing follows what the upstream actually produced (ai_usage_source=parsed) instead of the request-preset n (which overcharged when data.# < n) or nothing (which undercharged when EstimateToken=false); token-priced entries and responses carrying an explicit count are untouched
Fix early usage becoming the billing base of a stream whose terminal event carries no usage (issue #1497): an upstream that emits usage on a non-final event and omits it on the terminating event used to be billed by the early aggregate exactly, mismatching the reported totals; early usage now only feeds the protocol context (EarlyUsageSuppressed marker), and billing falls back to the forwarded-content estimate per the issue #1415 yardstick
Open the non-streaming usage parse gate for non-2xx responses so the Non2xxRealUsageKept counter is reachable (issues #1480/#1493, QA re-check SC2201-TC005): a real usage object on an error response is a real observation — it is now parsed, reported with ai_usage_source=parsed without billing, and counted; rows without a usage object keep the issue #1493 suppression (Non2xxEstimateDropped, token fields absent). The billing completion gate and estimate seeding stay 2xx-only, and the issue #1501 truncated-body semantics are preserved
Make the mod_ai_cache JSON monitor counters (REDIS_ERR/LATENCY_MS) consistent with the Prometheus-exported values (issue #1499): a monitoring-consistency fix on the cache path; no billing behaviour change
Deduct the full quota overage on exhaustion instead of leaving a residual (issue #1476), close EPP-scheduled long streams' billing on connection close correctly (issue #1475), and short-circuit EPP reject decisions before forwarding (issue #1482)