Repository navigation
Releases: SergeevDmitry/llm-kit
Release list
mend-json@1.2.1
Patch Changes
-
84fa15d: Fix string value completion, which skipped value counting and duplicate-key handling. This caused four issues:
- Trailing commas after strings (
["x",],{"a":"x",}) could be accepted as empty closes and reportedcomplete: true, violatingfinish()'s validity guarantee. duplicateKeyPolicy: 'first'could keep a later string value or drop following members.- Snapshots ending after a string value's comma (
"...",) could returnvalue: undefinedinstead of completed values. pending.pathignored completed strings when calculating array indexes.
- Trailing commas after strings (
usage-tab@1.3.0
Minor Changes
-
a721a5f: Refresh every provider's pricing data (observed 2026-10-03) and add the models released since the last pass. 152 models to 172.
Azure now charges OpenAI's first-party price for
gpt-5.6-sol,gpt-5.6-terraandgpt-5.6-luna. Azure cut sol from $5.00/$30.00 to $4.00/$20.00, terra from $2.50/$15.00 to $2.00/$12.00 and luna from $1.00/$6.00 to $0.20/$1.20 per million tokens. The new rates apply from 2026-10-03. Lookups dated earlier still return the prices observed then, so luna on 2026-08-05 is still 5x apart. The README's headline example now usesgemma-4-31b, which costs 2.5x more on Together than on AWS Bedrock today.Retired models are removed, so a lookup for them now throws
UNKNOWN_MODELat any date; price historical usage for them with an override. They are Mistral'sdevstral-2,devstral-small-2,magistral-medium,magistral-small,mistral-nemo,mixtral-8x7bandmixtral-8x22b; Groq'sllama-3.1-8b-instant,llama-3.3-70b-versatileandqwen3.6-27b; and Cohere'saya-expanse-8b.gemini-omni-1.1-flashandgemini-omni-flash-previewrecord the $9.00 text output rate and carry aPARTIAL_TIER_PRICINGwarning, because video output costs $17.50 per million tokens.openai:gpt-3.5-turboloses its batch rate, since OpenAI's Batch table no longer lists it: a batch request for it is priced at standard rates with aBATCH_PRICING_UNAVAILABLEwarning.Other price changes:
together:qwen3.7-maxrose from $1.25/$3.75 to $1.50/$4.50, andtogether:qwen3.8-flashfell from $0.15/$0.47 to $0.09/$0.28.New models:
claude-opus-5-5,claude-sonnet-5-5;gpt-6.1-sol,gpt-6-sol,gpt-6-luna,chat-latest,gpt-5.3-codex,gpt-5-search-api,gpt-rosalind-research,gpt-5.6-cyber,gpt-5.5-cyber;gemini-3-flash-preview,gemini-omni-1.1-flash,gemini-omni-flash-preview,gemini-robotics-er-2-preview,gemini-robotics-er-2-streaming-preview;gpt-6-astraon Azure;gemma-4-26b-a4b,gemma-4-e2b,gemma-3-4bandnemotron-3-nano-30bon Bedrock;command-r7b-12-2024andcommand-r-08-2024on Cohere;zai-glm-5-3on Mistral; seven more on Together.Newly published rates fill in fields that were missing:
cacheWriteon four OpenAI models and OpenRouter's Gemini entry, so their cache writes are priced at the published rate instead of the input rate with a warning.cachedInputongemini-3.5-flash-lite, five Together models and four Mistral models.batchMultiplieron five Mistral models and on Azure'sgpt-4-turboandgpt-4.
-
c5113e6:
createPriceCalculatoraccepts a defaultprovider.An id registered under several providers, such as
gpt-5.6-lunaunderopenaiandazure-openai, used to throwAmbiguousAliasErrorunless every call repeated the qualifier. A per-calloptions.providerorrequest.providerstill takes precedence, and the default is a hard constraint like either of them: an id the provider does not list throwsUnknownModelError.
Patch Changes
-
b94effd:
normalizeAnthropicUsagenow warns withCACHE_WRITE_TTL_NOT_MODELEDwhen a response reports 1-hour cache writes.The registry records one cache-write rate per model, the 5-minute TTL rate. A 1-hour write costs more (2x input against 1.25x), so those tokens were under-priced with no warning. The cost is unchanged; the warning now says it is low.
cache_creation.ephemeral_5m_input_tokensandephemeral_1h_input_tokensare also validated whencache_creation_input_tokensis present, so a malformed value throwsInvalidUsageErrorinstead of being ignored. -
ac175df:
normalizeOpenAICompatibleUsagenow warns about unrecognized fields insideprompt_tokens_detailsandcompletion_tokens_details, matchingnormalizeOpenAIUsage.It used to check top-level keys only, so a token class a gateway added inside a details object, such as
image_tokens, went unpriced with no warning.
vec-cache@1.2.0
Minor Changes
-
376af3e: Add
findMissing(texts, options), which returns the texts that still need embedding without reading any vectors.Answering "which of these documents do I still need to embed?" used to mean calling
getMany, which copies and decodes a full vector for every hit only for the caller to discard it.findMissingruns the same lookup over each row's key and dimensions only. It returns exactly the listgetOrCreatewould pass toembedfor the same batch: each missing text once, in first-occurrence order, with expired entries and wrong-width hits counted as missing. It takes the same options asgetMany.
Patch Changes
-
0965774:
getOrCreatenow returns every vector, hit or miss, as a typed array in the instance'svectorEncoding.A miss used to come back as whatever
embedreturned, usually a full-precisionnumber[], while a hit came back as a decodedFloat32Array. One result could mix both,JSON.stringifyrendered one as an array and the other as an object, and a text's first call returned different numbers from its later hits. A miss is now converted to the value the cache stored:Float32Arrayby default,Float64ArrayundervectorEncoding: 'float64'. SetvectorEncoding: 'float64'to keep the full precisionembedreturned.getManyalso converts a row written under anothervectorEncodingto the instance's. -
719a52f: Rewriting a cache entry now resets its creation time.
Re-embedding a TTL-expired entry, or calling
setManyover an existing key, used to keep the oldcreated_at_ms.prune({ olderThanMs })could then delete a row that had just been paid for, forcing another embed on the next request, andstats().newestEntryMsdid not reflect the rewrite. An entry's age now counts from its latest write, as itsexpires_at_msalready did.
chat-fit@1.2.0
Minor Changes
-
93eb4b5: Add
countChatTokens(messages, options?), which counts a conversation without trimming it.chat-fit's message accounting reads
ChatMessage-, OpenAI- and Anthropic-shaped messages, including tool calls and fields it does not recognize, and until nowfitChatwas the only way to reach it. Asking "how big is this conversation?" meant running a fit against a hugemaxTokensto readreport.initialTokenCount, or writing a counter by hand and undercounting the provider shapes.It returns
{ tokens, counterId, approximate, warnings }.tokensis exactly theinitialTokenCountafitChatcall would report for the same messages and tokenizer.approximateis true when the bundled tokenizer produced the count, in which case treat it as an upper bound, andwarningsnames any message whose shape had to be estimated.tokenizerandmaxMessageswork as they do inFitChatOptions. -
2092c69: Add
FitChatReport.trimmedForSummaryIndexes, naming the messages given up to make room for a summary.Under
strategy: 'summarize-middle'the summarizer only sees the dropped middle range. When the summary it returns still does not fit, the newest kept messages are trimmed next, and no summary replaces those.report.summarizedRange.messageCountcounts the range alone, so it accounted for fewer removals thanreport.removedIndexesactually listed, and no warning said so.The new field names the rest. It is present exactly when
summarizedRangeis and empty when nothing extra was given up, sosummarizedRange.messageCount + trimmedForSummaryIndexes.length === removedIndexes.lengthholds whenever a range was summarized; a non-empty value also appears inreport.warnings.summarizedRangeis not widened to cover these messages, since the summarizer never read them.
Patch Changes
- a43f3bf: Hand the summarizer its range in conversation order. Under
strategy: 'summarize-middle',SummaryRequest.messageswas built by flattening the dropped groups one at a time. Groups sort by their first index, but a tool-call group's messages need not be contiguous: parallel calls can have their results interleaved with other turns, and two assistant turns' calls can come back reversed. The summarizer then read a reply ahead of the turn it answered. It now gets the range ordered by original index, the same ordering the result itself gets.
usage-tab@1.2.0
Minor Changes
-
97a1ce9: Refresh every provider's pricing data (observed 2026-09-07) and add the models released since the last pass. 123 models to 152.
Correction:
anthropic:claude-sonnet-5was over-reported by 50% for any date from 2026-09-01. The previous snapshot modelled Anthropic's announced increase to $3.00/$15.00 per million tokens as a real pricing period. Anthropic's pricing page now states that increase will not occur and the $2.00/$10.00 launch rate is the standard price, so the period is removed - a rate that never took effect must not be reachable at any lookup date.Other price changes:
openai:gpt-5.6-solcut from $5.00/$30.00 to $4.00/$20.00 (Azure's resale of the same model did not follow, so it is now a confirmed divergence);google:gemini-3.6-flashmoved to a promotional $0.75/$3.75 published as running through 2026-12-31, with the standard $1.50/$7.50 resuming 2027-01-01.New models:
claude-fable-5-1,claude-mythos-5-1,claude-mythos-5and six legacy Claude models Anthropic still publishes rates for;gpt-6-astra;gemini-3.8-flash,gemini-3.7-flash,gemini-3.1-flash-lite;qwen3.8-27bon Groq;zai-glm-5-2on Mistral; eleven more on Together;gemma-3-27b,gemma-3-12bandnemotron-3-super-120bon Bedrock.Newly published per-model detail fills in fields previously omitted for want of a source:
cachedInputon eight Gemini models, three Mistral models and three OpenRouter models,cacheWriteon OpenRouter's Claude entry, andbatchMultiplieron the two Gemini Pro models, three Mistral models and OpenAI's-protiers. TheUNCERTAINflag onopenrouter:anthropic/claude-sonnet-5is withdrawn: the rate it questioned turned out to be correct.
mend-json@1.2.0
Minor Changes
-
566adce: Add
JsonMendResult.pending: which part of the document is still incomplete, as{ path, kind }.pathis object keys and array indexes from the root down, so it indexes straight intovalue;kinddistinguishes a scalar still arriving ('string','number','literal') from an object key still being read ('key') or a key whose value has not started ('member'). It isundefinedwhen nothing is incomplete - the snapshot iscomplete, no value has started, the scanner sits between values, or it froze on invalid input. Costs one O(depth) walk of the existing scanner stack, no extra scanning. -
24d3151: Add
JsonMendResult.removedRanges: the half-open[start, end)ranges cut out from inside the valid prefix, sorted and non-overlapping. This completes the identityappendedSuffixdocuments -input.slice(0, validPrefixLength), minusremovedRanges, plusappendedSuffix, is exactlyrepairedJson- which previously had a silent exception underduplicateKeyPolicy: 'first', where a caller slicing the raw buffer itself would render a member the package reports as dropped. Empty under every other policy and repair.Also fixes an aliasing bug this exposed: merging exclusion ranges widened a tuple the scanner still owned, mutating scanner state from a call documented as read-only.
Patch Changes
- 1460e14:
incompleteScalarPolicy: 'best-effort'no longer turns a contradicted number into a valid one. Inputs like{"a":01},{"a":1.x}and{"a":1e+x}are syntax errors rather than truncations, so they are now omitted under'best-effort'exactly as under'omit'- matching the behaviour already in place for contradicted literals such astru5. Genuine truncation (1.,1e+) still trims back to the typed digits with anumber-truncateddiagnostic.
token-chunk@1.2.0
Minor Changes
- 628c1b7: Add
TextChunk.syntheticPrefixLength: number, how many leading characters ofchunk.textwere synthesized rather than sliced out of the input (a rendered heading breadcrumb, a repeated table header, or both) — sochunk.text.slice(chunk.syntheticPrefixLength)is exactly the documented source reconstruction, and a consumer can strip the non-source-backed part before hashing, deduplicating or mapping a position back to an input offset without re-deriving the prefix. It is0when nothing was prepended, including when a prefix was dropped to keep the chunk within budget.
Patch Changes
- 0c727b4: Keep a table row's trailing newline when the row itself has to be split. The
row unit was built from the bare line, one character short of the extent
bodySpansrecords, so that newline landed in no chunk'stextand in no
chunk'ssourcerange: the chunk stopped reconstructing its own source slice,
and the row ran straight into whatever unit packed after it. - eba51b9: Stop dropping a split table's header row. When the header fit the budget on its
own but not alongside the first body row, that row was split out on a path that
bypassed the only code emitting the header as text — so the header and
separator lines appeared in no chunk'stextand in no chunk'ssource
range, leaving retrieved rows with no column names. The header is now emitted
once as its own source-backed chunk, and every fragment of that row repeats it
as a synthetic prefix, as documented.
vec-cache@1.1.0
Minor Changes
- cf49574:
getOrCreatenow checks an explicitdimensionsrequest against whatembed
actually returns, and passes the requested width to the callback as
request.dimensions. A callback that ignores it and returns another width
throwsEMBED_DIMENSION_MISMATCHbefore anything is written, instead of
storing a row keyed as the requested width and stored at another — which made
every later identical call demote the row and pay for the same embedding
again. - a1b0c2f: Add
getOrCreate'smaxEmbedBatchSizeoption: a cap on how many texts go to
embedin one call, so a cold cache over a large corpus no longer hands the
callback more inputs than the provider accepts. Sub-batches run in order and
each is cached as it succeeds — a later failure leaves the completed ones
cached and throws — andreport.embedCallCountis the number of calls
actually made. Omitting it keeps the previous behavior: one call for every
unique miss.
Patch Changes
- 09f228c: Honor
busyTimeoutMswhile the database is being opened. It was applied only
after the header probe, the migration reads and the WAL switch, so until then
the driver's own 5 s default was in force and a larger configured timeout was
silently capped — losing the race against another process checkpointing its
WAL onclose().
llm-backoff@1.1.0
Minor Changes
-
f88aef3: Add
attemptTimeoutMs, a per-attempt ceiling whose timeout is retryable.A hung call — connection established, tokens never arrive — is the failure this package could not recover from:
maxElapsedMsis only checked between attempts, so one hung attempt quietly eats the entire budget. WithattemptTimeoutMsset, an attempt that outlives it is cancelled and retried through the normal delay machinery, and surfaces as anAttemptTimeoutErroronLlmBackoffError.causeif every attempt hangs.Wiring this up yourself does not work, which is why it belongs in the package: an
AbortSignal.timeout()inside your operation produces aTimeoutErrorwith no status, so the classifier reads it as a non-retryable failure and gives up after one attempt. Only the retry loop can tell its own ceiling apart from a cancellation the caller asked for.options.signalkeeps absolute precedence — if both fire on the same attempt, your abort reason propagates unwrapped and unretried, exactly as before. The ceiling reaches your operation throughRetryContext.signal(combined withoptions.signal), so forwarding that signal cancels the in-flight request rather than leaving it holding a connection;fetchWithLlmBackoffalready forwards it. A failure that arrived before the ceiling is still classified as itself — a400under a 30 s ceiling is still non-retryable. Time spent on a timed-out attempt counts towardmaxElapsedMs, and the timer is armed through the injectablesleep, so an injected clock still sees no real timer.The option is off by default and nothing changes for callers who do not set it.
AttemptTimeoutErroris a new export.
Patch Changes
-
1bee72f: Honor the
AbortSignalaRequestinput carries.fetchWithLlmBackoffcombinedoptions.signalandinit.signalbut not the signal everyRequesthas, and it passes the combined signal on tofetchin the init bag — which, per the fetch spec, replaces the Request's own signal. SofetchWithLlmBackoff(request, undefined, { signal })stopped honoringrequest's controller entirely: aborting it no longer cancelled even the in-flight network call, something plainfetch(request)does for free.With no other signal supplied, the failure was quieter but worse. The retry loop's signal was
undefined, so an abort during aRetry-Aftersleep did nothing until the sleep elapsed — up to the full server-advised delay — and the eventual rejection was then classified as a non-retryable failure and wrapped inLlmBackoffError, breaking the documented contract that an abort propagates immediately and unwrapped with the exact reason object you passed toabort().All three channels are now combined, so an abort through any of them cancels the whole retry operation at every phase — before the first attempt, mid-fetch, and mid-sleep — and surfaces unwrapped.
-
2427b57: Detect every single-shot request body, not just
ReadableStream. Node'sfetchalso accepts an async iterable — an async generator, or anode:streamReadable, which is one — and a generator object, and turns each into a stream internally. All of them are drained by the first attempt, butfetchWithLlmBackoff's guard only recognized aReadableStream, so the exact situation this package promises to detect and refuse happened silently instead: attempt 1 sent the payload, the retry sent zero bytes, and the server answered 200 with nothing to signal that the body had vanished.Any of those bodies now throws
LlmBackoffError(REQUEST_BODY_NOT_REPLAYABLE) before the first request is made, with the same remediation — buffer into astring/Blob/ArrayBuffer/Uint8Array, or passmaxAttempts: 1.The test is single-shot-ness, not iterability:
FormData,URLSearchParamsand an array orSetof chunks are all iterable and all still retry normally, becausefetchasks each of them for a fresh iterator on every attempt.
usage-tab@1.1.0
Minor Changes
-
6b8eca2: Add
sumExactUsdandcreateCostAggregator— exact totals across many requests. The exact decimal channel previously dead-ended at a single request'stotalUsdExact: a caller wanting a daily or monthly figure — the package's namesake use case — either wrotesum += breakdown.totalUsd, reintroducing float drift at exactly the boundary the package pushed them past, or hand-parsed decimal strings. Both new functions sum on the samebigintfixed-point path every individual cost is already computed with, and reject anything that is not a non-negative decimal string (InvalidRateError) rather than coercing it.createCostAggregatoralso reportscount, per-model totals keyed by resolvedprovider:canonicalModel, and every distinctregistryVersionthe aggregate drew on, so a total that silently mixes pricing snapshots is visible.Also fix
parseDecimalRatethrowing a rawTypeError, outside the package's error taxonomy, for a non-string rate that reached it past the type — a hand-built override or a JSON round-trip.RegExp.teststringifies its argument, so a numeric1.5passed the decimal pattern and then failed onindexOf. It now throwsInvalidRateErrorlike every other malformed rate. -
da64bf2: Read every ISO
atstring as UTC, including the offset-less form.calculateCostpassedattoDate.parse, where a datetime without a UTC offset ("2026-09-01T05:00:00"— a log timestamp, adatetime-localinput, several DB drivers) is local time by specification, while an ISO date and an offset-carrying datetime are both UTC. Pricing periods start at UTC midnight, so the local-time reading landed on either side of a price restatement depending on the host: the identical historical lookup priced at $12.00 in one deployment and $18.00 in another, silently breaking the reproducibility a dated, committed price registry exists to provide. Such values are now normalized to UTC — the reading every other accepted ISO form already got — rather than inheriting the process's timezone. The same normalization applies toeffectiveFrom/effectiveToon a caller-supplied override, which never passed through the registry's schema validation.Non-ISO strings (
"2026/09/01","September 1, 2026") are implementation-defined rather than specified, so there is no single reading to normalize them to; they still go toDate.parseunchanged, and the documentation now says to pass aDateor an ISO form instead.Also fix an invalid
Dateinatescaping the error taxonomy: building theInvalidLookupDateErrorcalledtoISOString(), which throws on an invalid date, so callers saw an uncodedRangeErrorinstead. It now throwsInvalidLookupDateError(INVALID_LOOKUP_DATE) like the equivalent string.
Patch Changes
- 3cbd4c7: Fix a provider-qualified lookup silently resolving duplicate entries by array order instead of reporting ambiguity. When two caller-supplied
overrides(or two entries of a caller-supplied registry) shared the sameproviderandcanonicalId— the shape produced by merging an org-wide price list with a team's — the exact-canonical step took whichever came first, so the identical request could price differently depending on concatenation order. Supplying the provider qualifier, which the documentation tells callers to do, therefore made ambiguity detection weaker than the unqualified lookup, which already raisedAMBIGUOUS_ALIAS. Both channels now raiseAMBIGUOUS_ALIASwith every duplicate listed as a candidate. Well-formed data, where provider and canonical id are unique, is unaffected.