Skip to content

Releases: SergeevDmitry/llm-kit

mend-json@1.2.1

Choose a tag to compare

@github-actions github-actions released this 18 Oct 14:41
c2ead10

Patch Changes

  • 84fa15d: Fix string value completion, which skipped value counting and duplicate-key handling. This caused four issues:

    • Trailing commas after strings (["x",], {"a":"x",}) could be accepted as empty closes and reported complete: true, violating finish()'s validity guarantee.
    • duplicateKeyPolicy: 'first' could keep a later string value or drop following members.
    • Snapshots ending after a string value's comma ("...",) could return value: undefined instead of completed values.
    • pending.path ignored completed strings when calculating array indexes.

usage-tab@1.3.0

Choose a tag to compare

@github-actions github-actions released this 03 Oct 14:50
f1bdae0

Minor Changes

  • a721a5f: Refresh every provider's pricing data (observed 2026-10-03) and add the models released since the last pass. 152 models to 172.

    Azure now charges OpenAI's first-party price for gpt-5.6-sol, gpt-5.6-terra and gpt-5.6-luna. Azure cut sol from $5.00/$30.00 to $4.00/$20.00, terra from $2.50/$15.00 to $2.00/$12.00 and luna from $1.00/$6.00 to $0.20/$1.20 per million tokens. The new rates apply from 2026-10-03. Lookups dated earlier still return the prices observed then, so luna on 2026-08-05 is still 5x apart. The README's headline example now uses gemma-4-31b, which costs 2.5x more on Together than on AWS Bedrock today.

    Retired models are removed, so a lookup for them now throws UNKNOWN_MODEL at any date; price historical usage for them with an override. They are Mistral's devstral-2, devstral-small-2, magistral-medium, magistral-small, mistral-nemo, mixtral-8x7b and mixtral-8x22b; Groq's llama-3.1-8b-instant, llama-3.3-70b-versatile and qwen3.6-27b; and Cohere's aya-expanse-8b.

    gemini-omni-1.1-flash and gemini-omni-flash-preview record the $9.00 text output rate and carry a PARTIAL_TIER_PRICING warning, because video output costs $17.50 per million tokens. openai:gpt-3.5-turbo loses its batch rate, since OpenAI's Batch table no longer lists it: a batch request for it is priced at standard rates with a BATCH_PRICING_UNAVAILABLE warning.

    Other price changes: together:qwen3.7-max rose from $1.25/$3.75 to $1.50/$4.50, and together:qwen3.8-flash fell from $0.15/$0.47 to $0.09/$0.28.

    New models: claude-opus-5-5, claude-sonnet-5-5; gpt-6.1-sol, gpt-6-sol, gpt-6-luna, chat-latest, gpt-5.3-codex, gpt-5-search-api, gpt-rosalind-research, gpt-5.6-cyber, gpt-5.5-cyber; gemini-3-flash-preview, gemini-omni-1.1-flash, gemini-omni-flash-preview, gemini-robotics-er-2-preview, gemini-robotics-er-2-streaming-preview; gpt-6-astra on Azure; gemma-4-26b-a4b, gemma-4-e2b, gemma-3-4b and nemotron-3-nano-30b on Bedrock; command-r7b-12-2024 and command-r-08-2024 on Cohere; zai-glm-5-3 on Mistral; seven more on Together.

    Newly published rates fill in fields that were missing:

    • cacheWrite on four OpenAI models and OpenRouter's Gemini entry, so their cache writes are priced at the published rate instead of the input rate with a warning.
    • cachedInput on gemini-3.5-flash-lite, five Together models and four Mistral models.
    • batchMultiplier on five Mistral models and on Azure's gpt-4-turbo and gpt-4.
  • c5113e6: createPriceCalculator accepts a default provider.

    An id registered under several providers, such as gpt-5.6-luna under openai and azure-openai, used to throw AmbiguousAliasError unless every call repeated the qualifier. A per-call options.provider or request.provider still takes precedence, and the default is a hard constraint like either of them: an id the provider does not list throws UnknownModelError.

Patch Changes

  • b94effd: normalizeAnthropicUsage now warns with CACHE_WRITE_TTL_NOT_MODELED when a response reports 1-hour cache writes.

    The registry records one cache-write rate per model, the 5-minute TTL rate. A 1-hour write costs more (2x input against 1.25x), so those tokens were under-priced with no warning. The cost is unchanged; the warning now says it is low. cache_creation.ephemeral_5m_input_tokens and ephemeral_1h_input_tokens are also validated when cache_creation_input_tokens is present, so a malformed value throws InvalidUsageError instead of being ignored.

  • ac175df: normalizeOpenAICompatibleUsage now warns about unrecognized fields inside prompt_tokens_details and completion_tokens_details, matching normalizeOpenAIUsage.

    It used to check top-level keys only, so a token class a gateway added inside a details object, such as image_tokens, went unpriced with no warning.

vec-cache@1.2.0

Choose a tag to compare

@github-actions github-actions released this 27 Sep 10:11
60d4b29

Minor Changes

  • 376af3e: Add findMissing(texts, options), which returns the texts that still need embedding without reading any vectors.

    Answering "which of these documents do I still need to embed?" used to mean calling getMany, which copies and decodes a full vector for every hit only for the caller to discard it. findMissing runs the same lookup over each row's key and dimensions only. It returns exactly the list getOrCreate would pass to embed for the same batch: each missing text once, in first-occurrence order, with expired entries and wrong-width hits counted as missing. It takes the same options as getMany.

Patch Changes

  • 0965774: getOrCreate now returns every vector, hit or miss, as a typed array in the instance's vectorEncoding.

    A miss used to come back as whatever embed returned, usually a full-precision number[], while a hit came back as a decoded Float32Array. One result could mix both, JSON.stringify rendered one as an array and the other as an object, and a text's first call returned different numbers from its later hits. A miss is now converted to the value the cache stored: Float32Array by default, Float64Array under vectorEncoding: 'float64'. Set vectorEncoding: 'float64' to keep the full precision embed returned. getMany also converts a row written under another vectorEncoding to the instance's.

  • 719a52f: Rewriting a cache entry now resets its creation time.

    Re-embedding a TTL-expired entry, or calling setMany over an existing key, used to keep the old created_at_ms. prune({ olderThanMs }) could then delete a row that had just been paid for, forcing another embed on the next request, and stats().newestEntryMs did not reflect the rewrite. An entry's age now counts from its latest write, as its expires_at_ms already did.

chat-fit@1.2.0

Choose a tag to compare

@github-actions github-actions released this 14 Sep 10:56
b8e86b5

Minor Changes

  • 93eb4b5: Add countChatTokens(messages, options?), which counts a conversation without trimming it.

    chat-fit's message accounting reads ChatMessage-, OpenAI- and Anthropic-shaped messages, including tool calls and fields it does not recognize, and until now fitChat was the only way to reach it. Asking "how big is this conversation?" meant running a fit against a huge maxTokens to read report.initialTokenCount, or writing a counter by hand and undercounting the provider shapes.

    It returns { tokens, counterId, approximate, warnings }. tokens is exactly the initialTokenCount a fitChat call would report for the same messages and tokenizer. approximate is true when the bundled tokenizer produced the count, in which case treat it as an upper bound, and warnings names any message whose shape had to be estimated. tokenizer and maxMessages work as they do in FitChatOptions.

  • 2092c69: Add FitChatReport.trimmedForSummaryIndexes, naming the messages given up to make room for a summary.

    Under strategy: 'summarize-middle' the summarizer only sees the dropped middle range. When the summary it returns still does not fit, the newest kept messages are trimmed next, and no summary replaces those. report.summarizedRange.messageCount counts the range alone, so it accounted for fewer removals than report.removedIndexes actually listed, and no warning said so.

    The new field names the rest. It is present exactly when summarizedRange is and empty when nothing extra was given up, so summarizedRange.messageCount + trimmedForSummaryIndexes.length === removedIndexes.length holds whenever a range was summarized; a non-empty value also appears in report.warnings. summarizedRange is not widened to cover these messages, since the summarizer never read them.

Patch Changes

  • a43f3bf: Hand the summarizer its range in conversation order. Under strategy: 'summarize-middle', SummaryRequest.messages was built by flattening the dropped groups one at a time. Groups sort by their first index, but a tool-call group's messages need not be contiguous: parallel calls can have their results interleaved with other turns, and two assistant turns' calls can come back reversed. The summarizer then read a reply ahead of the turn it answered. It now gets the range ordered by original index, the same ordering the result itself gets.

usage-tab@1.2.0

Choose a tag to compare

@github-actions github-actions released this 07 Sep 15:35
1c106a5

Minor Changes

  • 97a1ce9: Refresh every provider's pricing data (observed 2026-09-07) and add the models released since the last pass. 123 models to 152.

    Correction: anthropic:claude-sonnet-5 was over-reported by 50% for any date from 2026-09-01. The previous snapshot modelled Anthropic's announced increase to $3.00/$15.00 per million tokens as a real pricing period. Anthropic's pricing page now states that increase will not occur and the $2.00/$10.00 launch rate is the standard price, so the period is removed - a rate that never took effect must not be reachable at any lookup date.

    Other price changes: openai:gpt-5.6-sol cut from $5.00/$30.00 to $4.00/$20.00 (Azure's resale of the same model did not follow, so it is now a confirmed divergence); google:gemini-3.6-flash moved to a promotional $0.75/$3.75 published as running through 2026-12-31, with the standard $1.50/$7.50 resuming 2027-01-01.

    New models: claude-fable-5-1, claude-mythos-5-1, claude-mythos-5 and six legacy Claude models Anthropic still publishes rates for; gpt-6-astra; gemini-3.8-flash, gemini-3.7-flash, gemini-3.1-flash-lite; qwen3.8-27b on Groq; zai-glm-5-2 on Mistral; eleven more on Together; gemma-3-27b, gemma-3-12b and nemotron-3-super-120b on Bedrock.

    Newly published per-model detail fills in fields previously omitted for want of a source: cachedInput on eight Gemini models, three Mistral models and three OpenRouter models, cacheWrite on OpenRouter's Claude entry, and batchMultiplier on the two Gemini Pro models, three Mistral models and OpenAI's -pro tiers. The UNCERTAIN flag on openrouter:anthropic/claude-sonnet-5 is withdrawn: the rate it questioned turned out to be correct.

mend-json@1.2.0

Choose a tag to compare

@github-actions github-actions released this 07 Sep 15:35
1c106a5

Minor Changes

  • 566adce: Add JsonMendResult.pending: which part of the document is still incomplete, as { path, kind }. path is object keys and array indexes from the root down, so it indexes straight into value; kind distinguishes a scalar still arriving ('string', 'number', 'literal') from an object key still being read ('key') or a key whose value has not started ('member'). It is undefined when nothing is incomplete - the snapshot is complete, no value has started, the scanner sits between values, or it froze on invalid input. Costs one O(depth) walk of the existing scanner stack, no extra scanning.

  • 24d3151: Add JsonMendResult.removedRanges: the half-open [start, end) ranges cut out from inside the valid prefix, sorted and non-overlapping. This completes the identity appendedSuffix documents - input.slice(0, validPrefixLength), minus removedRanges, plus appendedSuffix, is exactly repairedJson - which previously had a silent exception under duplicateKeyPolicy: 'first', where a caller slicing the raw buffer itself would render a member the package reports as dropped. Empty under every other policy and repair.

    Also fixes an aliasing bug this exposed: merging exclusion ranges widened a tuple the scanner still owned, mutating scanner state from a call documented as read-only.

Patch Changes

  • 1460e14: incompleteScalarPolicy: 'best-effort' no longer turns a contradicted number into a valid one. Inputs like {"a":01}, {"a":1.x} and {"a":1e+x} are syntax errors rather than truncations, so they are now omitted under 'best-effort' exactly as under 'omit' - matching the behaviour already in place for contradicted literals such as tru5. Genuine truncation (1., 1e+) still trims back to the typed digits with a number-truncated diagnostic.

token-chunk@1.2.0

Choose a tag to compare

@github-actions github-actions released this 01 Sep 16:53
fb41c6a

Minor Changes

  • 628c1b7: Add TextChunk.syntheticPrefixLength: number, how many leading characters of chunk.text were synthesized rather than sliced out of the input (a rendered heading breadcrumb, a repeated table header, or both) — so chunk.text.slice(chunk.syntheticPrefixLength) is exactly the documented source reconstruction, and a consumer can strip the non-source-backed part before hashing, deduplicating or mapping a position back to an input offset without re-deriving the prefix. It is 0 when nothing was prepended, including when a prefix was dropped to keep the chunk within budget.

Patch Changes

  • 0c727b4: Keep a table row's trailing newline when the row itself has to be split. The
    row unit was built from the bare line, one character short of the extent
    bodySpans records, so that newline landed in no chunk's text and in no
    chunk's source range: the chunk stopped reconstructing its own source slice,
    and the row ran straight into whatever unit packed after it.
  • eba51b9: Stop dropping a split table's header row. When the header fit the budget on its
    own but not alongside the first body row, that row was split out on a path that
    bypassed the only code emitting the header as text — so the header and
    separator lines appeared in no chunk's text and in no chunk's source
    range, leaving retrieved rows with no column names. The header is now emitted
    once as its own source-backed chunk, and every fragment of that row repeats it
    as a synthetic prefix, as documented.

vec-cache@1.1.0

Choose a tag to compare

@github-actions github-actions released this 28 Aug 13:42
50c33b3

Minor Changes

  • cf49574: getOrCreate now checks an explicit dimensions request against what embed
    actually returns, and passes the requested width to the callback as
    request.dimensions. A callback that ignores it and returns another width
    throws EMBED_DIMENSION_MISMATCH before anything is written, instead of
    storing a row keyed as the requested width and stored at another — which made
    every later identical call demote the row and pay for the same embedding
    again.
  • a1b0c2f: Add getOrCreate's maxEmbedBatchSize option: a cap on how many texts go to
    embed in one call, so a cold cache over a large corpus no longer hands the
    callback more inputs than the provider accepts. Sub-batches run in order and
    each is cached as it succeeds — a later failure leaves the completed ones
    cached and throws — and report.embedCallCount is the number of calls
    actually made. Omitting it keeps the previous behavior: one call for every
    unique miss.

Patch Changes

  • 09f228c: Honor busyTimeoutMs while the database is being opened. It was applied only
    after the header probe, the migration reads and the WAL switch, so until then
    the driver's own 5 s default was in force and a larger configured timeout was
    silently capped — losing the race against another process checkpointing its
    WAL on close().

llm-backoff@1.1.0

Choose a tag to compare

@github-actions github-actions released this 24 Aug 14:56
56c19ec

Minor Changes

  • f88aef3: Add attemptTimeoutMs, a per-attempt ceiling whose timeout is retryable.

    A hung call — connection established, tokens never arrive — is the failure this package could not recover from: maxElapsedMs is only checked between attempts, so one hung attempt quietly eats the entire budget. With attemptTimeoutMs set, an attempt that outlives it is cancelled and retried through the normal delay machinery, and surfaces as an AttemptTimeoutError on LlmBackoffError.cause if every attempt hangs.

    Wiring this up yourself does not work, which is why it belongs in the package: an AbortSignal.timeout() inside your operation produces a TimeoutError with no status, so the classifier reads it as a non-retryable failure and gives up after one attempt. Only the retry loop can tell its own ceiling apart from a cancellation the caller asked for.

    options.signal keeps absolute precedence — if both fire on the same attempt, your abort reason propagates unwrapped and unretried, exactly as before. The ceiling reaches your operation through RetryContext.signal (combined with options.signal), so forwarding that signal cancels the in-flight request rather than leaving it holding a connection; fetchWithLlmBackoff already forwards it. A failure that arrived before the ceiling is still classified as itself — a 400 under a 30 s ceiling is still non-retryable. Time spent on a timed-out attempt counts toward maxElapsedMs, and the timer is armed through the injectable sleep, so an injected clock still sees no real timer.

    The option is off by default and nothing changes for callers who do not set it. AttemptTimeoutError is a new export.

Patch Changes

  • 1bee72f: Honor the AbortSignal a Request input carries. fetchWithLlmBackoff combined options.signal and init.signal but not the signal every Request has, and it passes the combined signal on to fetch in the init bag — which, per the fetch spec, replaces the Request's own signal. So fetchWithLlmBackoff(request, undefined, { signal }) stopped honoring request's controller entirely: aborting it no longer cancelled even the in-flight network call, something plain fetch(request) does for free.

    With no other signal supplied, the failure was quieter but worse. The retry loop's signal was undefined, so an abort during a Retry-After sleep did nothing until the sleep elapsed — up to the full server-advised delay — and the eventual rejection was then classified as a non-retryable failure and wrapped in LlmBackoffError, breaking the documented contract that an abort propagates immediately and unwrapped with the exact reason object you passed to abort().

    All three channels are now combined, so an abort through any of them cancels the whole retry operation at every phase — before the first attempt, mid-fetch, and mid-sleep — and surfaces unwrapped.

  • 2427b57: Detect every single-shot request body, not just ReadableStream. Node's fetch also accepts an async iterable — an async generator, or a node:stream Readable, which is one — and a generator object, and turns each into a stream internally. All of them are drained by the first attempt, but fetchWithLlmBackoff's guard only recognized a ReadableStream, so the exact situation this package promises to detect and refuse happened silently instead: attempt 1 sent the payload, the retry sent zero bytes, and the server answered 200 with nothing to signal that the body had vanished.

    Any of those bodies now throws LlmBackoffError (REQUEST_BODY_NOT_REPLAYABLE) before the first request is made, with the same remediation — buffer into a string/Blob/ArrayBuffer/Uint8Array, or pass maxAttempts: 1.

    The test is single-shot-ness, not iterability: FormData, URLSearchParams and an array or Set of chunks are all iterable and all still retry normally, because fetch asks each of them for a fresh iterator on every attempt.

usage-tab@1.1.0

Choose a tag to compare

@github-actions github-actions released this 21 Aug 17:17
9e675e0

Minor Changes

  • 6b8eca2: Add sumExactUsd and createCostAggregator — exact totals across many requests. The exact decimal channel previously dead-ended at a single request's totalUsdExact: a caller wanting a daily or monthly figure — the package's namesake use case — either wrote sum += breakdown.totalUsd, reintroducing float drift at exactly the boundary the package pushed them past, or hand-parsed decimal strings. Both new functions sum on the same bigint fixed-point path every individual cost is already computed with, and reject anything that is not a non-negative decimal string (InvalidRateError) rather than coercing it. createCostAggregator also reports count, per-model totals keyed by resolved provider:canonicalModel, and every distinct registryVersion the aggregate drew on, so a total that silently mixes pricing snapshots is visible.

    Also fix parseDecimalRate throwing a raw TypeError, outside the package's error taxonomy, for a non-string rate that reached it past the type — a hand-built override or a JSON round-trip. RegExp.test stringifies its argument, so a numeric 1.5 passed the decimal pattern and then failed on indexOf. It now throws InvalidRateError like every other malformed rate.

  • da64bf2: Read every ISO at string as UTC, including the offset-less form. calculateCost passed at to Date.parse, where a datetime without a UTC offset ("2026-09-01T05:00:00" — a log timestamp, a datetime-local input, several DB drivers) is local time by specification, while an ISO date and an offset-carrying datetime are both UTC. Pricing periods start at UTC midnight, so the local-time reading landed on either side of a price restatement depending on the host: the identical historical lookup priced at $12.00 in one deployment and $18.00 in another, silently breaking the reproducibility a dated, committed price registry exists to provide. Such values are now normalized to UTC — the reading every other accepted ISO form already got — rather than inheriting the process's timezone. The same normalization applies to effectiveFrom/effectiveTo on a caller-supplied override, which never passed through the registry's schema validation.

    Non-ISO strings ("2026/09/01", "September 1, 2026") are implementation-defined rather than specified, so there is no single reading to normalize them to; they still go to Date.parse unchanged, and the documentation now says to pass a Date or an ISO form instead.

    Also fix an invalid Date in at escaping the error taxonomy: building the InvalidLookupDateError called toISOString(), which throws on an invalid date, so callers saw an uncoded RangeError instead. It now throws InvalidLookupDateError (INVALID_LOOKUP_DATE) like the equivalent string.

Patch Changes

  • 3cbd4c7: Fix a provider-qualified lookup silently resolving duplicate entries by array order instead of reporting ambiguity. When two caller-supplied overrides (or two entries of a caller-supplied registry) shared the same provider and canonicalId — the shape produced by merging an org-wide price list with a team's — the exact-canonical step took whichever came first, so the identical request could price differently depending on concatenation order. Supplying the provider qualifier, which the documentation tells callers to do, therefore made ambiguity detection weaker than the unqualified lookup, which already raised AMBIGUOUS_ALIAS. Both channels now raise AMBIGUOUS_ALIAS with every duplicate listed as a candidate. Well-formed data, where provider and canonical id are unique, is unaffected.