Skip to content

Quiver 1.5.0

Latest

Choose a tag to compare

@waynewbishop waynewbishop released this 29 Jun 23:16

Overview

Quiver 1.5.0 turns the loose retrieval calls of 1.4.0 into concrete design pattern. EmbeddingIndex is an on-device vector store: embed each entry once at ingest, then rank the whole corpus against a query with a single retrieve(_:k:) call — and because the index is Codable, it persists to disk and loads at the next launch. The Chunker protocol completes the front of that pipeline, cutting a document into attributable fragments that flow straight into the index, while every value behind a ranking — the full score field, its mean and standard deviation, the stored vectors — stays reachable, so the geometry is never hidden behind the convenience.

On the numerical side, luDecomposed() factors a square matrix once and then solves A · x = b for many right-hand sides cheaply, the O(n²) substitution replacing another O(n³) factorization each time. A new zScore(of:) standardizes a single value against an array, and the Coefficients protocol gained equation() so any linear model prints itself as a readable formula. Everything is additive over 1.4.0 — no existing call changed, no migration required, and the suite grew from 749 to 809 passing tests.


Feature — On-device vector store

Holding the embed → score → rank pipeline as one value instead of orchestrating the loose calls by hand, with the full score distribution kept visible behind every ranking.

  • EmbeddingIndex<Label>(embedder:) — a vector store generic over its payload; store String, Chunk, or any Codable & Equatable type. The embedder is text-to-vector only and never sees Label.
  • EmbeddingIndex.add(_:label:) — embeds the text and stores the label beside its vector; an add(vector:label:) overload stores a precomputed vector directly.
  • EmbeddingIndex.retrieve(_:k:) — scores every entry against the query and returns the closest k as a RetrievalResult.
  • EmbeddingIndex.remove(label:), remove(labels:), removeAll() — manage the corpus after ingest.
  • EmbeddingIndex.vectors, vector(for:), scores(for:) — read the stored geometry directly; the index conforms to Sequence over its entries.
  • EmbeddingIndex.snapshot / init(_:embedder:) — a Codable snapshot persists the index to disk and rehydrates it with a fresh embedder.
  • RetrievalResult<Label> — the hits for the common case alongside the full scores field with its mean, standardDeviation, and topZScore; isAboveGate(floor:outlierZ:) lets a caller apply their own relevance rule.
  • RetrievedHit<Label> — the labeled, named form of the (rank, label, score) tuple, with the raw cosine score carried through untouched.

The index ranks and exposes its math; it does not judge relevance. Whether the top hit is good enough to act on is the caller's decision, and whether the retrieved passages answer the question is a downstream language model's job.

Feature — Document chunking

Cutting a document into retrievable fragments that stay traceable to where they came from.

  • The Chunker protocol names a single operation: text in, ordered [Chunk] out. Quiver ships the contract, not a chunker — where a document is cut depends on the content, while everything downstream stays the same.
  • Chunk — a fragment paired with the index it was cut at, so a retrieved fragment stays attributable to its place in the source. Codable and Sendable, so it persists to disk and crosses task boundaries.
  • String.chunked(using:) — applies a chunker to a document.
  • Sequence.asChunks() — trims, drops empties, and numbers raw pieces, so a conformer cuts the string and hands the pieces to the helper. It is a convenience, not a requirement: a strategy that overlaps or numbers differently writes its own loop and conforms just as well.

Chunker mirrors Embedder one step earlier in the pipeline — both are one-method contracts Quiver defines but does not implement, so swapping paragraph splitting for sentence splitting changes one line and leaves every downstream line untouched.

Feature — LU decomposition

Factoring a square matrix once, then solving against it many times cheaply.

  • Array.luDecomposed() (on [[Double]]) — computes the P · A = L · U factorization with partial pivoting in O(n³); throws MatrixError.notSquare or MatrixError.singular.
  • LUDecomposition.solve(_:) — solves A · x = b for a right-hand side in O(n²), reusing the one factorization across many systems.
  • LUDecomposition.determinant — the determinant read directly off the triangular factors.
  • MatrixError — a dedicated error type for the square/singular preconditions.

Feature — Standardizing a single value

  • Array.zScore(of:ddof:) (on numeric arrays) — returns how many standard deviations a value sits from the array's mean, or nil when there is no spread to measure against. Defaults to the sample standard deviation (ddof: 1); pass ddof: 0 for population.

Feature — Models as equations

  • Coefficients.equation() — renders a fitted linear model as a readable formula, intercept first, then each weight paired with its variable in input order (a bare x for one feature, subscripted x₁, x₂, … for several). Zero weights drop their term, a weight of exactly one renders as the bare variable, and a negative weight joins with -. LinearRegression, Ridge, and GradientDescent all gain it from the protocol.

Documentation

  • Model Persistence (Models) — saving and reloading a fitted index and model to disk.
  • Rendering Math Primer — gained a Models as equations section connecting equation() to the polynomial-rendering conventions already covered.
  • Retrieving Context for Generation — the RAG worked example rebuilt around EmbeddingIndex and Chunker, replacing the hand-orchestrated pipeline with the packaged store.
  • Embedding Sources, Building an Effort Model, and the Model Interpretation, Regularization, and Gradient Descent pages picked up cross-links and a corpus-wide standards-and-editorial pass.