Skip to content

Releases: davccavalcante/bayesdecide

[PUBLISHED ON NPMJS] @takk/bayesdecide@1.0.0

Choose a tag to compare

@github-actions github-actions released this 22 Jun 03:41

STATUS: PUBLISHED ON NPMJS. This version was published to the npm registry on 2026-06-22T10:47:20Z with provenance attestation. View on npm: https://www.npmjs.com/package/@takk/bayesdecide/v/1.0.0

STATUS: REVIEW REQUIRED, NOT YET ON NPMJS. This GitHub Release was created by the release.yml workflow. The Creator must review the contents (tag, changelog, attached commit, pack-smoke result in the workflow logs) and then explicitly run npm-publish.yml to publish this version to the npm registry.

[1.0.0] - 2026-06-22T02:20:55Z

Initial stable release. A universal, zero-runtime-dependency NPM library and CLI for Bayesian multi-armed bandits over prompt variants, the decision layer for Massive Intelligence (IM) systems and non-human entities. BayesDecide runs the experiment for you: Thompson sampling routes more traffic to the prompt variant that looks best while staying calibrated, and a stopping rule promotes a winner once the evidence is decisive, instead of waiting for a fixed-N A/B test to finish.

Added

Bandit core

  • Beta-Bernoulli arm (@takk/bayesdecide/beta): BetaPosterior with mean, variance, mode, strength, cdf, quantile, credibleInterval, the closed-form conjugate update, observeReward (folds a reward in [0, 1] as fractional evidence, so a continuous LLM-as-judge score moves the belief in proportion), decayToward, sample, snapshot, and fromSnapshot. Built-in priors UNIFORM_PRIOR (Beta(1, 1)), JEFFREYS_PRIOR (Beta(0.5, 0.5)), and OPTIMISTIC_PRIOR (Beta(8, 2)), plus any custom prior, globally or per variant.
  • Gaussian arm (@takk/bayesdecide/normal): NormalPosterior, a Normal-inverse-Gamma conjugate over the unknown mean and variance for unbounded continuous rewards such as latency or cost. It updates online with Welford's algorithm so it never stores raw samples, draws a Gaussian Thompson sample, and reports the exact Student-t credible interval derived from the Beta quantile. Selected with rewardModel: "normal".
  • Special functions implemented from scratch with no dependency: lgamma via the Lanczos approximation, the regularized incomplete beta (the Beta CDF) and its inverse (the Beta quantile), plus seeded sampling (mulberry32, a standard normal, gamma, and Beta sampler) so every Thompson decision is reproducible and auditable.

Selection and best-arm identification

  • ThompsonSampler (@takk/bayesdecide/sampler): the "thompson" strategy draws one sample from every variant's posterior and serves the highest draw, balancing exploration and exploitation; the "greedy" strategy serves the posterior mean once you want to exploit a settled winner.
  • probabilityBest and leadingArm (@takk/bayesdecide/bestarm): the posterior probability that each variant is the best arm, estimated by Monte Carlo, the number the stopping rule reads.

Stopping rules, honest about what they guarantee

  • decideStop (@takk/bayesdecide/promotion): the default Bayesian sequential stopping rule. It stops the moment the posterior probability of being best clears a confidence threshold, subject to a minimum-trials floor and an optional maximum-trials cap. This is a sound decision heuristic, not an error-controlled test under repeated peeking.
  • confidenceSequence and decideStopAnytimeValid (@takk/bayesdecide/sequential): the anytime-valid counterpart, opted in with stopping: "anytime-valid". A Beta-Binomial mixture e-process whose coverage holds uniformly over time by Ville's inequality, so you may peek after every observation and still control the error rate. It declares a winner only when the leader's confidence-sequence lower bound clears every other arm's upper bound. Bernoulli rewards, time-decay disabled. Empirically coverage-verified in the test suite.

Promotion engine, advisory by design

  • promote returns the winning variant once a stopping condition is reached, otherwise null. It is advisory: it never mutates the experiment and never routes traffic on its own, so a human or policy stays in the loop.

Per-cohort categorical context

  • PosteriorStore (@takk/bayesdecide/store) keeps one posterior per (variant, cohort), the discrete form of a contextual bandit, so a variant that wins for one audience can lose for another without pooling the regimes. The store is model-aware (Beta or Gaussian), tracks trials, decays Beta evidence lazily on access when a half-life is set, and round-trips through a validated portable JSON snapshot. Feature-based contextual models are on the roadmap, not in 1.0.0.

Integration adapter and acceleration calculator

  • createPromptRouter (@takk/bayesdecide/adapter): a framework-agnostic, dependency-injected prompt router. It takes a generate function rather than importing any model SDK, so it binds an experiment to the Vercel AI SDK, Mastra, Genkit, a raw fetch, or a non-human entity's own generation loop without a single hard dependency. route picks a variant and returns its prompt, record folds a reward back in, and run closes the whole select, generate, score, observe loop.
  • estimateAcceleration (@takk/bayesdecide/calculator): runs the real bandit against a uniform equal-split A/B test over known true rates and returns the realized regret, traffic allocation, decision, and regret reduction. Every number it produces is the product of real execution.

Outcome evaluation, decay, and audit

  • @takk/bayesdecide/evaluator: binary, clampReward, fromThreshold, and normalize map raw signals to a reward in [0, 1], plus a Scorer type for an LLM-as-judge.
  • decayFactor (@takk/bayesdecide/decay): opt-in exponential time-decay relaxes stale evidence toward the prior with a configurable half-life, so a variant that won last quarter does not keep its lead on history alone.
  • @takk/bayesdecide/audit: an append-only AuditLog of selection, observation, and decision events with a tamper-evident SHA-256 hash chain you append to and verify, plus sha256Hex. It uses the Web Crypto API, not node:crypto, so the audit surface stays node-free. It is tamper-evident, any later edit is detectable, not an unalterable record or a digital signature.

Facade

  • createExperiment / Experiment wires the variant registry, the posterior store, the Thompson sampler, best-arm identification, and the stopping rule into one experiment. Methods: addVariant, variants, select, observe, pull (the one-call select, run, score, observe path), probabilityBest, best, report, decide, promote, snapshot, load, and reset. Config: variants, rewardModel, prior, perVariantPrior, strategy, stopping, confidence, alpha, minTrials, maxTrials, bestArmDraws, decay, seed, clock, and observer. The observer is the zero-dependency seam for telemetry and governance, and the experiment swallows observer errors so telemetry can never break a decision.

Persistence

  • @takk/bayesdecide/node ships createFileStore, a durable file-backed snapshot store using node:fs and node:path with atomic writes (write to a temporary file, then rename), so learning survives restarts with no database. It is the only entry point that touches a Node built-in.

CLI

  • Binary bayesdecide exposed via package.json#bin.
  • bayesdecide simulate compares the adaptive Thompson bandit to a uniform equal-split A/B test on variants with known true rates, running the real bandit and reporting the regret reduction; bayesdecide replay folds a whitespace-separated sequence of variant reward pairs from a file or stdin and prints the learned report and the stopping decision. --version and help round out the surface.
  • Standard sysexits-style exit codes: 0 ok, 64 usage, 65 data error, 66 missing input.

Distribution

  • Sixteen entry points, each a subpath export with split import/require conditions: the root facade plus beta, normal, adapter, calculator, registry, store, sampler, bestarm, promotion, sequential, evaluator, decay, audit, node, and edge.
  • Dual ESM + CJS bundles built with tsup 8, with separate .d.ts and .d.cts type files per entry point.
  • Node-free, platform-neutral core importable in Node, edge runtimes, and the browser; @takk/bayesdecide/edge re-exports it verbatim.
  • Zero required runtime dependencies. @takk/keymesh and @takk/modelchain are optional peer dependencies.

Documentation and examples

  • Complete project site (index.html, 404.html) and documentation set (README.md, SPEC.md, SECURITY.md, PRIVACY.md, CONTRIBUTING.md, RELEASING.md, CLA.md, CODE_OF_CONDUCT.md), with JSON-LD structured data, an Open Graph card, and a robots and sitemap pair.
  • Six runnable, offline, deterministic examples (prompt A/B/n, LLM-as-judge continuous rewards, the Gaussian arm on latency, anytime-valid stopping, the non-human-entity agent loop through the adapter, and per-cohort context).
  • A real regret benchmark (benchmarks/decide-benchmark.mjs) that runs the bandit against a uniform A/B test across five scenarios; on the committed seed the bandit cuts the uniform test's regret by 66 to 99.5 percent. Every number in the docs comes from real execution against the compiled dist.

Quality

  • 106 tests across 22 suites passing under Vitest 4, including an empirical coverage simulation that validates the anytime-valid guarantee under unlimited peeking.
  • Coverage: statements 96.21%, branches 91.36%, functions 96.89%, lines 97.56%.
  • Lint clean under Biome 2.5. Typecheck clean under TypeScript 6 in maximum strict mode (exactOptionalPropertyTypes, noUncheckedIndexedAccess, verbatimModuleSyntax, noPropertyAccessFromIndexSignature).
  • publint clean and @arethetypeswrong/cli green across all sixteen subpaths and package.json.
  • size-limit under budget on every bundle (brotli core 4.63 kB against a 7 kB limit, the Gaussian arm 1.42 kB, the anytime-val...
Read more