srbf 0.21.0

Added

• Operon as a subprocess worker (worker: operon): pyoperon 0.6.1's SymbolicRegressor in the configuration
  its first author published for benchmarking (NSGA-II on R² and length, one Levenberg–Marquardt step per individual,
  population 1,000, length at most 50, depth at most 10, MDL selection from the Pareto front), without its
  hyperparameter search. srbf sets the operators and a budget in evaluations (rung files evals_<n>.pkl); the whole
  Pareto front is stored.
• DSO as a subprocess worker (worker: dso; DSO v3.0.0 in its own Python 3.7 environment,
  scripts/envs/build_dso_env.sh): options.arm: dsr runs DSO's shipped regression defaults, udsr the uDSR paper's
  benchmark configuration (uDSR*: priority-queue training, GP-meld, the LINEAR token). Three patches, documented in
  docs/models.md: GP-meld evaluates the trees it breeds, LINEAR's inverse table gains neg and n4, and a GP-meld
  clone shares the primitive set. On the NGGP Nguyen anchor the uDSR* path recovers 14 of 15 runs.
• GP-GOMEA as a subprocess worker (worker: gpgomea): the original GP-GOMEA at commit 6a92cb6, the one SRBench 2021
  ran, in its first author's benchmark configuration. Its protected operators are written as the functions it
  computes, and the linear-scaling coefficients are recomputed in float64. On SRBench 2021's Feynman protocol it
  recovers 71.7 % (published: 70.9 %).
• RILS-ROLS as a subprocess worker (worker: rilsrols): rils-rols 1.6.7 in its first author's SRBench
  configuration, with an output-only patch that prints constants at full precision; the budget is fit calls
  (evals_<n>.pkl). On Feynman it recovers 81.7 % (published: 82.59 %).
• The Flash-ANSR oracle: with generation_config.method: oracle (flash-ansr's OracleConfig), the Flash-ANSR adapter
  hands the model each problem's ground truth as its only candidate, which shows how well the fitting does when the
  form is given. A ladder over the refiner's restarts (restarts_<n>.pkl,
  configs/evaluation/scaling/flash-ansr-v25.0-T8-oracle_srbf.yaml).
• srbf table judges result trees (every method, draw, catalog, rung and shard) into one CSV with every metric, in
  parallel and with a per-file cache. With scripts/site_timing.py (the time axis from reference-machine result files;
  --pattern names a method's own rung files) and scripts/catalog_mu.py, the results site is rebuilt from the result
  files with srbf alone.
• QLattice as a subprocess worker (worker: qlattice): feyn 3.5.0 through its authors' SRBench loop (200 epochs,
  at most 10 edges, wide_parsimony, squared error). srbf sets the operators, the budget (n_epochs, rung files
  epochs_<n>.pkl) and the seed; the answer is the first model the loop returns, written at full precision, with
  feyn's protected functions written out where they clip. scripts/envs/build_qlattice_env.sh builds a pinned
  Python 3.12 environment; feyn runs offline without a licence key (CC BY-NC-ND 4.0: research use). On SRBench 2025's
  first-principles data it reproduces the published QLattice accuracy: mean mid-rank 0.41 among the published runs
  (2 epochs: 0.07).
• The flash_ansr_hybrid adapter type is built in again. The hybrid method moved into flash-ansr 0.19
  (flash_ansr.hybrid); srbf's adapter hands it each problem through its evaluation path (HybridRegressor.solve)
  and records the answer like every adapter. Configs that named the private plugin (hybrid_adapter:build) use
  type: flash_ansr_hybrid. pip install srbf[hybrid] brings flash-ansr with PySR; flash-ansr 0.19 is allowed.
• srbf table stores the expressions it judged: predicted_expression (in the ground truth's variable names and
  the engine's spelling) and ground_truth_expression, as prefix tokens at full precision.
• The results site's Predictions view: one problem at a time, the true formula and every method's formula, typeset,
  with whether each recovered it. scripts/site_export_v2.py writes one file per method, problem set, budget,
  finished run and block of 500 problems; a finished file never changes.
• srbf status -c CONFIG: how far every run of a config is (done, started, not started, with the
  row counts), without loading a model; the exit code is 0 when every run is done.
  Benchmark.runs_from_config(..., build_adapter=False) is the same from Python.
• An adapter from your own package: model_adapter.type: mypackage.module:function names a builder
  function; nothing in srbf has to be edited for an in-process adapter.
• What a worker reports about itself is stored with the results: __meta__["worker"] holds the return
  value of the worker's info() (its interpreter, package versions, checkpoint).
• Symbolic recovery at three levels of masking. symbolic_recovery masks every number (the structure is
  the ground truth's); symbolic_recovery_mask_fittable masks only the fittable constants, so exponents and root
  indices have to be the ground truth's as well; symbolic_recovery_mask_none also asks for the constants, measured as
  numeric recovery is. Each level implies the one before it, and agreement at a stricter level is a witness
  at the looser one. The two new columns need an engine.
• precision_score, recall_score and edit_distance_norm among the derived metrics: the two parts of
  the token F1, and the edit distance over the length of the longer sequence.
• The documentation is tested against the package (tests/test_docs.py): every command, flag, adapter
  type, suite catalog and derived metric is documented; every documented flag, import and repository path
  exists; every example parses; and the site builds with broken links as errors.

Changed

• srbf no longer depends on flash-ansr, and its root is SRBF_ROOT. {{ROOT}} in a config stands for the
  SRBF_ROOT directory (the current directory when it is unset); FLASH_ANSR_ROOT means nothing to srbf any more,
  so a script that set only it must set SRBF_ROOT. srbf reads its configs and resolves {{ROOT}} itself
  (srbf.paths). flash-ansr is an optional install, pip install "srbf[flash-ansr]", needed only by the adapters
  built on it (flash_ansr, and flash_ansr_hybrid through srbf[hybrid]) and by the two baselines that fit their
  constants with its refiner (lample_charton, brute_force); they import it when they are built and say how to
  install it when it is missing. A run of any other method never imports it: the FVU the NeSymReS and hybrid adapters
  record is srbf's own srbf.metrics.fvu, and the run's provenance reads flash-ansr's version without importing it.
  Flash-ANSR checkpoints are downloaded with hf download <checkpoint> --local-dir "$SRBF_ROOT/models/<checkpoint>".
  CI runs the suite with and without flash-ansr, and a test imports every srbf module where flash-ansr cannot be
  imported.
• The results site averages over problem sets, with the problem as the unit. A problem's value is the mean of
  its runs, and the catalogs are combined by a random-effects average (Paule–Mandel between-catalog variance,
  intervals over these problem sets, rates on the logit scale, ratios as geometric means), so erbench-syneq no longer decides
  a pooled number on its own; tables and tooltips add where one more catalog would fall. Paired contrasts compare
  every run of one method with every run of the other and test over catalogs (rates on the difference, other metrics
  on the per-problem superiority); ranks are built from the pairwise chances to beat, with Holm-corrected pairwise
  tests in place of Friedman and the Nemenyi critical difference. Two predictions that both meet Numeric Recovery tie
  in comparisons, and a failed run counts as worst in the medians, distributions, ranks and comparisons of log10 FVU
  and R². The time axis averages the catalogs the same way. scripts/site_random_effects.py is the reference; the
  site suite runs the page's own code against it.
• log10_fvu is floored at the float64 epsilon (srbf.metrics.numeric.LOG10_FVU_FLOOR, (\log_{10} 2^{-52}
  \approx -15.65)). An exact fit used to be (-\infty), which every mean left out, while the same formula written
  another way landed at finite rounding noise anywhere down to about (-320), which every mean took in: on the
  2026-09 board, 22 % of T8-120M's usable predictions sat below the float64 epsilon and at budget 128 values below
  (-20) made 45 % of its mean's sum. Below the epsilon the unexplained variance is smaller than about one rounding
  unit of the variance it is divided by, so all of these are one value now, and an exact fit counts in a mean.
  Blow-ups stay (+\infty). The judge's fingerprint changes, so srbf table re-judges its cache.
• The results site is written for a first-time reader. Every hint, popover, metric definition, method note and
  protocol text, and the prose below the explorer, say what they mean in plain words ("problem set", "budget",
  "run"; no pipeline vocabulary), with the longer explanations moved to a from-scratch "How to read the results"
  section. The Numeric Recovery threshold is described as what it is (FVU at most 2^-23: a typical error of at most
  0.035 % of the values' spread). A method missing at a budget is said to be not run there, or not finished yet;
  the export carries each method's planned budgets for that. copy_lint.py and a rendered-text test in the site
  suite keep the pipeline's words out.
• A worker is handed the problem and nothing of the ground truth. meta carries the problem's identifiers and
  sampling parameters (benchmark_eq_id, eval_row_index, n_support, noise_level, the variable
  names); the ground truth's skeleton, expression, constants and complexity stay on srbf's side.
• Every method sees every column. drop_unused_variables (and the PySR block's padding, and
  remove_padding of the NeSymReS...

srbf 0.20.3

A second sign of a hang for out-of-process workers: overdue (a problem that runs many times longer than the problems before it), for workers that are stuck while busy. Opt-in via hang_overdue_factor; see the CHANGELOG.

Release notes written by Claude (Claude Code) on behalf of @psaegert.

v0.20.2

Added

• A hang policy for out-of-process workers (opt-in: hang_after_idle_s on a worker-backed adapter).
  While a problem is in flight, the worker's process tree is sampled from /proc; less than
  hang_idle_cpu_s CPU seconds (default 1) over hang_after_idle_s seconds of wall is a hang, and so is a
  missed hard timeout. The problem is retried once in a fresh worker and a second hang fails it. Every
  hang is written to hang_log (JSON lines: problem, attempt, kind, seconds, CPU in the window, outcome)
  and counted in the row's worker_hangs. Hang restarts do not spend max_restarts, which stays the
  budget for crashes. Without the key nothing changes.
• PySR on the whole suite, on the reference machine. scripts/make_pysr_suite_config.py writes one
  experiment per catalog with the data sources and runners of a Flash-ANSR scaling config, PySR with
  upstream defaults, the iteration count as the ladder, and the hang policy on (60 s);
  scripts/run_timing_ladder.py --full-suite runs such a config unit by unit on whole catalogs, and the
  solomon timing queue runs it (three runs, RUN_PYSR_SUITE=0 skips them).
• scripts/stop_timing_queue.sh: stops the timing queue without corrupting its marks (the queue first,
  processes matched by their exact command, any mark written while stopping rolled back).

v0.20.1

Fixed

• The flash_ansr_hybrid adapter builds again. Its builder read emission from the Flash-ANSR
  adapter, which has not carried it since the flash-ansr 0.17 port made emission a sampling policy
  of the flash_ansr block; every hybrid config failed at build time with an AttributeError. The
  builder now takes emission from the flash_ansr block, as the Flash-ANSR builder does.
• scripts/make_hybrid_config.py writes configs this release can run: adapter type
  flash_ansr_hybrid (not the retired flash_ansr_pysr); a pysr: block holding only
  PySRSettings fields, since PySR runs in-process (the worker-protocol keys and the --pysr-python,
  --worker-log and --engine options are gone); draws instead of choices; and
  ranking: {mode: mdl}, the two-part code, instead of the fixed mdl_strength: 1.0e-2.

Changed

• Timing read-out: failures do not count towards the time. scripts/timing_readout.py drops
  every row that carries an error or prediction_success: False, whether or not a fit_time was
  recorded, so the reported seconds are the mean over the problems a method answered. Before, a fit
  that returned without a surviving candidate was still counted.
• scripts/site_export_v2.py publishes no as-run wall clock: fit_time and generation_time
  are no longer release metrics; the only time a release carries is the reference-machine table.

Added

• scripts/timing_queue_solomon.sh: RUN_PYSR=0 and RUN_NESYMRES=0 defer the PySR-alone and
  NeSymReS rows of the timing queue.

v0.20.0

Changed

• flash-ansr 0.18: the two-part code is the ranking. Pin flash-ansr>=0.18,<0.19. Every shipped
  scaling config ranks with ranking: {mode: mdl} -- flash-ansr's two-part code
  (n/2) log2 FVU + bits (the score study's S1, owner ruling 2026-09-16) -- instead of the fixed
  mdl_strength: 1.0e-2 (S0) they carried before; a config that still names mdl_strength runs
  the fixed weight, so the old numbers stay reproducible on request. Result files record the
  ranking as before (mdl_strength: null = the two-part code).

v0.19.0

Changed

• flash-ansr 0.17 API. The Flash-ANSR adapter is built from refine, ranking and compute
  mappings (FlashANSR.load(..., refine=, ranking=, compute=)), the sampling policy from a
  SoftmaxSamplingConfig / PriorSamplingConfig (draws, emission, ...), and every problem runs
  through model.fit(X, y, variable_names=..., draws=..., complexity=...); candidate curves and the
  per-rank ledger are read from the returned FitResult (result.predict(X, rank=...)). Requires
  flash-ansr>=0.17,<0.18.
• draws replaces choices in every evaluation config (configs/evaluation/scaling/*.yaml,
  scripts/make_scaling_configs.py); a config still spelling choices is refused with a ValueError.

0.18.0

Fixed

• symbolic_recovery judges both sides through the same simplify. The ground truth's skeleton is
  simplified before the comparison (pow x1 / <c> <c> -> pow x1 <c>); a prediction whose canonical
  form is not strictly shorter was judged by its stored skeleton UNsimplified, so a prediction
  byte-identical to the ground truth (pow x1 / 2 3) was not exact. Every law with a rational
  exponent -- 396 of 6,660, 7 % of erbench-syneq -- was unjudgeable as exact, whatever the answer. The
  stored skeleton now goes through the same simplify on the fallback path. A second-stage metric over
  stored prefixes: re-derive, nothing needs re-running. Results judged under 0.17.x and 0.18.x are not
  comparable on those laws, which is what the minor bump marks.

Added

• Answer provenance in the flash_ansr adapter's rows: predicted_typed_frozen (predicted typed
  literals kept verbatim), predicted_typed_thaw (a thawed duplicate: the typed token indices it
  re-fitted; None otherwise) and predicted_spelling (the constant ladder's re-spelling record) for
  the rank-0 answer, from flash-ansr >= 0.16.1 (whose ladder children carry the thaw mark). Pin
  flash-ansr>=0.16.1,<0.17.

---

Release notes are the [0.18.0] section of CHANGELOG.md, assembled by Claude Code (https://claude.ai/code/session_01MktGcCWQs8zePRutL56sT4).

0.17.0

Changed

• Evaluates flash-ansr 0.16 (flash-ansr>=0.16,<0.17). flash-ansr 0.16.0 keeps the numbers a
  model predicts in a mixed emission (0.15.x post-processing deleted them from 72 % of valid
  candidates) and defaults to refiner_typed_spans='freeze_then_free', so every flash-ansr arm
  evaluated under this srbf draws from a different candidate pool than under 0.16.x; results
  across the two are not comparable, which is what the minor bump marks. Nothing in the driver,
  the metrics or the judge changed.

Added

• The diffsym worker (worker: diffsym, configs/evaluation/baselines/diffsym_fastsrb.yaml):
  Mara Eliana's discrete diffusion model (D3PM) conditioned on the support set, run out of process in
  its own interpreter (it pins simplipy 0.2.15 and its own torch). Per problem it samples n_samples
  token sequences, decodes/simplifies/validates them into prefix candidates, fits each candidate's
  constants with diffsym's own ConstantFitter and returns the best by R^2. Two translations sit
  between the two: the pre-0.12 vocabulary (pow2/pow1_3) is respelled with
  respell_legacy_prefix, and diffsym's positional x1..xn become the catalog's own variable names.
  A candidate naming a variable the problem does not have (the decoder's vocabulary is the
  checkpoint's x1..x8, not the problem's) is dropped and counted in diffsym_n_out_of_range rather
  than failing in the fitter. Smoked on the v4.0 checkpoint (119M parameters, step 1.7M): srbf check
  green on fastsrb, II.38.14 recovered at 32 draws.

---

Release notes are the [0.17.0] section of CHANGELOG.md, assembled by Claude Code (https://claude.ai/code/session_01MktGcCWQs8zePRutL56sT4).

0.16.1

Fixed

• The judged skeleton is the canonical form the prediction was priced as. flash-ansr < 0.15.2
  emitted a fitted candidate as its skeleton with the numbers filled in, so a factor the fit made
  cancel (exp(-x^2) * tanh(x)^2 / tanh(x)^2) or a constant the fit made fold stayed in the spelling
  while the certified price had collapsed it -- and symbolic_recovery, a masked-skeleton identity,
  missed the recovery. derive_metrics now judges the masked, simplified skeleton of the prediction's
  canonical form whenever that form is strictly shorter than the emitted one (the stored skeleton
  stays under predicted_skeleton_prefix_as_emitted), so files from before and after flash-ansr
  0.15.2 are judged alike. Measured on the T8-20M r = 0 readings (766 problems): symbolic recovery
  9.3 % as emitted -> 13.2 % canonical. Numeric metrics and the MDL price are unaffected.

0.16.0

Removed

• The hybrid arm's method code. srbf is the evaluation framework; a method does not live in it.
  srbf/hybrid_adapter.py (model_adapter.type: flash_ansr_pysr), scripts/make_hybrid_config.py,
  scripts/run_hybrid_sweep.py, scripts/hybrid_rescore.py and scripts/hybrid_curve.py are gone. The method -- Flash-ANSR
  generating by the clock until its share (1 - r) T, its top-K candidates seeding PySR, PySR on its own
  clock (its share minus the running means of its fixed cost and of the pricing that follows), its hall
  of fame re-fitted with flash-ansr's refiner, re-spelled by the constant ladder, priced and scored like
  Flash-ANSR's own candidates, Flash-ANSR's sorting picking rank 0 -- and its sweep tooling now live in
  the flash-ansr-hybrid package (0.1.0: HybridRegressor, flash-ansr-hybrid-make-config,
  flash-ansr-hybrid-run-sweep, flash-ansr-hybrid-rescore, flash-ansr-hybrid-curve). PySR runs in-process there, so the
  pysr: block no longer names an interpreter.

Added

• model_adapter.type: flash_ansr_hybrid (FlashANSRHybridAdapter): the thin adapter for the
  flash-ansr-hybrid regressor -- flash_ansr: (the model block), hybrid: (the regressor's
  HybridConfig fields plus snapshot_dir), optionally pysr: (its PySRSettings). It hands each
  problem's arrays to the regressor and records its answer with the usual FVU columns; nothing of the
  method is in srbf. The package is the hybrid extra (pip install "srbf[hybrid]"), imported lazily.