Skip to content

Refactor theta_prior into extensible Prior objects#9

Merged
blankjul merged 1 commit into
docsfrom
feat/saas-prior
Jul 7, 2026
Merged

Refactor theta_prior into extensible Prior objects#9
blankjul merged 1 commit into
docsfrom
feat/saas-prior

Conversation

@blankjul

@blankjul blankjul commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

What

Lifts the Dace MAP length-scale prior from a bare (mean, lam) tuple (hardcoded as a Gaussian ridge inside DaceProblem) into a small, extensible Prior abstraction:

  • Priorpenalty(Z) + grad(Z) in the encoded log10 search space; the extension point.
  • GaussianPrior(mean, lam) — the ridge lam·Σ(log10 θ − mean)², reproducing the tuple byte-for-byte.
  • resolve_priorNone → None, (mean, lam) → GaussianPrior, a Prior → itself.

DaceProblem now delegates the penalty and its gradient to the Prior. theta_prior (on Dace, and via the MAP selection) accepts either the tuple or a Prior object.

Why

theta_prior was the one un-abstracted knob in the otherwise-object-based Selection framework. This makes it extensible and byte-identical.

The backstory (honest)

This branch started as a SAAS sparse prior (SAASBO-style). I built it and benchmarked it, and found — and verified directly — that a MAP SAAS is not competitive: learned-τ MAP is degenerate (it prefers dense solutions — the classic MAP-of-a-hierarchical-model pathology, exactly why SAASBO uses full-Bayesian NUTS), and fixed-τ prefers sparse but its accuracy benefit is seed-dependent. Rather than ship an uncompetitive feature, I dropped SAASPrior/SAAS and kept only the clean, byte-identical Prior refactor. A genuine sparse prior returns with the planned fully-Bayesian GP work (HMC/NUTS + Bayesian model averaging), where marginalization — not optimization — is what makes it work; the Prior base is the seam it will plug into.

Safety

Pure refactor: theta_prior=(mean, lam) behaves identically, pure-MLE untouched. Golden 19/19 byte-identical, full pyclawd check green (523 tests).

Targets the docs integration branch.

🤖 Generated with Claude Code

…rior)

The Dace MAP prior on the length-scales was a bare (mean, lam) tuple hardcoded as a
Gaussian ridge inside DaceProblem. Lift it into a small Prior abstraction:

- dace/prior.py: Prior (penalty(Z) + grad(Z) in the encoded log10 space), GaussianPrior
  (the ridge lam*sum((log10 theta - mean)^2)), and resolve_prior (None -> None,
  (mean, lam) tuple -> GaussianPrior, a Prior -> itself).
- DaceProblem delegates the penalty and its gradient to the Prior. GaussianPrior
  reproduces the tuple's penalty and gradient byte-for-byte, so theta_prior=(mean, lam)
  is unchanged and the pure-MLE path is untouched -- golden 19/19 byte-identical.
- theta_prior (on Dace, and via the MAP selection) now also accepts a Prior object, and
  the Prior base is the extension point for future priors (e.g. a sparse prior for the
  planned full-Bayesian / SAASBO work, which needs marginalization -- a MAP sparse prior
  was found to be uncompetitive, so it is deferred rather than shipped).

Tests: resolve_prior dispatch; GaussianPrior penalty/gradient match the tuple exactly and
the gradient matches finite differences; a Prior object fits identically to the tuple
through Dace; MAP resolves to a GaussianPrior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCtCbiNgKky7iYHeT1ozdJ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant