Refactor theta_prior into extensible Prior objects#9
Merged
Conversation
…rior) The Dace MAP prior on the length-scales was a bare (mean, lam) tuple hardcoded as a Gaussian ridge inside DaceProblem. Lift it into a small Prior abstraction: - dace/prior.py: Prior (penalty(Z) + grad(Z) in the encoded log10 space), GaussianPrior (the ridge lam*sum((log10 theta - mean)^2)), and resolve_prior (None -> None, (mean, lam) tuple -> GaussianPrior, a Prior -> itself). - DaceProblem delegates the penalty and its gradient to the Prior. GaussianPrior reproduces the tuple's penalty and gradient byte-for-byte, so theta_prior=(mean, lam) is unchanged and the pure-MLE path is untouched -- golden 19/19 byte-identical. - theta_prior (on Dace, and via the MAP selection) now also accepts a Prior object, and the Prior base is the extension point for future priors (e.g. a sparse prior for the planned full-Bayesian / SAASBO work, which needs marginalization -- a MAP sparse prior was found to be uncompetitive, so it is deferred rather than shipped). Tests: resolve_prior dispatch; GaussianPrior penalty/gradient match the tuple exactly and the gradient matches finite differences; a Prior object fits identically to the tuple through Dace; MAP resolves to a GaussianPrior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MCtCbiNgKky7iYHeT1ozdJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Lifts the Dace MAP length-scale prior from a bare
(mean, lam)tuple (hardcoded as a Gaussian ridge insideDaceProblem) into a small, extensiblePriorabstraction:Prior—penalty(Z)+grad(Z)in the encodedlog10search space; the extension point.GaussianPrior(mean, lam)— the ridgelam·Σ(log10 θ − mean)², reproducing the tuple byte-for-byte.resolve_prior—None → None,(mean, lam) → GaussianPrior, aPrior → itself.DaceProblemnow delegates the penalty and its gradient to thePrior.theta_prior(onDace, and via theMAPselection) accepts either the tuple or aPriorobject.Why
theta_priorwas the one un-abstracted knob in the otherwise-object-based Selection framework. This makes it extensible and byte-identical.The backstory (honest)
This branch started as a SAAS sparse prior (SAASBO-style). I built it and benchmarked it, and found — and verified directly — that a MAP SAAS is not competitive: learned-
τMAP is degenerate (it prefers dense solutions — the classic MAP-of-a-hierarchical-model pathology, exactly why SAASBO uses full-Bayesian NUTS), and fixed-τprefers sparse but its accuracy benefit is seed-dependent. Rather than ship an uncompetitive feature, I droppedSAASPrior/SAASand kept only the clean, byte-identicalPriorrefactor. A genuine sparse prior returns with the planned fully-Bayesian GP work (HMC/NUTS + Bayesian model averaging), where marginalization — not optimization — is what makes it work; thePriorbase is the seam it will plug into.Safety
Pure refactor:
theta_prior=(mean, lam)behaves identically, pure-MLE untouched. Golden 19/19 byte-identical, fullpyclawd checkgreen (523 tests).Targets the
docsintegration branch.🤖 Generated with Claude Code