docs(pd-conjugate): record #276 end-to-end optimization result (16.1% improvement)#282
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Updates the pd-conjugate system documentation to record the issue #276 end-to-end optimization outcome and to reflect the post-loader-refactor ratio and benchmark metrics.
Changes:
- Updates the reported JaxLoss/ObjectiveFunction ratio to 0.985 and refreshes the benchmark summary table (initial/final ObjectiveFunction + improvement).
- Replaces the per-category table with published vs optimized R² columns.
- Adds an “End-to-end optimization (issue #276 follow-up)” section documenting the experiment conclusion.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ericchansen
force-pushed
the
experiment/pd-conjugate-no-ratio-tol
branch
from
ea4cef8 to
768bbfd
Compare
… improvement) Issue #276 asked whether bypassing the JaxLoss ratio gate (`--ratio-tol none`) would unlock useful optimization for pd-conjugate at its pre-refactor borderline ratio of 1.20. The loader API refactor (#281) already resolved the underlying surrogate mismatch — pd-conjugate's ratio is now 0.985 (passing the default [0.85, 1.15] gate without intervention). Running the experiment with `--ratio-tol none` confirms a 16.1% real ObjectiveFunction improvement (8.61e6 → 7.23e6), well past the 5% decision threshold from the issue. No change to the default ratio_tol is recommended — the gate was diagnosing a real problem (the overwritten OPT values), and the fix lived in the loader, not in the gate threshold. Updates docs/systems/pd-conjugate.md with: - Real ratio (0.985, was 0.96 estimate) - Initial/final ObjectiveFunction scores - Per-category R² for the optimized FF - A short "End-to-end optimization (issue #276 follow-up)" section with the conclusion Optimized FF + validation_results.json + paper_metrics.json are committed to ericchansen/q2mm-data on a companion branch. Closes #276. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves issue #276 (pd-conjugate end-to-end optimization with
--ratio-tol none).Companion data PR: ericchansen/q2mm-data#5
Background
#276 was filed against the pre-refactor convergence baseline where pd-conjugate's JaxLoss/ObjectiveFunction ratio sat at 1.20 — borderline above the [0.85, 1.15] gate. The hypothesis was that bypassing the gate via
--ratio-tol nonewould unlock useful optimization.What changed
The loader API refactor (#281) resolved the underlying surrogate mismatch by no longer overwriting Wahlers OPT values with raw QFUERZA projections. As a result, pd-conjugate's ratio is now 0.985 — comfortably inside the default gate. The
--ratio-tol noneflag in this experiment is effectively a no-op.Result
End-to-end optimization (700 s, 2 evaluations):
Per-category R² (published → optimized):
Decision
Per the issue's decision rule (>5 % improvement = success), this exceeds threshold. No change to the default
ratio_tolis recommended — the gate was diagnosing a real problem (the overwritten OPT values), and the fix lived in the loader, not in the gate threshold.This PR
docs/systems/pd-conjugate.mdwith the actual ratio (0.985, was 0.96 estimate), pre/post optimization scores, and a short "End-to-end optimization (issue Experiment: pd-conjugate optimization with ratio_tol=None (borderline ratio 1.20) #276 follow-up)" section.Closes #276.Provenance
8c7f898(master, post-loader-refactor)65faf95null["cuda:0"]Raw artifacts (optimized FF, validation_results.json, paper_metrics.json) live in the companion data PR.