docs(submitqueue): add outcome predictor RFC - #682
Open
behinddwalls wants to merge 4 commits into
Open
Conversation
## Summary ### Why? Speculation ranks paths on the scorer's price, which only reflects the change (lines, files) and ignores what the run has since observed — a passed build, a failed build, batch state. The predictor design needs a written contract before the implementation lands. ### What? Adds doc/rfc/submitqueue/outcome-predictor.md: scorer vs predictor, the four YAML factors and what their values mean, which path results count as evidence, and rejected alternatives. Links it from doc/rfc/index.md. ## Test Plan Docs-only change.
This was referenced Sep 7, 2026
## Summary ### Why? The RFC's rejected-options list used implementation shorthand and included alternatives already explained elsewhere, making the design tradeoffs difficult to evaluate without branch context. ### What? Keep the four alternatives reviewers are likely to raise and state each rejected option, why it fails, and the selected design. Remove redundant and scorer-specific entries.
behinddwalls
force-pushed
the
preetam/outcome-predictor-rfc
branch
from
September 7, 2026 23:54
451805e to
3227132
Compare
behinddwalls
marked this pull request as ready for review
September 8, 2026 00:08
## Summary ### Why? The outcome predictor RFC described failed-path evidence and ranking differently from the path-set identity and best-first behavior, while older speculation RFCs still assigned dynamic pricing to the scorer. ### What? Document that prediction applies to unresolved dependencies, failed all-succeeds evidence applies once, neutral factors preserve exact scorer prices, and factors must be finite. Update the existing speculation RFCs to consistently describe predictor-based ranking and state-aware pricing.
Document per-queue factor inheritance and the shared batch/path-set snapshot consumed by best-first ranking.
behinddwalls
force-pushed
the
preetam/outcome-predictor-rfc
branch
from
September 8, 2026 00:57
01a9de2 to
9696e21
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why?
Speculation ranks paths on the scorer's price, which only reflects the change and ignores what the run has since observed — a passed build, a failed build, or batch state. The predictor design needs a written contract before the implementation lands.
What?
Add the outcome-predictor RFC covering scorer versus predictor, finite YAML factors and inheritance, which path results count as evidence, and rejected alternatives. Update the speculation and best-first RFCs so the shared batch/path-set snapshot and predictor-backed ranking are described consistently.
Test Plan
Docs-only change.
Issues
Stack