Skip to content

ref(utils): Small SafeRolloutComparator refactors#116257

Merged
lobsterkatie merged 3 commits into
masterfrom
kmclb-more-small-rollout-comparator-refactors
May 27, 2026
Merged

ref(utils): Small SafeRolloutComparator refactors#116257
lobsterkatie merged 3 commits into
masterfrom
kmclb-more-small-rollout-comparator-refactors

Conversation

@lobsterkatie
Copy link
Copy Markdown
Member

This makes a few small changes to the SafeRolloutComparator code.

  • For the initial round of span-first detector testing, only the comparison part of the comparator is needed. This therefore splits the check part of check_and_choose off into its own compare method.

  • In cases where only the comparison is being used, it's not accurate to say that either the experimental or control data is being used. Therefore, "both" and "neither" have been added as allowable source_of_truth tag values, to account for (as the names would suggest) both results being used or both results being discarded.

  • Finally, now that it's part of the compare method rather than the check_and_choose method, for new comparators the SafeRolloutComparator.check_and_choose metric has been renamed to SafeRolloutComparator.compare. (The one existing comparator now has a flag which preserves the old behavior, so as not to break existing dashboards.)

h/t Claude for his help with parts of these changes

@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label May 26, 2026
@lobsterkatie lobsterkatie marked this pull request as ready for review May 26, 2026 23:53
@lobsterkatie lobsterkatie requested review from a team as code owners May 26, 2026 23:53
@lobsterkatie lobsterkatie merged commit 0a64492 into master May 27, 2026
70 checks passed
@lobsterkatie lobsterkatie deleted the kmclb-more-small-rollout-comparator-refactors branch May 27, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants