feat(tb-lf): add flag-cohort-stratified subcommand#19
Merged
Conversation
New CLI subcommand that consumes the stratified_flag_stats DevPortal endpoint. Shows per-cohort ON vs OFF comparison with flag diffs between cohorts, controlling for confounding from other feature flags. - --max-cohorts param (default 3, sorted by size) - --min-cohort-size param (default 10) - --detail metrics|traces modes - Flag diff display (green +flag / red -flag between cohorts) - Delta cost summary per cohort - Extracts print_cohort to shared function (was inline in flag-cohort) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document simple vs stratified flag-cohort analysis, when to use each, how to interpret stratified results (cohort diff, delta consistency, interaction effects), and the recommended 4-step workflow. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tb-lf flag-cohort-stratifiedsubcommand that consumes thestratified_flag_statsDevPortal endpoint (productiveio/devportal#254)print_cohortto shared function (was inline inflag-cohorthandler)Usage
Features
--max-cohorts(default 3) — sorted by cohort size, largest first--min-cohort-size(default 10) — minimum traces per cohort--detail metrics|traces— metrics mode shows cost/latency/errors, traces mode shows trace IDs+flag/ red-flag)Test plan
cargo build)🤖 Generated with Claude Code