You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
No experiment reached PROMOTE or REJECT. Core decisions are reported unchanged. The interaction safety hold is not triggered.
Top-level counts match the 2026-11-20 history record (51 workflows, 54 analyses, 45 EXTEND, 9 INCONCLUSIVE). Per-experiment sample counts were not compared to that record.
Run data: last 10 assignment-bearing runs per workflow (395 total). 331 (83%) joined to GitHub run conclusions. Full 30-run statistics need artifact-based assignment inference and are not included.
Key Metrics
Success rates are descriptive context only. The core decision is the canonical conclusion.
Canonical decisions (all 54 analyses)
Workflow
Experiment
Decision
Reason
Readiness
Runs
min_samples
Issue
agentperformanceanalyzer
prompt_compression
EXTEND
guardrail_unsupported
READY
183
14
#33280
agentpersonaexplorer
sub_agent_strategy_prefix_v1
EXTEND
insufficient_observations
COLLECTING
1
14
none
architectureguardian
sub_agent_strategy
EXTEND
insufficient_samples
COLLECTING
38
20
none
auditworkflows
audit_decomposition
EXTEND
insufficient_samples
COLLECTING
137
264
#43177
awfailureinvestigator
tone_variant
EXTEND
insufficient_observations
COLLECTING
1
50
#36105
blogauditor
prompt_style
EXTEND
insufficient_samples
COLLECTING
19
20
#32603
breakingchangechecker
tone_variant
EXTEND
insufficient_samples
COLLECTING
87
194
#42467
cicoach
prompt_style
EXTEND
guardrail_unsupported
READY
136
20
#32335
copilotprnlpanalysis
nlp_prompt_style
EXTEND
insufficient_samples
COLLECTING
1
30
#63156
dailyagentrxtraceoptimizer
sub_agent_strategy
EXTEND
guardrail_unsupported
READY
124
20
none
dailyarchitecturediagram
detail_level
EXTEND
guardrail_unsupported
READY
25
10
#31926
dailyastrostylelitemarkdownspellcheck
prompt_style
EXTEND
guardrail_unsupported
READY
200
30
none
dailycachestrategyanalyzer
model_size
EXTEND
guardrail_unsupported
READY
75
20
none
dailycavemanoptimizer
model_size
EXTEND
guardrail_unsupported
READY
130
20
none
dailycommunityattribution
prompt_style
EXTEND
insufficient_observations
READY
186
20
none
dailydochealer
model_size
EXTEND
guardrail_unsupported
READY
122
20
none
dailydocupdater
model_size
EXTEND
guardrail_unsupported
READY
107
20
none
dailyfact
reasoning_depth
EXTEND
guardrail_unsupported
READY
137
30
#31324
dailyfunctionnamer
model_size
EXTEND
insufficient_observations
COLLECTING
3
20
none
dailynews
prompt_style
EXTEND
guardrail_unsupported
READY
152
30
#31190
dailyrenderingscriptsverifier
remove_redundant_context_v1
EXTEND
insufficient_samples
COLLECTING
1
20
none
dailysafeoutputoptimizer
log_fetch_strategy
EXTEND
insufficient_observations
COLLECTING
1
20
#38094
dailysecurityredteam
reasoning_depth
EXTEND
insufficient_observations
COLLECTING
1
30
#31673
dailystorify
deterministic_precondition_v1
EXTEND
insufficient_observations
COLLECTING
1
20
none
dataflowprdiscussiondataset
caveman_mode
EXTEND
guardrail_unsupported
READY
24
10
#37102
dependabotcampaign
summary_detail
EXTEND
insufficient_samples
COLLECTING
12
20
none
gofan
prompt_tightening_v1
EXTEND
insufficient_samples
COLLECTING
1
20
none
gpclean
tool_verbosity
EXTEND
guardrail_unsupported
READY
180
30
none
issuearborist
prompt_style
EXTEND
guardrail_unsupported
READY
140
30
#30015
prsouschef
remove_redundant_context_v1
EXTEND
insufficient_observations
COLLECTING
1
20
none
smokeantigravity
sub_agent_strategy
EXTEND
insufficient_observations
READY
208
20
none
smokecopilot
caveman
EXTEND
insufficient_observations
READY
443
20
none
smokecopilot
subagent_model
EXTEND
insufficient_observations
READY
423
20
none
smokecopilotaoaiapikey
caveman
EXTEND
insufficient_observations
READY
235
20
none
smokecopilotaoaiapikey
subagent_model
EXTEND
insufficient_observations
READY
235
20
none
smokecopilotaoaientra
caveman
EXTEND
insufficient_observations
READY
208
20
none
smokecopilotaoaientra
subagent_model
EXTEND
insufficient_observations
READY
208
20
none
smokecopilotsubagents
sub_agent_strategy
EXTEND
insufficient_samples
COLLECTING
63
30
#47551
smokegemini
sub_agent_strategy
EXTEND
guardrail_unsupported
READY
318
30
none
smokepi
sub_agent_decomposition
EXTEND
insufficient_observations
READY
76
20
none
smokeproject
prompt_style_test
EXTEND
insufficient_observations
COLLECTING
1
20
#37302
smoketemporaryid
sub_agent_strategy
EXTEND
insufficient_observations
COLLECTING
1
20
none
testqualitysentinel
model_size
EXTEND
insufficient_samples
COLLECTING
16
20
none
typist
tone_style
EXTEND
guardrail_unsupported
READY
130
10
#34032
weeklyblogpostwriter
prefetch_strategy
EXTEND
guardrail_unsupported
READY
23
10
#38590
copilotagentanalysis
output_format
INCONCLUSIVE
unsupported_multi_variant
COLLECTING
125
30
none
dailycodemetrics
output_format
INCONCLUSIVE
unsupported_multi_variant
READY
159
20
#1
dailycompilerquality
output_format
INCONCLUSIVE
unsupported_multi_variant
READY
149
20
#32390
dailyissuesreport
output_format
INCONCLUSIVE
unsupported_multi_variant
COLLECTING
157
30
#30573
dailysemgrepscan
semgrep_output_format
INCONCLUSIVE
unsupported_multi_variant
COLLECTING
77
30
#32795
dailysubagentoptimizer
timeout_setting
INCONCLUSIVE
unsupported_multi_variant
COLLECTING
4
20
none
deepreport
output_format
INCONCLUSIVE
unsupported_multi_variant
READY
291
15
none
dependabotgochecker
prompt_style
INCONCLUSIVE
unsupported_multi_variant
COLLECTING
76
30
none
plan
reasoning_depth
INCONCLUSIVE
unsupported_multi_variant
COLLECTING
7
137
#42941
Descriptive per-variant stats (recent joined runs)
Workflow
Experiment
Variant
n
Success
Mean duration (s)
agentperformanceanalyzer
prompt_compression
caveman
7
86%
978
agentperformanceanalyzer
prompt_compression
verbose
3
67%
613
agentpersonaexplorer
sub_agent_strategy_prefix_v1
per_scenario
1
100%
747
auditworkflows
audit_decomposition
phased_sub_agents
5
0%
1473
auditworkflows
audit_decomposition
single_agent
5
40%
1540
awfailureinvestigator
tone_variant
narrative
1
100%
490
blogauditor
prompt_style
concise
4
100%
643
blogauditor
prompt_style
detailed
6
67%
489
breakingchangechecker
tone_variant
neutral
4
25%
578
breakingchangechecker
tone_variant
urgent
6
33%
518
cicoach
prompt_style
concise
4
50%
1501
cicoach
prompt_style
detailed
6
83%
1445
copilotagentanalysis
output_format
prose
4
75%
548
copilotagentanalysis
output_format
ste
2
100%
862
copilotagentanalysis
output_format
structured
4
100%
676
copilotprnlpanalysis
nlp_prompt_style
structured
1
100%
79
dailyagentrxtraceoptimizer
sub_agent_strategy
single_agent
8
88%
1158
dailyagentrxtraceoptimizer
sub_agent_strategy
sub_agents
2
100%
1126
dailyarchitecturediagram
detail_level
brief
4
100%
864
dailyarchitecturediagram
detail_level
comprehensive
6
83%
491
dailyastrostylelitemarkdownspellcheck
prompt_style
concise
4
75%
508
dailyastrostylelitemarkdownspellcheck
prompt_style
detailed
6
100%
490
dailycachestrategyanalyzer
model_size
gpt-5.3-codex
6
17%
1854
dailycachestrategyanalyzer
model_size
gpt-5.3-codex-spark
4
0%
1973
dailycavemanoptimizer
model_size
claude-haiku-4.5
5
100%
573
dailycavemanoptimizer
model_size
claude-sonnet-5
5
80%
490
dailycodemetrics
output_format
executive_summary
5
80%
734
dailycodemetrics
output_format
full_detail
4
100%
830
dailycodemetrics
output_format
ste
1
100%
920
dailycommunityattribution
prompt_style
concise
5
60%
443
dailycommunityattribution
prompt_style
verbose
5
20%
558
dailycompilerquality
output_format
concise
2
100%
696
dailycompilerquality
output_format
detailed
2
50%
1977
dailycompilerquality
output_format
ste
6
50%
695
dailydochealer
model_size
claude-haiku-4.5
4
50%
373
dailydochealer
model_size
claude-sonnet-5
6
100%
480
dailydocupdater
model_size
claude-haiku-4.5
5
20%
860
dailydocupdater
model_size
claude-sonnet-5
5
40%
900
dailyfact
reasoning_depth
multi_candidate
6
100%
504
dailyfact
reasoning_depth
single_pass
4
75%
452
dailyissuesreport
output_format
collapsible
5
100%
631
dailyissuesreport
output_format
inline
2
50%
768
dailyissuesreport
output_format
ste
3
33%
1600
dailynews
prompt_style
concise
5
100%
1063
dailynews
prompt_style
detailed
5
100%
988
dailyrenderingscriptsverifier
remove_redundant_context_v1
candidate
1
100%
491
dailysafeoutputoptimizer
log_fetch_strategy
eager
1
100%
1053
dailysecurityredteam
reasoning_depth
iterative
1
0%
783
dailysemgrepscan
semgrep_output_format
bullet_list
3
100%
538
dailysemgrepscan
semgrep_output_format
prose
3
100%
527
dailysemgrepscan
semgrep_output_format
structured_sections
4
100%
440
dailystorify
deterministic_precondition_v1
control
1
100%
903
dataflowprdiscussiondataset
caveman_mode
no
6
83%
498
dataflowprdiscussiondataset
caveman_mode
yes
4
75%
531
deepreport
output_format
annotated_brief
2
100%
1284
deepreport
output_format
executive_brief
3
33%
1675
deepreport
output_format
full_briefing
4
75%
1578
deepreport
output_format
ste
1
0%
2622
dependabotgochecker
prompt_style
concise
3
0%
685
dependabotgochecker
prompt_style
detailed
4
50%
570
dependabotgochecker
prompt_style
step_by_step
3
67%
538
gofan
prompt_tightening_v1
candidate
1
0%
578
gpclean
tool_verbosity
full_bash
4
100%
444
gpclean
tool_verbosity
minimal_toolset
6
0%
481
issuearborist
prompt_style
concise
4
50%
401
issuearborist
prompt_style
detailed
6
100%
466
plan
reasoning_depth
baseline
4
100%
448
plan
reasoning_depth
deep
2
100%
545
plan
reasoning_depth
shallow
1
100%
470
smokecopilot
caveman
no
5
20%
695
smokecopilot
caveman
yes
5
0%
645
smokecopilot
subagent_model
large
4
0%
643
smokecopilot
subagent_model
small
6
17%
687
smokecopilotaoaiapikey
caveman
no
4
0%
945
smokecopilotaoaiapikey
caveman
yes
6
33%
876
smokecopilotaoaiapikey
subagent_model
large
6
17%
900
smokecopilotaoaiapikey
subagent_model
small
4
25%
910
smokecopilotaoaientra
caveman
no
5
20%
942
smokecopilotaoaientra
caveman
yes
5
40%
927
smokecopilotaoaientra
subagent_model
large
5
20%
1001
smokecopilotaoaientra
subagent_model
small
5
40%
868
smokecopilotsubagents
sub_agent_strategy
delegated_sequential
5
100%
421
smokecopilotsubagents
sub_agent_strategy
inline_strict
5
80%
388
smokegemini
sub_agent_strategy
single_agent
1
0%
404
smokegemini
sub_agent_strategy
sub_agents
3
0%
1791
smokeproject
prompt_style_test
step_by_step
1
0%
416
smoketemporaryid
sub_agent_strategy
single_agent
1
100%
399
typist
tone_style
conversational
3
33%
1161
typist
tone_style
formal
7
29%
785
weeklyblogpostwriter
prefetch_strategy
eager
5
80%
663
weeklyblogpostwriter
prefetch_strategy
lazy
5
100%
617
Variants with fewer than 2 joined runs have no confidence interval. Variants with 0 joined runs are omitted.
Zero usable samples: copilotprnlpanalysis/nlp_prompt_style:concise, copilotprnlpanalysis/nlp_prompt_style:structured, dailyrenderingscriptsverifier/remove_redundant_context_v1:candidate, dailyrenderingscriptsverifier/remove_redundant_context_v1:control, gofan/prompt_tightening_v1:candidate, gofan/prompt_tightening_v1:control.
Analyses with no workflow source file in the repo: dailysubagentoptimizer, dependabotcampaign, smokeantigravity.
Analyses whose source has no experiments: frontmatter (likely imported or stale branch state): architectureguardian, dailyfunctionnamer, dailystorify, prsouschef, smokepi, testqualitysentinel.
dailycodemetrics has issue: 1, which points to a merged PR, not a tracking issue. Excluded from comments; needs a frontmatter fix.
Stale labels (flagged, not removed): #34032 typist and #31673 daily-security-red-team carry experiment:concluded, but their core decision is EXTEND.
Self-Tuning Continuation Plan
Implement native empty_output_rate and run_success_rate metrics so guardrails stop reporting unsupported. This is the largest source of EXTEND.
Add a primary metric: to the workflows listed above, and fix issue: 1 in dailycodemetrics.
Keep EXTEND for the smokecopilot factorial workflows. Cells are 2-4 runs against min_samples 20, so do not promote an interaction effect.
Reconcile the 9 unmatched analyses: restore missing source workflows or remove stale experiment branches.
Add artifact-based assignment inference to reach the full 30-run window in a later report.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Overview
Canonical output of
gh aw experiments analyzefor 51 experiment workflows (54 analyses).Key Metrics
Success rates are descriptive context only. The core decision is the canonical conclusion.
Canonical decisions (all 54 analyses)
#33280#43177#36105#32603#42467#32335#63156#31926#31324#31190#38094#31673#37102#30015#47551#37302#34032#38590#1#32390#30573#32795#42941Descriptive per-variant stats (recent joined runs)
Variants with fewer than 2 joined runs have no confidence interval. Variants with 0 joined runs are omitted.
Factorial interaction cells (two-experiment workflows)
smokecopilot (10 joined runs, min_samples 20 means every cell is under-powered:
SPARSE_CELL_RISK)smokecopilotaoaiapikey (10 joined runs, min_samples 20 means every cell is under-powered:
SPARSE_CELL_RISK)smokecopilotaoaientra (10 joined runs, min_samples 20 means every cell is under-powered:
SPARSE_CELL_RISK)Data Quality and Pipeline Gaps
guardrail_unsupported.metric: None): architectureguardian, dailycommunityattribution, dailyfunctionnamer, dailystorify, dailysubagentoptimizer, dependabotcampaign, prsouschef, smokeantigravity, smokecopilot, smokecopilotaoaiapikey, smokecopilotaoaientra, smokepi, testqualitysentinel.experiments:frontmatter (likely imported or stale branch state): architectureguardian, dailyfunctionnamer, dailystorify, prsouschef, smokepi, testqualitysentinel.issue: 1, which points to a merged PR, not a tracking issue. Excluded from comments; needs a frontmatter fix.#34032typist and#31673daily-security-red-team carryexperiment:concluded, but their core decision is EXTEND.Self-Tuning Continuation Plan
empty_output_rateandrun_success_ratemetrics so guardrails stop reporting unsupported. This is the largest source of EXTEND.metric:to the workflows listed above, and fixissue: 1in dailycodemetrics.References: §44737623756, §44525978833, §44306076058
All reactions