[experiments] Daily Experiment Report — 2026-06-09

[github-actions[bot]]

🧪 Daily Experiment Report — 2026-06-09

15 active experiments across 14 workflows — all in sample collection (🟡 EXTEND). No experiments have reached min_samples for significance testing.

⚡ Quick Stats

Metric	Value
Active experiments	15
Ready for analysis	0
Statistically significant (p<0.05)	0
Recommendations	🟡 EXTEND: 15

---

sub_agent_strategy · smokeantigravity

🟡 COLLECTING · 61 total runs · min_samples=30

📈 Statistics & Charts

Variant	n	Progress	Succ%	Dur(s)
single_agent	28	███████░ 28/30	100%	276
sub_agents	33	████████ 33/30	90%	217

Success rate p-value (single_agent vs sub_agents): 0.4237 (below min_samples — low power)

Recommendation: EXTEND — Waiting for all variants to reach min_samples (30)

---

reasoning_depth · dailysecurityredteam

🟡 COLLECTING · 28 total runs · min_samples=30

📈 Statistics & Charts

Variant	n	Progress	Succ%	Dur(s)
iterative	16	████░░░░ 16/30	100%	410
single_pass	12	███░░░░░ 12/30	75%	341

Success rate p-value (iterative vs single_pass): 0.0343* (below min_samples — low power)

Recommendation: EXTEND — Waiting for all variants to reach min_samples (30)

---

prompt_compression · agentperformanceanalyzer

🟡 COLLECTING · 20 total runs · min_samples=14

📈 Statistics & Charts

Variant	n	Progress	Succ%	Dur(s)
caveman	13	███████░ 13/14	69%	672
verbose	7	████░░░░ 7/14	100%	657

Success rate p-value (caveman vs verbose): 0.1008 (below min_samples — low power)

Recommendation: EXTEND — Waiting for all variants to reach min_samples (14)

---

📊 All Experiments

View all 15 experiments

Experiment	Workflow	Runs	min_samples	Lowest variant	Rec
sub_agent_strategy	smokeantigravity	61	30	28/30 (93%)	🟡
prompt_style	dailyastrostylelitemarkdownspellcheck	37	20	17/20 (85%)	🟡
reasoning_depth	dailysecurityredteam	28	30	12/30 (40%)	🟡 #31673
semgrep_output_format	dailysemgrepscan	23	30	6/30 (20%)	🟡 #32795
output_format	dailycodemetrics	23	20	11/20 (55%)	🟡
prompt_compression	agentperformanceanalyzer	20	14	7/14 (50%)	🟡
prompt_style	cicoach	17	20	6/20 (30%)	🟡 #32335
tool_verbosity	gpclean	16	30	7/30 (23%)	🟡
caveman	smokecopilotaoaiapikey	9	20	4/20 (20%)	🟡
subagent_model	smokecopilotaoaiapikey	9	20	4/20 (20%)	🟡
model_size	dailydochealer	5	20	1/20 (5%)	🟡
prompt_style	issuearborist	4	30	2/30 (6%)	🟡 #30015
model_size	dailyfunctionnamer	4	20	1/20 (5%)	🟡
detail_level	dailyarchitecturediagram	2	10	2/10 (20%)	🟡 #31926
summary_detail	dependabotcampaign	2	30	2/30 (6%)	🟡 #37533

Window: last 30 completed runs · p-value threshold: 0.05 (two-tailed)
Run: §27194994815

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

• proxy.golang.org
• releaseassets.githubusercontent.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "proxy.golang.org"
    - "releaseassets.githubusercontent.com"

See Network Configuration for more information.

Generated by 🧪 daily-experiment-report · ⌖ 21.2 AIC · ⊞ 23.1K · ◷

• expires on Jun 12, 2026, 1:15 AM UTC-08:00