[experiments] Daily Experiment Report — 2026-06-08

[github-actions[bot]]

🧪 Daily Experiment Report — 2026-06-08

Analysed 32 experiments across 30 workflows. No experiments cleared all guardrails with p < 0.05. 15 ABANDON (guardrail/no-effect), 17 EXTEND (collecting data).

⚡ Quick Stats

Metric	Value
Active experiments	32
All variants ≥ min_samples	3
p < 0.05 (any variant)	1
Recommendations	✅ PROMOTE: 0 · 🟡 EXTEND: 17 · ❌ ABANDON: 15

---

smoke-copilot / subagent_model ❌ ABANDON

Variant	n (state)	n (obs)	Success Rate	Mean Dur	p-value	Progress
small	51	15	20.0%	16m	(ref)	████████ 51/20
large	51	15	53.3%	16m	0.058	████████ 51/20

Rationale: No significant effect detected (p = 0.0582, all variants reached min_samples).

---

smoke-gemini / sub_agent_strategy ❌ ABANDON

Variant	n (state)	n (obs)	Success Rate	Mean Dur	p-value	Progress
single_agent	33	6	83.3%	6m	(ref)	████████ 33/30
sub_agents	54	7	100.0%	6m	0.261	████████ 54/30

⚠️ Guardrail failures: single_agent.success_rate = 0.83 (need >=0.95)

Rationale: Guardrail failure: success_rate >=0.95 (got 0.83 for single_agent)

---

smoke-antigravity / sub_agent_strategy ❌ ABANDON

Variant	n (state)	n (obs)	Success Rate	Mean Dur	p-value	Progress
single_agent	27	5	100.0%	5m	(ref)	███████░ 27/30
sub_agents	31	8	87.5%	3m	0.411	████████ 31/30

⚠️ Guardrail failures: sub_agents.success_rate = 0.88 (need >=0.95)

Rationale: Guardrail failure: success_rate >=0.95 (got 0.88 for sub_agents)

---

daily-astrostylelite-markdown-spellcheck / prompt_style 🟡 EXTEND

Variant	n (state)	n (obs)	Success Rate	Mean Dur	p-value	Progress
concise	16	15	93.3%	8m	(ref)	████░░░░ 16/30
detailed	20	15	100.0%	7m	0.309	█████░░░ 20/30

Rationale: Still collecting data: concise=16/30, detailed=20/30

---

agent-performance-analyzer / prompt_compression ❌ ABANDON

Variant	n (state)	n (obs)	Success Rate	Mean Dur	p-value	Progress
verbose	7	7	100.0%	11m	(ref)	████░░░░ 7/14
caveman	12	12	66.7%	11m	0.086	███████░ 12/14

⚠️ Guardrail failures: caveman.run_success_rate = 0.67 (need >=0.90)

Rationale: Guardrail failure: run_success_rate >=0.90 (got 0.67 for caveman)

---

📋 All 32 Experiments Summary

Workflow	Experiment	Runs	Rec
agent-performance-analyzer	prompt_compression	19	❌ ABANDON
ci-coach	prompt_style	16	❌ ABANDON
daily-architecture-diagram	detail_level	1	❌ ABANDON
daily-cache-strategy-analyzer	model_size	0	❌ ABANDON
daily-caveman-optimizer	model_size	0	❌ ABANDON
daily-compiler-quality	output_format	23	❌ ABANDON
daily-doc-healer	model_size	1	❌ ABANDON
daily-doc-updater	model_size	0	❌ ABANDON
daily-fact	reasoning_depth	17	❌ ABANDON
daily-function-namer	model_size	1	❌ ABANDON
daily-security-red-team	reasoning_depth	27	❌ ABANDON
smoke-antigravity	sub_agent_strategy	58	❌ ABANDON
smoke-copilot	caveman	122	❌ ABANDON
smoke-copilot	subagent_model	102	❌ ABANDON
smoke-gemini	sub_agent_strategy	87	❌ ABANDON
agent-persona-explorer	sub_agent_strategy	19	🟡 EXTEND
aw-failure-investigator	tone_variant	30	🟡 EXTEND
daily-agentrx-trace-optimizer	sub_agent_strategy	5	🟡 EXTEND
daily-astrostylelite-markdown-spellcheck	prompt_style	36	🟡 EXTEND
daily-code-metrics	output_format	22	🟡 EXTEND
daily-community-attribution	prompt_style	34	🟡 EXTEND
daily-issues-report	output_format	32	🟡 EXTEND
daily-news	prompt_style	27	🟡 EXTEND
daily-semgrep-scan	semgrep_output_format	22	🟡 EXTEND
deep-report	output_format	24	🟡 EXTEND
dependabot-campaign	summary_detail	1	🟡 EXTEND
dependabot-go-checker	prompt_style	4	🟡 EXTEND
gpclean	tool_verbosity	15	🟡 EXTEND
issue-arborist	prompt_style	4	🟡 EXTEND
smoke-copilot-aoai-apikey	caveman	4	🟡 EXTEND
smoke-copilot-aoai-apikey	subagent_model	4	🟡 EXTEND
typist	tone_style	9	🟡 EXTEND

References: §27127971961

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• proxy.golang.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "proxy.golang.org"

See Network Configuration for more information.

Generated by 🧪 daily-experiment-report · ⌖ 31.3 AIC · ⊞ 23.1K · ◷

• expires on Jun 11, 2026, 1:44 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by daily-experiment-report.

A newer discussion is available at Discussion #38076.