[experiments] Daily Experiment Report — 2026-06-07

[github-actions[bot]]

🧪 Daily Experiment Report — 2026-06-07

33 experiments across 31 workflows. 4 ready · 1 significant · ✅ 0 · 🟡 27 · ❌ 6

⚡ Quick Stats

Metric	Value
Active experiments	33
Ready (min_samples)	4
Significant (p<0.05)	1
PROMOTE / EXTEND / ABANDON	0 / 27 / 6

---

📊 Featured Experiments

caveman — smoke-copilot 🟢 READY
no: 60 runs, 36% succ · yes: 61 runs, 38% succ
p = 0.919 · ❌ ABANDON — No significant effect detected (p=0.9193) with sufficient samples.

📈 Charts

subagent_model — smoke-copilot 🟢 READY
large: 50 runs, 53% succ · small: 51 runs, 20% succ
p = 0.058 · ❌ ABANDON — No significant effect detected (p=0.0582) with sufficient samples.

📈 Charts

reasoning_depth — daily-security-red-team 🟡 COLLECTING
iterative: 15 runs, 100% succ · single_pass: 11 runs, 73% succ
p = 0.032* · ❌ ABANDON — Guardrail metric violation detected.

📈 Charts

output_format — daily-issues-report 🟡 COLLECTING
collapsible: 14 runs, 0% succ · inline: 17 runs, 0% succ
p = N/A · 🟡 EXTEND — Still collecting data — min_samples (30) not yet reached for all variants.

📈 Charts

output_format — deep-report 🟡 COLLECTING
executive_brief: 9 runs, 100% succ · full_briefing: 9 runs, 100% succ · annotated_brief: 6 runs, 83% succ
p = 0.205 · 🟡 EXTEND — Still collecting data — min_samples (15) not yet reached for all variants.

📈 Charts

---

📋 All Experiments

View All 33 Experiments

Experiment	Workflow	Variants (n)	p-value	Recommendation
caveman	smoke-copilot	no(60), yes(61)	0.919	❌ ABANDON
subagent_model	smoke-copilot	large(50), small(51)	0.058	❌ ABANDON
sub_agent_strategy	smoke-gemini	sub_agents(54), single_agent(32)	N/A	🟡 EXTEND
sub_agent_decomposition	smoke-pi	single_agent(34), parallel_sub_agents(42)	N/A	🟡 EXTEND
sub_agent_strategy	smoke-antigravity	single_agent(27), sub_agents(30)	N/A	🟡 EXTEND
prompt_style	daily-astrostylelite-markdown-spellcheck	detailed(20), concise(15)	N/A	🟡 EXTEND
prompt_style	daily-community-attribution	concise(16), verbose(17)	N/A	🟡 EXTEND
output_format	daily-issues-report	collapsible(14), inline(17)	N/A	🟡 EXTEND
prompt_style	daily-news	detailed(17), concise(10)	N/A	🟡 EXTEND
tone_variant	aw-failure-investigator	clinical(10), narrative(10), assertive(6)	N/A	🟡 EXTEND
reasoning_depth	daily-security-red-team	iterative(15), single_pass(11)	0.032*	❌ ABANDON
output_format	deep-report	executive_brief(9), full_briefing(9), annotated_brief(6)	0.205	🟡 EXTEND
output_format	daily-compiler-quality	detailed(9), concise(13)	0.421	❌ ABANDON
output_format	daily-code-metrics	executive_summary(11), full_detail(10)	0.476	🟡 EXTEND
semgrep_output_format	daily-semgrep-scan	structured_sections(9), prose(6), bullet_list(6)	0.205	❌ ABANDON
prompt_compression	agent-performance-analyzer	caveman(11), verbose(7)	0.070	❌ ABANDON
sub_agent_strategy	agent-persona-explorer	batch(13), per_scenario(5)	N/A	🟡 EXTEND
reasoning_depth	daily-fact	single_pass(11), multi_candidate(6)	N/A	🟡 EXTEND
prompt_style	ci-coach	concise(6), detailed(10)	N/A	🟡 EXTEND
tool_verbosity	gpclean	full_bash(5), minimal_toolset(9)	N/A	🟡 EXTEND
tone_style	typist	conversational(5), formal(4)	N/A	🟡 EXTEND
sub_agent_strategy	daily-agentrx-trace-optimizer	sub_agents(1), single_agent(3)	N/A	🟡 EXTEND
timeout_setting	None	default(2), relaxed(1), tight(1)	N/A	🟡 EXTEND
prompt_style	dependabot-go-checker	step_by_step(1), concise(2), detailed(1)	N/A	🟡 EXTEND
prompt_style	issue-arborist	concise(2), detailed(2)	N/A	🟡 EXTEND
model_size	daily-doc-healer	small-agent(1), agent(2)	N/A	🟡 EXTEND
caveman	smoke-copilot-aoai-apikey	yes(2), no(1)	N/A	🟡 EXTEND
subagent_model	smoke-copilot-aoai-apikey	small(2), large(1)	N/A	🟡 EXTEND
model_size	daily-cache-strategy-analyzer	small-agent(1), agent(1)	N/A	🟡 EXTEND
model_size	daily-caveman-optimizer	small-agent(2)	N/A	🟡 EXTEND
model_size	daily-doc-updater	small-agent(2)	N/A	🟡 EXTEND
model_size	daily-function-namer	agent(2)	N/A	🟡 EXTEND
detail_level	daily-architecture-diagram	brief(1)	N/A	🟡 EXTEND

Run: 27087826542

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

• proxy.golang.org
• releaseassets.githubusercontent.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "proxy.golang.org"
    - "releaseassets.githubusercontent.com"

See Network Configuration for more information.

Generated by 🧪 daily-experiment-report · ⌖ 31.4 AIC · ⊞ 24.6K · ◷

• expires on Jun 10, 2026, 9:13 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by daily-experiment-report.

A newer discussion is available at Discussion #37786.