[experiments] Daily Experiment Report — 2026-06-28

[github-actions[bot]]

🧪 Daily Experiment Report — 2026-06-28

29 experiments in github/gh-aw: 5 reached READY_FOR_ANALYSIS (all variants ≥ min_samples); 24 still collecting. Outcome metrics require per-run artifact data — this report covers assignment balance and sample progress.

⚡ Quick Stats

Metric	Value
Active experiments	29
🟢 Ready for analysis	5
🟡 Collecting	24

---

🟢 Ready for Analysis

sub_agent_strategy · smoke-gemini

🟢 READY · 176 runs · balance p=0.010 ⚠️IMBALANCED · min_samples=30
H0: no change. H1: sub_agents reduces tokens by >=20%

Variant	n	Progress
single_agent(ctrl)	71	████████ 71/30 (100%)
sub_agents	105	████████ 105/30 (100%)

Recommendation: READY_FOR_ANALYSIS — All variants reached min_samples=30; outcome data needed for final recommendation.

---

sub_agent_strategy · smoke-antigravity

🟢 READY · 147 runs · balance p=0.680 OK · min_samples=30
H0: no change. H1: sub_agents reduces tokens by >=20%

Variant	n	Progress
single_agent(ctrl)	71	████████ 71/30 (100%)
sub_agents	76	████████ 76/30 (100%)

Recommendation: READY_FOR_ANALYSIS — All variants reached min_samples=30; outcome data needed for final recommendation.

---

output_format · daily-compiler-quality · Issue #32390

🟢 READY · 43 runs · balance p=0.647 OK · min_samples=20
H0: no change. H1: concise achieves equal engagement with ≥30% fewer tokens

Variant	n	Progress
detailed(ctrl)	20	████████ 20/20 (100%)
concise	23	████████ 23/20 (100%)

Recommendation: READY_FOR_ANALYSIS — All variants reached min_samples=20; outcome data needed for final recommendation.

---

prompt_compression · agent-performance-analyzer · Issue #33280

🟢 READY · 38 runs · balance p=0.105 OK · min_samples=14
H0: no change in effective_tokens. H1: caveman reduces tokens by ≥20% while maintaining quality ≥90%

Variant	n	Progress
verbose(ctrl)	14	████████ 14/14 (100%)
caveman	24	████████ 24/14 (100%)

Recommendation: READY_FOR_ANALYSIS — All variants reached min_samples=14; outcome data needed for final recommendation.

---

tone_style · typist · Issue #34032

🟢 READY · 25 runs · balance p=0.841 OK · min_samples=10
H0: no change. H1: conversational tone increases engagement by 20%+

Variant	n	Progress
formal(ctrl)	13	████████ 13/10 (100%)
conversational	12	████████ 12/10 (100%)

Recommendation: READY_FOR_ANALYSIS — All variants reached min_samples=10; outcome data needed for final recommendation.

---

🟡 Collecting Data (24 experiments)

View collecting experiments table

Experiment	Workflow	Total	min_s	Worst progress	Issue
output_format	daily-code-metrics	42	20	████████ 19/20 (95%)	#1
sub_agent_strategy	agent-persona-explorer	38	14	███████░ 13/14 (92%)	—
output_format	daily-issues-report	53	30	███████░ 26/30 (86%)	#30573
prompt_style	ci-coach	31	20	██████░░ 15/20 (75%)	#32335
prompt_style	daily-news	47	30	██████░░ 22/30 (73%)	#31190
output_format	deep-report	39	15	██████░░ 11/15 (73%)	—
reasoning_depth	daily-security-red-team	47	30	█████░░░ 20/30 (66%)	#31673
tone_variant	aw-failure-investigator	109	50	█████░░░ 32/50 (64%)	#36105
tool_verbosity	gpclean	35	30	████░░░░ 16/30 (53%)	—
model_size	daily-doc-updater	21	20	████░░░░ 10/20 (50%)	—
model_size	daily-doc-healer	21	20	████░░░░ 9/20 (45%)	—
reasoning_depth	daily-fact	32	30	███░░░░░ 12/30 (40%)	#31324
semgrep_output_format	daily-semgrep-scan	42	30	███░░░░░ 12/30 (40%)	#32795
sub_agent_strategy	daily-agentrx-trace-optimizer	19	20	███░░░░░ 7/20 (35%)	—
model_size	daily-caveman-optimizer	21	20	███░░░░░ 7/20 (35%)	—
output_format	copilot-agent-analysis	20	30	██░░░░░░ 9/30 (30%)	—
log_fetch_strategy	daily-safe-output-optimizer	13	20	██░░░░░░ 6/20 (30%)	—
model_size	daily-cache-strategy-analyzer	11	20	█░░░░░░░ 3/20 (15%)	—
sub_agent_strategy	architecture-guardian	8	30	█░░░░░░░ 3/30 (10%)	#39062
caveman_mode	dataflow-pr-discussion-dataset	3	10	█░░░░░░░ 1/10 (10%)	#37102
prompt_style	dependabot-go-checker	13	30	█░░░░░░░ 3/30 (10%)	—
prompt_style	issue-arborist	4	30	█░░░░░░░ 2/30 (6%)	#30015
detail_level	daily-architecture-diagram	4	10	░░░░░░░░ 0/10 (0%)	#31926
prefetch_strategy	weekly-blog-post-writer	2	10	░░░░░░░░ 0/10 (0%)	#38590

Run: 28316981917

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

• proxy.golang.org
• releaseassets.githubusercontent.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "proxy.golang.org"
    - "releaseassets.githubusercontent.com"

See Network Configuration for more information.

Generated by 🧪 daily-experiment-report · 202.6 AIC · ⌖ 30.9 AIC · ⊞ 11.3K · ◷

• expires on Jul 1, 2026, 1:13 AM UTC-08:00