Skip to content

Architecture ablation

Mike edited this page Jun 21, 2026 · 2 revisions
========================================================================
ALL EXPERIMENTS COMPLETE
Results in: runs/ablation_csl_tinyvit_23m_arch_fixes_dp02/
========================================================================
Compare with:
  uv run python tools/ablation_summary.py runs/ablation_csl_tinyvit_23m_arch_fixes_dp02


═════════════════════════════════════════════════════════════════════════════════════
  ABLATION SUMMARY: runs/ablation_csl_tinyvit_23m_arch_fixes_dp02  (8 experiments)
═════════════════════════════════════════════════════════════════════════════════════

Experiment                             mAP     R1   Δ mAP Best@   Loss   Trip
─────────────────────────────────────────────────────────────────────────────────────
3_metric/a_raw_concat_norm_infer    89.59% 95.37%   +0.00   200  1.309 0.6471 ★
0_control/a_recipe_legacy           89.42% 95.61%   -0.17   200  1.302 0.6282
1_attention/a_rect_mask_signed_shift 89.27% 95.25%   -0.32   200  1.319 0.6428
3_metric/b_raw_concat_global_parts  89.25% 95.07%   -0.34   200  1.322 0.6508
2_pool/b_rect_gelu_gem              89.22% 94.95%   -0.37   190  1.301 0.6373
4_stage3/a_rect_gelu_rawconcat_globalblock 89.14% 94.92%   -0.45   190  1.328 0.6625
2_pool/c_rect_softplus_gem          89.00% 95.19%   -0.59   190  1.311 0.6377
2_pool/a_rect_avg                   87.59% 94.74%   -2.00   180  1.362 0.6488
─────────────────────────────────────────────────────────────────────────────────────
  Baseline: 89.59% mAP  |  Δ mAP = experiment − baseline

⚠  Possibly undertrained (best epoch near end):
   0_control/a_recipe_legacy: best@200/200
   1_attention/a_rect_mask_signed_shift: best@200/200
   3_metric/a_raw_concat_norm_infer: best@200/200
   3_metric/b_raw_concat_global_parts: best@200/200

┌─ 0_control
│  a_recipe_legacy                mAP=89.42%  R1=95.61%
└─ Spread: 0.00% → INCONCLUSIVE (<0.5%)

┌─ 1_attention
│  a_rect_mask_signed_shift       mAP=89.27%  R1=95.25%
└─ Spread: 0.00% → INCONCLUSIVE (<0.5%)

┌─ 2_pool
│  b_rect_gelu_gem                mAP=89.22%  R1=94.95%
│  c_rect_softplus_gem            mAP=89.00%  R1=95.19%
│  a_rect_avg                     mAP=87.59%  R1=94.74%
└─ Spread: 1.63% → SIGNIFICANT (Δ=1.63%)

┌─ 3_metric
│  a_raw_concat_norm_infer        mAP=89.59%  R1=95.37%
│  b_raw_concat_global_parts      mAP=89.25%  R1=95.07%
└─ Spread: 0.34% → INCONCLUSIVE (<0.5%)

┌─ 4_stage3
│  a_rect_gelu_rawconcat_globalblock mAP=89.14%  R1=94.92%
└─ Spread: 0.00% → INCONCLUSIVE (<0.5%)

Clone this wiki locally