fix(prune): separate file and context savings

.codearbiter/open-tasks.md

@@ -41,7 +41,7 @@ per task. Schema and the count rule: see `plugins/ca/hooks/init-codearbiter.py`
 - [x] v2.release.0005 - Assert the notes-file `## vX.Y.Z` heading matches the tag before `gh release create` (LOW 11), so a stale notes-file can't publish the wrong section under the right tag.  (from release-skill red-team 2026-06-23)  (done 2026-06-26)
 - [x] v2.release.0006 - Confirm pre-release/beta tag exclusion in baseline resolution (LOW 12) is fully covered by the new `grep -Ev '-(beta|rc|alpha)'` filter; close if so, else tighten.  (from release-skill red-team 2026-06-23)  (done 2026-06-26)
 - [ ] v2.docs.0003 - Decision-log reconciliation form of the forge-allowlist drift check (CONFIRM-05-dependent). The shipped site check is self-consistency only (slug maps to a real command file); the stronger form reconciles PREVIEW_COMMANDS against a recorded promotion in the decision log and belongs in /ca:doctor (reaches outside site/). Blocked on CONFIRM-05 (the --farm promotion bar).  (from sprint release-hardening-debt-paydown)
-- [ ] v2.prune.0001 - [MED] The prune metric conflates file-bytes-freed with context-tokens-freed, overstating the model-context benefit that feeds the dry->on go/no-go.
+- [x] v2.prune.0001 - [MED] The prune metric conflates file-bytes-freed with context-tokens-freed, overstating the model-context benefit that feeds the dry->on go/no-go.  (done 2026-07-20)
   - Desc: hook_run / append_dry_metrics / append_audit_log report bytes_before/after, freed_bytes, est_tokens (= bytes//4 over the WHOLE file) and per-strategy byte deltas. But sidecar-collapse (the single biggest contributor — 522KB of 890KB in session ff1c472d's PreCompact run) trims the top-level `toolUseResult` sidecar, which is Claude Code bookkeeping NOT sent to the model. Verified against the pre-prune backup: for large Reads the sidecar duplicates the model-visible message.content tool_result (line 214: sidecar 59.8KB ~= message 67KB); for others it holds bulk the model never received (line 475: sidecar 13.8KB vs message 0.26KB). So est_tokens overstates the compaction-relevant / cold-re-cache-cost benefit. That run's headline was -223k est-tokens (40.3%) but the actual model-context slice was ~87k (aged-result-condense only); the ~131k sidecar slice is disk/IO/resume-parse hygiene, not context. prune-dry.jsonl — the evidence base for flipping prune to `on` — accrues this conflated number.
   - Done when: prune metrics separate file_bytes_freed from context_bytes_freed (strategies tagged by whether they edit message.content vs sidecar/bookkeeping), the dry-run records + statusline surface the context-relevant figure distinctly labeled, and the dry->on go/no-go is reassessed on the corrected number.
   - Boundaries: none (observability/metrics; no enforcement surface)

.codearbiter/reports/2026-07-20-prune-metric-reassessment.md

@@ -0,0 +1,28 @@
+# Prune metric reassessment — 2026-07-20
+
+Scope: aggregate-only recalculation of the local
+`~/.codearbiter/metrics/prune-dry.jsonl` evidence. No transcript content,
+repository identity, paths, or session identifiers were read into this report.
+
+## Evidence
+
+- Rows: 9 total; 9 `dry-run` with zero validation errors.
+- Legacy whole-file estimate: 4,545,576 bytes, approximately 1,136,394 tokens.
+- File-only `sidecar-collapse`: 2,860,049 bytes.
+- Model-visible strategies: 1,611,362 bytes, approximately 402,840 tokens.
+- Individual rows clearing the default 80,000 context-token nudge floor: 0.
+
+Strategy deltas do not sum exactly to the serialized file delta because JSON
+container punctuation and reserialization overhead are outside the replaced
+payloads. The context estimate deliberately counts only strategy deltas whose
+targets live in model-visible message content.
+
+## Decision
+
+The safety evidence remains clean, but the earlier benefit headline was about
+64.5% too high because it treated file-only bookkeeping as model context. This
+sample does not justify arming the cold-cache nudge at its default threshold and
+does not support graduating pruning from preview on benefit evidence alone.
+Keep the feature off by default, collect new dry records with the corrected
+schema, and base any later `dry` to `on` decision on
+`context_est_tokens_freed`, not the legacy whole-file estimate.

.github/scripts/test_prune_policy_parity.py

@@ -15,6 +15,7 @@
 from _prunepolicy import (  # noqa: E402
     PrunePolicy,
     SemanticEntry,
+    STRATEGY_METRIC_SCOPES,
     audit_outcomes,
     has_marker,
     marker_for,
@@ -110,11 +111,25 @@ def test_markers_tiers_metrics_and_plans_are_idempotent(self):
         self.assertEqual(once.metrics["entries_before"], 5)
         self.assertEqual(once.metrics["candidate_entries"], 4)
         self.assertEqual(once.audit_codes, ("CA-PRUNE-PLAN",))
-        self.assertEqual(reduction_metrics(1000, 600), {
+        self.assertEqual(set(STRATEGY_METRIC_SCOPES), set(select_strategies("aggressive")))
+        self.assertEqual(STRATEGY_METRIC_SCOPES["sidecar-collapse"], "file-only")
+        self.assertTrue(all(
+            scope == "context"
+            for name, scope in STRATEGY_METRIC_SCOPES.items()
+            if name != "sidecar-collapse"
+        ))
+        self.assertEqual(reduction_metrics(1000, 600, {
+            "sidecar-collapse": 300,
+            "aged-result-condense": 100,
+        }), {
             "bytes_before": 1000, "bytes_after": 600, "freed_bytes": 400,
             "est_tokens_before": 250, "est_tokens_after": 150, "pct": 40.0,
+            "file_bytes_freed": 400, "file_est_tokens_freed": 100,
+            "file_est_tokens_before": 250, "file_est_tokens_after": 150,
+            "file_pct": 40.0, "context_bytes_freed": 100,
+            "context_est_tokens_freed": 25,
         })
-        self.assertEqual(reduction_metrics(100, 120)["freed_bytes"], -20)
+        self.assertEqual(reduction_metrics(100, 120, {})["freed_bytes"], -20)
         self.assertEqual(audit_outcomes(parse_errors=1, orphans=0, unpaired=2),
                          ("FAIL", "OK", "WARN", "OK"))
 

CHANGELOG.md

@@ -32,6 +32,12 @@ predate the plugin rewrite and are grouped by date.
   affirmative host project trust; unknown or foreign tool replacements fail
   closed.
 
+### Fixed
+
+- Prune dry-run records, audit logs, CLI, footer, and cold-cache metrics now separate
+  model-visible context savings from file-only sidecar cleanup; sidecar bytes
+  no longer inflate the context-benefit decision or arm the cold-cache nudge.
+
 ## [2.9.0] — 2026-07-12
 
 Statusline customization and correctness hardening across concurrent sessions.

README.md

@@ -422,7 +422,7 @@ Some features are built, tested, and shipping in the box, but not yet *blessed*.
 | Pluggable execution farm | <kbd>/ca:sprint --farm</kbd> | `preview` | run it on a real sprint, report results |
 | ca-sandbox (local Codespace) | install the `ca-sandbox` plugin | `preview` | explore real repos in it; run `--with-claude` and report |
 
-**Live transcript pruning.** Long sessions bloat the transcript until Claude Code compacts early and you lose working headroom; `CODEARBITER_PRUNE=dry` computes every prune it would make and logs the evidence without touching your transcript. It's preview because the `dry → on` go/no-go needs that real-session evidence first. Details and tuning knobs: [What's in the Forge](https://arbiterforge.github.io/codeArbiter/feature-forge/whats-in-the-forge/).
+**Live transcript pruning.** Long sessions bloat the transcript until Claude Code compacts early and you lose working headroom; `CODEARBITER_PRUNE=dry` computes every prune it would make and logs the evidence without touching your transcript. The evidence separates model-visible context savings from file-only sidecar cleanup, and only the context figure informs the `dry → on` benefit decision. It's preview because that go/no-go needs real-session evidence first. Details and tuning knobs: [What's in the Forge](https://arbiterforge.github.io/codeArbiter/feature-forge/whats-in-the-forge/).
 
 #### Pluggable execution farm
 

core/pysrc/_prunelib.py

@@ -202,7 +202,12 @@ def _is_small_scalar(v, limit=200):
 # --------------------------------------------------------------------------- #
 
 def _record(report, name, touched, before, after):
-    report[name] = {"lines": touched, "bytes_before": before, "bytes_after": after}
+    report[name] = {
+        "lines": touched,
+        "bytes_before": before,
+        "bytes_after": after,
+        "metric_scope": _policy.STRATEGY_METRIC_SCOPES[name],
+    }
 
 
 def s_sidecar_collapse(lines, index, cfg, report):
@@ -224,13 +229,15 @@ def s_sidecar_collapse(lines, index, cfg, report):
         new = dict(kept)
         new["_ca_condensed"] = _marker(orig)
         new_s = _dumps(new)
-        if len(new_s) >= len(orig):  # net-negative guard
+        if len(new_s) >= len(orig):  # preserve legacy transformation eligibility
             continue
+        orig_size = len(orig.encode("utf-8"))
+        new_size = len(new_s.encode("utf-8"))
         ln.obj["toolUseResult"] = new
         ln.dirty = True
         touched += 1
-        before += len(orig)
-        after += len(new_s)
+        before += orig_size
+        after += new_size
     _record(report, "sidecar-collapse", touched, before, after)
 
 
@@ -348,7 +355,7 @@ def s_reasoning_fold(lines, index, cfg, report):
         if not keep:  # never empty a message
             continue
         for b in thinking:
-            before += len(_dumps(b))
+            before += len(_dumps(b).encode("utf-8"))
         ln.obj["message"]["content"] = keep
         ln.dirty = True
         touched += 1
@@ -406,11 +413,14 @@ def s_mcp_payload_condense(lines, index, cfg, report):
                     if isinstance(inp, dict) else {})
             new = dict(kept)
             new["_ca_condensed"] = _marker(orig)
-            if len(_dumps(new)) >= len(orig):
+            new_s = _dumps(new)
+            if len(new_s) >= len(orig):  # preserve legacy transformation eligibility
                 continue
+            orig_size = len(orig.encode("utf-8"))
+            new_size = len(new_s.encode("utf-8"))
             blk["input"] = new
-            before += len(orig)
-            after += len(_dumps(new))
+            before += orig_size
+            after += new_size
             changed = True
         if changed:
             ln.dirty = True
@@ -1112,14 +1122,12 @@ def _idle_seconds(data, now):
 
 def _nudge_advisory(rec):
     """Build the advisory string from state-record numbers only.
-    Never includes transcript content — only sizes / est-tokens / percent."""
-    freed = int(rec.get("freed_bytes", 0) or 0)
-    pruned = int(rec.get("last_pruned_size", 0) or 0)
-    approx_tokens = est_tokens(pruned + freed)   # ~ in-memory (bloated) size
-    pct = rec.get("pct", 0)
-    return (f"Cold cache miss imminent on ~{approx_tokens // 1000}k tokens. "
-            f"/compact or exit + --resume lands that re-cache on pruned context "
-            f"(~{pct}% smaller). Submit again to proceed.")
+    Never includes transcript content — only context-byte/token aggregates."""
+    context_freed = int(rec.get("context_bytes_freed", 0) or 0)
+    context_tokens = est_tokens(context_freed)
+    return (f"Cold cache miss can re-cache ~{context_tokens // 1000}k avoidable "
+            f"context tokens. /compact or exit + --resume lands that re-cache "
+            f"on pruned context. Submit again to proceed.")
 
 
 def nudge_decision(rec, idle_secs, e):
@@ -1151,8 +1159,8 @@ def _num(key, default):
             return (False, "", nr)
         return (False, "", rec)
 
-    freed = rec.get("freed_bytes", 0)
-    if not isinstance(freed, (int, float)) or est_tokens(int(freed)) < min_tokens:
+    freed = rec.get("context_bytes_freed")
+    if type(freed) is not int or freed < 0 or est_tokens(freed) < min_tokens:
         return (False, "", rec)
 
     if rec.get("cold_nudged"):              # already fired this cold window
@@ -1252,7 +1260,11 @@ def hook_run(payload, env=None):
     except Exception:  # noqa: BLE001 — never let pruning break the turn
         return 0
     b0, b1 = res["bytes_before"], res["bytes_after"]
-    reduction = _policy.reduction_metrics(b0, b1)
+    strategy_savings = {
+        name: row["bytes_before"] - row["bytes_after"]
+        for name, row in res["strategies"].items()
+    }
+    reduction = _policy.reduction_metrics(b0, b1, strategy_savings)
     if not cfg.execute:
         # Dry mode: persist the would-be outcome to the shared data-collection
         # log so confidence accrues across sessions before flipping to on.
@@ -1266,6 +1278,8 @@ def hook_run(payload, env=None):
             "validation_errors": len(res["validation_errors"]),
             "strategies": {k: v["bytes_before"] - v["bytes_after"]
                            for k, v in res["strategies"].items()},
+            "strategy_scopes": {k: v["metric_scope"]
+                                for k, v in res["strategies"].items()},
         }, env=e)
     # Carry cold_nudged forward so the once-per-cold-window marker survives a
     # normal (non-armed) prune run. Re-read from state in case the nudge block
@@ -1278,6 +1292,11 @@ def hook_run(payload, env=None):
         "last_run_ts": int(time.time()),
         "pct": reduction["pct"],
         "freed_bytes": reduction["freed_bytes"],
+        "file_pct": reduction["file_pct"],
+        "file_bytes_freed": reduction["file_bytes_freed"],
+        "file_est_tokens_freed": reduction["file_est_tokens_freed"],
+        "context_bytes_freed": reduction["context_bytes_freed"],
+        "context_est_tokens_freed": reduction["context_est_tokens_freed"],
         "verdict": res["verdict"],
     }
     if _cold_nudged:
@@ -1335,13 +1354,15 @@ def run(path, cfg, session="session", data=None, lines=None, pre_stat=None):
     new_bytes = serialize(lines)
 
     errs = validate(orig_bytes, new_bytes, lines, cfg)
-    reduction = _policy.reduction_metrics(len(orig_bytes), len(new_bytes))
+    strategy_savings = {
+        name: row["bytes_before"] - row["bytes_after"]
+        for name, row in report.items()
+    }
+    reduction = _policy.reduction_metrics(
+        len(orig_bytes), len(new_bytes), strategy_savings)
     result = {
         "path": path,
-        "bytes_before": reduction["bytes_before"],
-        "bytes_after": reduction["bytes_after"],
-        "est_tokens_before": reduction["est_tokens_before"],
-        "est_tokens_after": reduction["est_tokens_after"],
+        **reduction,
         "strategies": report,
         "validation_errors": errs,
         "executed": False,
@@ -1357,9 +1378,10 @@ def run(path, cfg, session="session", data=None, lines=None, pre_stat=None):
         append_audit_log({
             "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
             "session": session, "path": path,
-            "bytes_before": len(orig_bytes), "bytes_after": len(new_bytes),
+            **reduction,
             "strategies": {k: v["bytes_before"] - v["bytes_after"]
                            for k, v in report.items()},
+            "strategy_scopes": {k: v["metric_scope"] for k, v in report.items()},
             "verdict": verdict,
         })
     return result

core/pysrc/_prunepolicy.py

@@ -7,8 +7,9 @@
 """
 #
 # Public API:
-#   reduction_metrics(bytes_before, bytes_after) -> dict  dry/run reduction metrics
-#                                            with the shared token estimate
+#   reduction_metrics(bytes_before, bytes_after, strategy_savings) -> dict
+#                                            file and model-context reductions
+#                                            with shared token estimates
 #   audit_outcomes(parse_errors, orphans, unpaired) -> tuple  host-neutral integrity
 #                                            levels; codecs retain native messages
 #   marker_for(original_text) -> str        deterministic condensed-content marker;
@@ -50,6 +51,17 @@
     "aged-result-condense",
     "oversize-result-clamp",
 )
+STRATEGY_METRIC_SCOPES = {
+    "sidecar-collapse": "file-only",
+    "reasoning-fold": "context",
+    "mcp-payload-condense": "context",
+    "shell-tail-keep": "context",
+    "superseded-read-condense": "context",
+    "repeat-reminder-fold": "context",
+    "inline-image-evict": "context",
+    "aged-result-condense": "context",
+    "oversize-result-clamp": "context",
+}
 
 
 @dataclass(frozen=True)
@@ -85,19 +97,36 @@ class PrunePlan:
 STANDARD_POLICY = PrunePolicy(tier="standard")
 
 
-def reduction_metrics(bytes_before, bytes_after):
-    """Host-neutral dry/run reduction metrics with the shared token estimate."""
+def reduction_metrics(bytes_before, bytes_after, strategy_savings):
+    """Host-neutral file and model-context reduction metrics."""
     if type(bytes_before) is not int or type(bytes_after) is not int \
             or bytes_before < 0 or bytes_after < 0:
         raise ValueError("prune byte metrics are invalid")
+    if not isinstance(strategy_savings, dict):
+        raise ValueError("prune strategy savings are invalid")
+    context_bytes_freed = 0
+    for name, value in strategy_savings.items():
+        if name not in STRATEGY_METRIC_SCOPES or type(value) is not int or value < 0:
+            raise ValueError("prune strategy savings are invalid")
+        if STRATEGY_METRIC_SCOPES[name] == "context":
+            context_bytes_freed += value
+    file_bytes_freed = bytes_before - bytes_after
+    file_pct = (round(100.0 * file_bytes_freed / bytes_before, 1)
+                if bytes_before else 0.0)
     return {
         "bytes_before": bytes_before,
         "bytes_after": bytes_after,
-        "freed_bytes": bytes_before - bytes_after,
+        "freed_bytes": file_bytes_freed,
         "est_tokens_before": bytes_before // 4,
         "est_tokens_after": bytes_after // 4,
-        "pct": round(100.0 * (bytes_before - bytes_after) / bytes_before, 1)
-        if bytes_before else 0.0,
+        "pct": file_pct,
+        "file_bytes_freed": file_bytes_freed,
+        "file_est_tokens_before": bytes_before // 4,
+        "file_est_tokens_after": bytes_after // 4,
+        "file_est_tokens_freed": file_bytes_freed // 4,
+        "file_pct": file_pct,
+        "context_bytes_freed": context_bytes_freed,
+        "context_est_tokens_freed": context_bytes_freed // 4,
     }
 
 

core/pysrc/_segmentslib.py

@@ -242,7 +242,7 @@ def seg_pr(data):
 
 
 def seg_prune(data, sid):
-    """Transcript-pruner indicator: cumulative reduction and age of the last
+    """Transcript-pruner indicator: model-context reduction and age of the last
     prune for this session, read from ~/.codearbiter/prune-state.json (written
     by prune-transcript.py). Fail-soft — returns None on any problem, and never
     raises, so it can never break statusline rendering."""
@@ -255,11 +255,16 @@ def seg_prune(data, sid):
         rec = st.get(sid) if isinstance(st, dict) else None
         if not isinstance(rec, dict):
             return None
-        pct = rec.get("pct") or 0
-        if pct <= 0:
+        file_freed = num(rec.get("file_bytes_freed", rec.get("freed_bytes")), 0)
+        if file_freed <= 0:
             return None
         age = human_dur(max(0, time.time() - num(rec.get("last_run_ts"), time.time())))
-        return f"{GREY}✂{RESET} {WHITE}{pct:.0f}%{RESET} {GREY}{age}{RESET}"
+        if "context_est_tokens_freed" not in rec:
+            pct = num(rec.get("file_pct", rec.get("pct")), 0)
+            return f"{GREY}✂{RESET} {WHITE}file:{pct:.0f}%{RESET} {GREY}{age}{RESET}"
+        context_tokens = max(0, num(rec.get("context_est_tokens_freed"), 0))
+        return (f"{GREY}✂{RESET} {WHITE}ctx:{fmt_tok(context_tokens)}{RESET} "
+                f"{GREY}{age}{RESET}")
     except Exception:  # noqa: BLE001
         return None