Phase 46.0 — tiered script-policy gate (v0.23.0 released)

.github/workflows/ci.yml

@@ -45,8 +45,12 @@ jobs:
           key: ${{ runner.os }}-cargo-lint-${{ env.RUST_TOOLCHAIN }}-${{ hashFiles('**/Cargo.lock') }}
           restore-keys: ${{ runner.os }}-cargo-lint-${{ env.RUST_TOOLCHAIN }}-
 
+      # --all-targets covers library, binary, test, example, and bench
+      # targets. The bare `--workspace` variant misses lints in test
+      # modules (Chunk 4 caught two pre-existing `assert_eq!(x, false)`
+      # lints in build_state.rs that this stricter invocation surfaces).
       - name: Clippy
-        run: cargo clippy --workspace -- -D warnings
+        run: cargo clippy --workspace --all-targets -- -D warnings
 
       - name: Check formatting
         run: cargo fmt --check

Cargo.toml

@@ -13,6 +13,7 @@ members = [
     "crates/lpm-extractor",
     "crates/lpm-workspace",
     "crates/lpm-security",
+    "crates/lpm-sandbox",
     "crates/lpm-runner",
     "crates/lpm-runtime",
     "crates/lpm-task",
@@ -35,7 +36,7 @@ members = [
 default-members = ["crates/lpm-cli"]
 
 [workspace.package]
-version = "0.22.0"
+version = "0.23.0"
 edition = "2024"
 license = "MIT OR Apache-2.0"
 repository = "https://github.com/lpm-dev/rust-client"
@@ -53,6 +54,7 @@ lpm-lockfile = { path = "crates/lpm-lockfile" }
 lpm-extractor = { path = "crates/lpm-extractor" }
 lpm-workspace = { path = "crates/lpm-workspace" }
 lpm-security = { path = "crates/lpm-security" }
+lpm-sandbox = { path = "crates/lpm-sandbox" }
 lpm-runner = { path = "crates/lpm-runner" }
 lpm-runtime = { path = "crates/lpm-runtime" }
 lpm-task = { path = "crates/lpm-task" }
@@ -157,6 +159,12 @@ filetime = "0.2"
 # Pure-Rust, GNU patch format, no fuzzy matching by default.
 diffy = "0.4"
 
+# Phase 46 P2: POSIX shell tokenizer for the Layer 1 static-gate
+# classifier. Pure-Rust, no dependencies, understands standard shell
+# quoting so `lpm-security::static_gate` can split script bodies into
+# argv without spawning a real shell.
+shlex = "1.3"
+
 [profile.release]
 opt-level = 3
 lto = true

bench/baselines/2026-04-23-46.0-macos-arm64.md

@@ -0,0 +1,213 @@
+# Baseline: 2026-04-23 — Phase 46.0 tag cut — macOS arm64
+
+> Phase 46.0 close-out bench readings on the release-manager
+> machine, captured during the 46.0 tag-cut validation.
+> Companion to the [Next.js validation report](../../../../a-package-manager/DOCS/new-features/37-rust-client-phase46-nextjs-validation.md).
+
+## Environment
+
+| | |
+|---|---|
+| Date | 2026-04-23 |
+| Machine | arm64, Darwin 25.3.0 (Apple Silicon) |
+| LPM binary | `/tmp/lpm-rs-46-tag/release/lpm-rs` built from `phase-46` tip `f19d23e` |
+| LPM version string | `lpm 0.22.0` (pre-`v0.23.0` tag-bump) |
+| Methodology | `bench/run.sh` medians, plus a custom A/B runner (`/tmp/lpm-ab-bench.sh`) for cross-binary comparison |
+| Harness note | Bench workdir redirected to `/tmp/lpm-ab-work` via `BENCH_WORK_DIR` (Chunk 7 commit `ed001fa`) so VS Code's `--no-ignore` rg search doesn't storm the process table during cold-install cycles |
+
+## §12.7 cold-install-triage bench (Axis 1 + Axis 2)
+
+Captured 2026-04-22 23:45 via `bench/run.sh cold-install-triage`
+against the in-tree `bench/project/` fixture (17 direct deps → 51
+packages). RUNS=3, median per axis:
+
+| Axis | Policy | autoBuild | Wall-clock median | Delta vs deny | Target |
+|------|--------|-----------|-------------------|---------------|--------|
+| 1 | deny | off | 820 ms | — | — |
+| 1 | triage | off | 696 ms | **−15 %** | ≤ 5 % regression |
+| 2 | deny | on | 658 ms | — | — |
+| 2 | triage | on | 680 ms | **+3 %** | ≤ 5 % regression |
+
+Both axes inside the §12.7 v2.11 budget. Axis 1's −15 % reading
+is noise-dominated at RUNS=3 (the meaningful claim is "triage is
+no slower than deny"). Axis 2 is +3 % control-path overhead.
+
+v2.11 caveat preserved: **Axis 2 does not exercise P5 sandbox
+spawn or P6 tier auto-execution on this fixture.** The 51-package
+tree's scripts are all `prepare` / `prepublishOnly`, neither of
+which is in `EXECUTED_INSTALL_PHASES`
+([lpm-security/src/lib.rs:70][executed]), so `build::run`'s
+scriptable set is empty, the sandbox never spawns, and Axis 2
+measures only the tier-evaluation control-path walk. True
+execution-path benchmarking needs a pinned script-bearing fixture
+and is deferred.
+
+[executed]: ../../crates/lpm-security/src/lib.rs
+
+## §13.1 A/B cross-binary (main vs phase-46 under deny, 277 pkgs)
+
+A/B between `origin/main @ 5a4c33f` (v0.22.0) and `phase-46 @ f19d23e`
+on a larger 277-package fixture at `/tmp/lpm-large-fixture`
+(webpack-adjacent mix: lodash / axios / express / zod / eslint /
+typescript / vitest / … — representative of a real team project,
+roughly 5× the scale of `bench/project/`). Both binaries invoked
+with bare `lpm install --allow-new` (phase-46's default
+`script-policy = "deny"` kicks in through package.json /
+config.toml precedence, same output as pre-phase-46). Wipe
+sequence between iters: `node_modules lpm.lock lpm.lockb
+~/.lpm/cache ~/.lpm/store`.
+
+RUNS=10 with order alternation per pair (A→B on odd iters,
+B→A on even) so neither binary gets a systematic CF-edge-warm
+advantage:
+
+| Iter | Order | main (ms) | phase-46 (ms) |
+|------|-------|-----------|---------------|
+| 1 | A→B | 2491 | 2754 |
+| 2 | B→A | 2600 | 2771 |
+| 3 | A→B | 2993 | 2765 |
+| 4 | B→A | 2624 | 2719 |
+| 5 | A→B | 2740 | 3148 |
+| 6 | B→A | 2712 | 2915 |
+| 7 | A→B | 2389 | 2915 |
+| 8 | B→A | 2563 | 2748 |
+| 9 | A→B | 2437 | **8044 ⚠** |
+| 10 | B→A | 2593 | 2803 |
+
+Iter 9's phase-46 outlier (8044 ms) is an APFS `extract`
+scheduling hiccup, not a phase-46 regression — per-stage JSON
+from an earlier run with a similar outlier showed
+`extract_sum ≈ fetch_ms` (1:1 ratio, meaning extracts ran nearly
+serial vs the usual 5× parallelism). Median-of-10 discards it
+cleanly.
+
+### Median of 10
+
+| | median wall-clock | delta |
+|---|---|---|
+| main (v0.22.0 / `5a4c33f`) | **2600 ms** | — |
+| phase-46 (`f19d23e` tip) | **2803 ms** | **+7.8 %** |
+
+Per the 2026-04-10 baseline doc limitations:
+> "Numbers can fluctuate by upstream registry conditions. …
+> variations under ~10 % should be ignored."
+
+**+7.8 % sits at the noise floor for this class of measurement.**
+Process-count delta across 10 iterations was ±5 (normal OS jitter),
+confirming no per-install process leak under either binary.
+
+### What the fix achieved
+
+The pre-fix reading of the same bench on the same binary-phase-46
+tip (before commit `f19d23e`) was **+32 %** — ~770 ms of serial
+per-package overhead in `build_blocked_set_metadata`, which
+`.await`ed `get_package_metadata` + `fetch_provenance_snapshot`
+one package at a time. `f19d23e` fans the loop out via
+`futures::future::join_all`, saving ~570 ms / 277 packages and
+dropping the cross-binary delta onto the noise floor.
+
+### Tree parity
+
+| | Packages | Store size | node_modules top-level |
+|---|---|---|---|
+| main | 277 | 25 MB | 18 |
+| phase-46 | 277 (**identical set**) | 25 MB | 18 |
+
+Resolved trees verified byte-identical via JSON envelope
+comparison (`set(main["packages"]) == set(phase46["packages"])`,
+diff = 0). Phase-46 does not skip or short-circuit any work.
+
+## Per-stage JSON breakdown (post-fix, cold edge)
+
+Single-iteration `--json` capture after the fix, same 277-pkg
+fixture, A→B pair (main cold, phase-46 on warm edge):
+
+```
+run          count  total resolve  fetch  link  initial_batch_ms
+main_cold      277   2376    1932    378    44             885
+p46_warm       277   3335    1962    973    54             884
+p46_cold       277   2583    1887    295    52             844
+main_warm      277   2560    2153    342    43             997
+```
+
+Cold-vs-cold (apples-to-apples):
+- `total_ms` delta: **+207 ms (+8.7 %)**
+- `resolve_ms` delta: **−45 ms (−2.3 %)** — phase-46 ≥ main
+- `fetch_ms` delta: **−83 ms (−22.0 %)** — phase-46 ≥ main
+- `link_ms` delta: **+8 ms (+18.2 %)** — within jitter on a 44 ms base
+- `sum(stages)` delta: phase-46 −120 ms vs main
+- `UNACCOUNTED (total − sum_stages)` delta: **+327 ms**
+
+The +327 ms "unaccounted" sits in the post-stage
+`capture_blocked_set_after_install_with_metadata` pass:
+per-package `compute_script_hash` + `read_install_phase_bodies`
+on the store tree, plus the trust-snapshot write. That's real
+Phase 46 security work (classifier output, `static_tier`
+annotation, published_at / behavioral_tags_hash persistence,
+trust-delta fingerprinting) — not in main at all. **Acceptable at
+this scale and inside the 10 % noise floor.** If it ever shows up
+as a user complaint on large monorepos (1000+ packages where
+300 ms becomes 1 s+), the per-package `package.json` parse is
+easily parallelizable via `rayon::par_iter` or
+`tokio::task::spawn_blocking` fanout.
+
+## Other phase-46 serial-await hotspots (for the backlog)
+
+Not triggered on the standard `lpm install --allow-new` invocation
+measured above, but same serial-await pattern as the
+`build_blocked_set_metadata` hot spot:
+
+- [`install.rs:1674`](../../crates/lpm-cli/src/commands/install.rs#L1674)
+  — minimum-release-age cooldown gate. Only fires on
+  `!allow_new && !used_lockfile` (first install with cooldown, no
+  lockfile present). Users hit this once, either bypass with
+  `--allow-new` or set `minimumReleaseAge: 0` in package.json.
+- [`install.rs:1803`](../../crates/lpm-cli/src/commands/install.rs#L1803)
+  — provenance-drift gate. Short-circuits when
+  `trustedDependencies` has no rich-form entries (the common case
+  today — very few projects have written rich approvals yet).
+
+Both would benefit from a fanout but neither regresses the common
+path. Queued for a follow-up perf pass if usage patterns ever
+exercise them.
+
+## How to reproduce
+
+```bash
+# 1. Build both binaries (separate target dirs so neither trashes
+#    the other's incremental cache).
+cd /Users/tolgaergin/Documents/Projects/tolgaergin/lpm/rust-client
+CARGO_TARGET_DIR=/tmp/lpm-rs-46-tag cargo build --release
+git worktree add /tmp/lpm-rs-main origin/main
+(cd /tmp/lpm-rs-main && CARGO_TARGET_DIR=/tmp/lpm-rs-main-target cargo build --release)
+
+# 2. Drop the larger fixture.
+mkdir -p /tmp/lpm-large-fixture
+# … package.json contents inline in the bench commit message …
+
+# 3. Run the A/B. The script is /tmp/lpm-ab-bench.sh — not
+#    committed to the repo because it duplicates
+#    `median_ms_ab_with_setup` for cross-binary use. Paste inline
+#    from the commit or re-derive from bench/run.sh.
+RUNS=10 /tmp/lpm-ab-bench.sh
+```
+
+## Limitations
+
+- **Single machine.** Apple Silicon / Darwin 25.3.0. Linux x64
+  numbers still to come from a CI-class runner; the 10 % noise
+  floor observation holds across platforms per prior baselines.
+- **No isolated network.** Bench hits `lpm.dev` + `registry.npmjs.org`
+  over the release-manager's ISP. DNS was normal at measurement
+  time (verified via `dig`: 20–50 ms / query). Prior tag-cut
+  attempts on this same day were distorted by a local DNS / VS
+  Code rg-process-leak interaction — see the commit message on
+  `ed001fa` for the harness-side fix.
+- **RUNS=10 for the A/B.** Enough to resolve the +7.8 % signal
+  above the noise floor but not enough to publish sub-percent
+  precision. Acceptable for the release-gate check.
+- **Fixture is synthetic** (webpack-adjacent mix). Real-world
+  installs with Next.js + Tailwind + many workspace members may
+  have different profiles; the Next.js 16.2.4 reference install
+  captured in the validation doc complements this data at the
+  small-tree end (32 packages, 1 amber-by-design postinstall).