Skip to content

claude-ops/inventory: recover the bundled-skill lane on 2.1.263 and report integrity per lane #4048

Description

@kyle-sexton

Parent

Refs #4047. Unit 1 of 9 (tooling fix, plan Phases 1 to 3). Predecessor: none.

Agent Brief

Type: Bug

Summary: the inventory extractor's bundled-skill lane returns empty on Claude Code 2.1.263, which makes every native-overlap run report broken; recover the lane, report integrity per lane in both engines, close the reverse-parity blind spot, and seed the two missing pairs.

Current behavior:

  • The extractor selects the single longest printable run of the binary and looks up the registrar through the CJS getter shape registerBundledSkill:()=>xu. The 2.1.263 binary is a // @bun @bytecode layout fragmented into thousands of printable runs; the getter shape is gone (an ESM export list eo as registerBundledSkill names the registrar now), and registrations sit in runs as small as 1.3 KB, so the lane resolves nothing and the whole run is broken.
  • Integrity is all-or-nothing: one failed lane marks the entire extraction broken, and the overlap engine's detect exits 1 while still writing candidates.
  • The overlap self-check cannot see a description that names a bundled surface behind a presence condition ("where the bundled design skill is available") without the gate token.
  • Two store rows (doctor and skill-doctor against claude-ops:audit-skill-visibility) are absent from the seeded canonical pairs, so a consumer repository's detect never proposes them.
  • Two bundled registrations can share one name (on 2.1.263, design: the canvas skill registered by registerDesignCanvasSkill, model-invocable, and the claude.ai/design hub, model-disabled), and a name-keyed extraction reads whichever it meets last, so a per-registration field such as disable_model_invocation is unreliable for that name.

Desired behavior:

  • Region rule: from the first bundle marker to end of file, every printable run of at least 256 bytes, found with one regex pass over [\t\n\r\x20-\x7e]{256,}, joined with newlines; the extraction records run count, joined bytes, the region rule, registration tokens below the floor (counted, never silently lost), and elapsed seconds. The longest-run path stays as the fallback when no marker is found. Measured on the real binary, this floor recovers the named surfaces in about 4 seconds; a 64 KiB floor recovers 13 of 33 names and a 1-byte floor takes a minute.
  • Registrar discovery order: CJS getter, then the ESM export list (<ident> as registerBundledSkill), then the canary registration (the callee immediately preceding ({name:"doctor",aliases:["checkup"]); the notes record which route resolved (export-map, esm-export, canary); with none, the lane is broken with the existing error text.
  • Hoisted-constant names resolve by locality: the nearest preceding ident="kebab" binding, a farther binding never winning over a nearer one (the ehrpd phantom is the negative case); a single-character identifier is trusted only when that binding lies within a 64 KiB window; no preceding binding is unresolved, never guessed. Bundle markers are not module boundaries in the bytecode layout (the constants sit megabytes and about 170 marker occurrences ahead of their registrations), so locality is measured in bytes. A loop or template-literal registration is recorded as a dynamic_roster note; a registrar-identifier call whose object carries no name: is another module's function and is counted apart.
  • Each registration records user_invocable, disable_model_invocation, terminal_oriented, and survives_kill_switch when present. A function-valued disable_model_invocation (the verify registration's shape) is recorded as true with the key listed under flag_driven, matching the binary's own serializer. On 2.1.263, doctor reads model-disabled and terminal-oriented; simplify and run carry no invocation-control field.
  • Same-name bundled collision: two distinct bundled registrations sharing one name are both kept under bundled_skills.<name> as a list of registrations, each with collision: true, named in bundled_skill_notes.collisions; never merged and never last-writer-wins. The fixture is the two design objects.
  • The registrar-shaped-export advisory scan also matches the ESM form, and the known-registrar set gains registerDesignCanvasSkill and registerWorkflowAuthoringSkill.
  • Integrity carries a lanes block (builtin_commands, bundled_skills, plugin_backed, each {status, problems, advisories}); the top-level status is the worst lane, where broken means every lane is broken or the binary is unreadable and a run with at least one healthy lane is at most degraded. plugin_backed gains a canary (security-review). Existing exit mappings (ok 0, degraded 3, broken 1) are unchanged in both inventory.py --self-check and overlap.py detect.
  • detect on an inventory with one broken lane states per-lane floors and marks every candidate whose lane is broken with re_derivable: false, deciding the lane by the seeded class when the native name is absent from the extraction and by the observed class when present, in both directions on a collision; session-provided and marketplace candidates carry re_derivable: null. Where a bundled name carries a registration list, detect lists every registration in the candidate's evidence with its invocation mode. An inventory without lanes keeps today's behaviour.
  • The self-check emits an advisory (exit 3, never a break) for a frontmatter description that names a native surface by class word (bundled, built-in, plugin-backed built-in, session-provided) plus skill or command inside a when, where, or if clause with an availability word and no gate token in that clause; a seam-phrasing "if that plugin is installed" clause, a Not-for clause with no presence condition, and a plain mention with no availability word are not flagged.
  • --upstream-sha repeats, one value per upstream repository the store cites; a recorded commit matches when any provided value matches it.
  • The two audit-skill-visibility pairs are seeded into the canonical pairs with a why.
  • Skill bodies: the audit-native-overlap rule "if it reports broken, the report carries no native-side counts at all" becomes per-lane (counts omitted only for a broken lane, the lane and cause named); the inventory skill's integrity paragraph and its extraction reference describe the lanes block and the registration-list shape.
  • Evals: this repository's evals are evals.json in the skill-quality format and the CLI eval runner consumes a different case format, so "evals pass" means the deterministic evals-quality lint and the schema check pass after any expectation naming the old build or the old integrity wording is updated; then VALIDATED_AGAINST moves to 2.1.263.
  • Windows: where a Windows host with the 2.1.263 claude.exe is reachable, run the self-check there and record it; otherwise the CHANGELOG entry states the Linux-only validation basis and that the PE path is covered by a fixture.
  • Version: next minor of claude-ops at PR time; CHANGELOG states the rules, never the incident.

Key interfaces:

  • read_bundle() (extractor): returns the joined source under the new region rule; gains its first unit tests (fragmented runs, PE container, legacy single run, a registration below the floor counted not lost).
  • discover_registrar_route() and extract_bundled_skills(): three discovery routes, locality-based name resolution, the four new per-registration fields, the same-name registration list, registrations_of().
  • check_integrity(): returns lanes; the top-level status is the worst lane.
  • overlap.py cmd_detect(): reads lanes when present; re_derivable per candidate; registration-list aware.
  • overlap.py check_presence_mentions(): the new advisory for presence-gated mentions without the gate token.

Acceptance criteria:

  • python3 plugins/claude-ops/skills/inventory/scripts/test_inventory.py exits 0 and contains at least four test_read_bundle tests and one same-name collision test.
  • time python3 plugins/claude-ops/skills/inventory/scripts/inventory.py --binary-only --out /tmp/inv.json exits 0 in under 30 seconds (measured about 14 seconds on the Linux build); jq -e '[.bundled_skills.doctor, .bundled_skills.simplify, .bundled_skills.run, .bundled_skills.design, .bundled_skills["code-review"]] | all(. != null)' /tmp/inv.json and jq -e '.plugin_backed["security-review"] != null' /tmp/inv.json print true.
  • jq -e '.bundled_skills.doctor.disable_model_invocation == true and .bundled_skills.simplify.disable_model_invocation == null and .bundled_skills.doctor.terminal_oriented == true' /tmp/inv.json prints true; jq -r .bundled_skill_notes.registrar_route /tmp/inv.json prints esm-export; jq -r .integrity.status /tmp/inv.json prints ok or degraded; the bundled-skill keys do not contain ehrpd.
  • jq -e '.bundled_skills.design | (type == "array" and length == 2) and (map(.disable_model_invocation // false) | sort == [false, true])' /tmp/inv.json prints true.
  • python3 plugins/claude-ops/skills/audit-native-overlap/scripts/test_overlap.py exits 0; a fixture inventory with lanes.bundled_skills.status == "broken" and healthy other lanes makes detect exit 3 and mark at least one candidate re_derivable: false.
  • grep -c "no native-side counts at all" plugins/claude-ops/skills/audit-native-overlap/SKILL.md prints 0.
  • python3 plugins/claude-ops/skills/audit-native-overlap/scripts/overlap.py self-check --upstream-sha ed404106fcd80ba98ecb7c851e531dcb626d13b7 --upstream-sha d7dbd9a09f59775726ed14bbea8fc9dfdff62f7b exits 3 with no problems, and its advisories are the version-drift advisory plus exactly two reverse-parity advisories naming visualization:visualize and prototype:explore-directions.
  • jq '.pairs | length' plugins/claude-ops/skills/audit-native-overlap/reference/canonical-pairs.json prints 16 and a pair with native.name == "skill-doctor" exists.
  • grep -n 'VALIDATED_AGAINST = "2.1.263"' plugins/claude-ops/skills/inventory/scripts/inventory.py matches; bash plugins/skill-quality/scripts/check-evals-quality.sh plugins/claude-ops/skills/inventory/evals/evals.json exits 0; scripts/affected-tests.sh --run passes; the claude-ops version is above 0.47.1 with a CHANGELOG entry.
  • The PR is opened as a draft, its body follows the PR-body contract and quotes the two seeded pairs, and it flips to ready when green.

Out of scope:

  • The integration field, the two new grammars, and any convention edit beyond none: those are the policy unit (unit 2).
  • Editing any skill body's routing text or any store row's verdict.
  • The follow-up items listed on the parent container.

Blocked by

None; can start immediately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.priority: mediumReal value, no hard deadline; normal backlog flow.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions