feat(cli): PM-driven install flow with ask.json#52
Conversation
Introduces ask.json + ask install architecture where PM lockfile is the single source of truth for versions. Refs #51
Replaces Config/Lock with the PM-driven install model: - AskJsonSchema: top-level libraries[] with PM-driven (npm:) and standalone github entries (require explicit ref). - ResolvedJsonSchema: per-entry resolved version + content hash + format tag, used by .ask/resolved.json as the install short-circuit cache. Drops the legacy config.ts and lock.ts modules. Spec reference: pm-driven-install-20260410.
Renames packages/cli/src/manifest/ to lockfiles/ and decomposes the
monolithic NpmManifestReader into per-format readers (bun, npm, pnpm,
yarn, package-json). The npm-ecosystem facade now lives in
lockfiles/index.ts as npmEcosystemReader, probing the chain in priority
order.
Legacy getReader('npm') is kept as a thin back-compat shim until the
legacy addCmd is removed in Phase E.
Replaces .ask/config.json + .ask/ask.lock + `ask docs *` with a flat
top-level command surface (`ask install/add/remove/list`) and a root
ask.json that declares which libraries the project wants documentation
for.
Architecture
- packages/schema: AskJsonSchema (PM-driven `{spec}` and standalone
github `{spec, ref}`) and ResolvedJsonSchema (per-entry resolved
version + content hash + format tag).
- packages/cli/src/lockfiles: per-format readers (bun/npm/pnpm/yarn/
package.json) under a single npmEcosystemReader facade. Renamed from
packages/cli/src/manifest/.
- packages/cli/src/install.ts: runInstall orchestrator. PM-driven
entries resolve via the lockfile chain; standalone github entries
use their explicit ref. Per-entry failures emit warnings and exit
code is always 0 (postinstall-friendly, FR-10).
- packages/cli/src/io.ts: ask.json + .ask/resolved.json helpers.
Removed all Lock/Config helpers.
- packages/cli/src/storage.ts: listDocs joins ask.json with
resolved.json. Declared-but-not-installed entries surface as
'unresolved'.
- packages/cli/src/index.ts: flat command surface; legacy docsCmd /
syncCmd / deprecated docs list wrapper deleted.
- packages/cli/src/spec.ts: new parseSpec / libraryNameFromSpec /
slugifyNpmName helpers, the single source of truth for translating
an ask.json spec into a library slug.
Behavior changes
- `ask install` bootstraps an empty ask.json when missing (FR-8).
- Resolved-cache short-circuit on .ask/resolved.json (NFR-1, AC-9).
- intent-skills format is detected at first-pass via the existing
local-intent adapter; the AGENTS.md marker block writer is reused
unchanged.
- ignore-files now manages .ask/resolved.json alongside .ask/docs/.
Tests
- Schema validation tests for AskJson and ResolvedJson.
- Lockfile reader unit tests (bun/npm/pnpm/yarn/package.json).
- Install orchestrator tests covering bootstrap, warn-and-skip,
empty libraries list, and force re-fetch.
- CLI command surface tests for install/add/remove/list.
Docs
- README + CLAUDE.md rewritten to describe the new flow.
Related: pm-driven-install-20260410.
- Sweep AC-10 violations: rewrite skills/add-docs/SKILL.md and references/recovery.md for the new ask add / ask install surface; sed-replace ask docs add|sync|list|remove → ask add|install|list|remove in setup-docs, sync-docs, inline-pipeline.md, and the registry Vue pages. - Update resolver error messages (npm/pypi/pub/maven) to point at github:owner/repo --ref instead of the deprecated owner/repo shorthand. - Update skill.ts and agents.ts comment templates to reference the new command names; refresh the empty-list message in list/render.ts. - Update test/skill.test.ts and test/registry.test.ts assertions for the new command tokens. - Reconcile spec.md FR-5 with implementation: github specs require an explicit --ref, no implicit 'main' default.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
0 issues found across 4 files (changes from recent commits).
Requires human review: This is a major architectural refactor and feature change that replaces the core config and installation logic. The impact and risk are too high for auto-approval.
There was a problem hiding this comment.
1 issue found across 64 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/cli/src/install.ts">
<violation number="1" location="packages/cli/src/install.ts:121">
P2: Normalize standalone GitHub refs to match the GitHub source’s resolvedVersion (strips a leading `v` for tags). Otherwise the resolved-cache check and docs path never match for tags like `v1.2.3`, forcing a re-fetch every install.</violation>
</file>
Architecture diagram
sequenceDiagram
participant User as "CLI / User"
participant Orchestrator as "Install Orchestrator (install.ts)"
participant Lockfiles as "NPM Ecosystem Reader"
participant Cache as "Resolved Cache (.ask/resolved.json)"
participant Source as "Source Adapters (npm, github, web)"
participant FS as "File System (.ask/docs, AGENTS.md)"
User->>Orchestrator: NEW: ask install | add
alt NEW: ask.json missing
Orchestrator->>FS: NEW: Bootstrap empty ask.json
end
Orchestrator->>FS: NEW: Read ask.json (Libraries list)
loop for each library entry
alt NEW: PM-driven (e.g., npm:next)
Orchestrator->>Lockfiles: CHANGED: Resolve version from lockfile
Note over Lockfiles: Priority: bun.lock -> package-lock.json ->\npnpm-lock.yaml -> yarn.lock -> package.json
Lockfiles-->>Orchestrator: Resolved version
else NEW: Standalone (e.g., github:owner/repo)
Note over Orchestrator: Use explicit --ref from ask.json
end
Orchestrator->>Cache: NEW: Check .ask/resolved.json
alt NEW: Cache Hit (Same spec + version + files exist)
Cache-->>Orchestrator: Skip fetch
else NEW: Cache Miss / Force
opt NEW: npm: entry
Orchestrator->>Source: CHANGED: Check local node_modules
Source-->>Orchestrator: Return local docs if version matches
end
alt Local Miss
Orchestrator->>Source: Fetch documentation
Source->>Source: Registry lookup / Network download
Source-->>Orchestrator: Doc files + Content Hash
end
Orchestrator->>FS: CHANGED: Materialize .ask/docs/<name>@<ver>/
Orchestrator->>FS: NEW: Generate Claude Skill (.claude/skills/)
Orchestrator->>Cache: NEW: Update resolved entry + fetchedAt
end
end
Orchestrator->>FS: CHANGED: Update AGENTS.md (Marker block)
Orchestrator->>FS: CHANGED: manageIgnoreFiles (.gitignore, .prettierignore)
Orchestrator-->>User: Exit 0 (Warnings for skipped/failed entries)
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Strip a leading `v` from `lib.ref` in installOne so the resolved-cache check and docs-path (`libName@resolvedVersion`) match the version that GithubSource.fetch() returns. GithubSource always passes `ref` as `opts.tag` and normalizes it via `opts.tag?.replace(/^v/, '')`, so tags like `v1.2.3` become `1.2.3` in resolvedVersion. Without the matching normalization in installOne, the cache key contained `v1.2.3` while stored entries used `1.2.3`, causing a re-fetch on every `ask install`. RE_LEADING_V defined at module scope per e18e/prefer-static-regex.
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Requires human review: Major architectural refactor (5k+ lines) that replaces the core configuration system and CLI command structure. High impact and risk require human verification of the new logic.
…ers, GITHUB_TOKEN auth) (#117) * chore: bump vendor/opensrc submodule to upstream v0.7.3 (f96078a) Pulls in 9 upstream commits (2026-04-14 .. 2026-06-23): - opensrc fetch subcommand + shared core::fetcher module (#53, #56) - Bitbucket Cloud support via BITBUCKET_TOKEN + auth docs (#52) - Lockfile parsers rewritten with transitive pnpm resolution (#51) - Structured error handling via thiserror across the CLI (#56) - Authenticated clone host validation fix (#66) - Trusted npm publishing, v0.7.2/v0.7.3 releases, docs favicon * fix(cli): rewrite pnpm/yarn lockfile parsers with transitive pnpm resolution Port of vercel-labs/opensrc#51. The previous regex-based parsers misidentified versions in common real-world cases: yarn multi-specifier headers only matched the first specifier, pnpm peer-dep suffixes like 18.2.0(react@17.0.0) leaked false matches for the inner package, and pnpm monorepos returned whichever version the regex engine hit first. - pnpm: indent-aware stack parser covering v5 through v9 (importers, devDependencies, optionalDependencies, nested peer-dep suffixes) with BFS transitive resolution over the snapshots/packages dep graph - yarn: block-based parser handling classic v1 and Berry v2+ in one path - package.json fallback now skips protocol versions (workspace:*, link:, file:, git+..., npm: aliases) via isRegistryVersion - fixtures (yarn v1, yarn Berry, pnpm v9 monorepo) and 58 unit tests ported from upstream * feat(cli): add ask fetch command to warm the source cache Port of opensrc fetch (vercel-labs/opensrc#53/#56). ask fetch <spec...> resolves each spec through the shared ensureCheckout helper without printing paths — the prefetch counterpart to ask src / ask docs. - supports multiple specs, -q/--quiet, per-spec ✓ fetched / already cached lines and a summary; per-spec failures print to stderr and the command exits non-zero at the end - EnsureCheckoutResult gains a fromCache flag so callers can tell a store hit from a fresh fetch - warms the cache that ask add's offline-first docs-path prompt reads, removing the need to abuse ask docs for prefetching * feat(cli): support GITHUB_TOKEN for private repos with strict host validation Port of vercel-labs/opensrc#52 + #66 (scoped to GitHub — ASK's store layout is github.com-only for MVP, so the GitLab/Bitbucket token paths do not apply). - authenticatedCloneUrl parses the clone URL and injects x-access-token:<GITHUB_TOKEN> ONLY on an exact https://github.com host match; prefix-confusable hosts (github.com.evil.com, evilgithub.com, subdomains), non-https schemes, and ssh remotes pass through untouched (#66 host-confusion fix) - auth is applied at the exec boundary (clone, ls-remote); error messages keep the token-free URL and redactToken scrubs git/ execFileSync output so the secret never reaches logs or thrown errors - tar.gz fallback switches to the api.github.com tarball endpoint when a token is set: undici drops the Authorization header on the cross-origin redirect to codeload, but GitHub embeds a temporary token in the redirect Location, so private downloads still work * test(cli): disable gpg signing in git test fixtures Machines with global tag.gpgsign=true failed createLocalRemote setup — git tag <name> becomes a signed annotated tag that requires a message. Pass -c tag.gpgsign=false / -c commit.gpgsign=false so the fixtures are hermetic regardless of the developer's global git config. * docs: document ask fetch and GITHUB_TOKEN support - README / SKILL.md: add ask fetch to the command tables, note the cache-warming use case and private-repo token support - CLAUDE.md: update the top-level command list, point cache-warming guidance at ask fetch, and add gotchas for the format-aware lockfile parsers and the exact-host token injection rule * chore: apply AI code review suggestions - pnpm parser: support v5 /name/version keys in the packages:/snapshots: pass with a last-slash split (scoped names intact, _peerhash stripped) and canonical name@version node keys so v5 fallback and transitive resolution connect (cubic review) - ensure-checkout: propagate GithubSource's store-hit via a new FetchResult.fromStoreCache flag so fromCache is not misreported as a fresh fetch when the source found a ref-candidate cache entry with zero network I/O (cubic review) * refactor(cli): split parsePnpmLock state machine into per-frame handlers Addresses the SonarCloud cognitive-complexity finding (85 > 15) raised via CodeRabbit review. The line loop now delegates to one small handler per frame kind (root/importers/importer/depGroup/depBlock/packages/ pkgEntry/pkgDeps) over a shared ParseState, with the root scope represented by an empty stack instead of a sentinel frame. Also clears the Codacy warnings on the same function: no while(true), no non-null assertions (currentFrame returns Frame | undefined; resolveTransitive uses index-pointer BFS instead of queue.shift()!). Behavior unchanged — covered by the 84 lockfile parser tests. * chore: drop unnecessary optional chain on fetch result fetcher.fetch() returns a non-nullable FetchResult, so the ?. on fromStoreCache access was dead — flagged by Codacy. * fix: restore null-safe access to the injected fetcher result bec57a0 dropped the optional chain because FetchResult was declared non-nullable, but the ensureCheckout test seam legitimately simulates a successful fetch by materializing the checkout dir without returning a FetchResult — cache-sharing.test.ts crashed in CI. Widen the seam type to Promise<FetchResult | undefined> so the type states the real contract and the ?. is justified rather than dead.
Closes #51
Summary
Replaces
.ask/config.json+.ask/ask.lock+ theask docs *namespace with a PM-driven install flow rooted in a top-level
ask.json. The project's package manager lockfile is now the singlesource of truth for dependency versions, and ASK is repositioned as a
downstream tool of the user's PM (the same relationship Prisma /
TypeScript have to npm).
Architecture
packages/schema/src/{ask-json,resolved}.ts. Two newentry shapes (PM-driven
{spec}and standalone{spec, ref, docsPath?})validated via Zod. Drops the legacy
Config/Lockschemas entirely.packages/cli/src/lockfiles/. The legacymanifest/module is renamed and decomposed into per-format readers(
bun,npm,pnpm,yarn,package-json) under a singlenpmEcosystemReaderfacade that probes them in priority order.packages/cli/src/install.ts:runInstall.Reads
ask.json, resolves each entry against the right source oftruth, short-circuits via
.ask/resolved.json, dispatches to theexisting source adapters unchanged, and writes
.ask/docs/,AGENTS.md, and.claude/skills/<name>-docs/. Per-entry failuresemit warnings and exit code is always 0 (postinstall-friendly).
ask install | add | remove | list. Thedocsnamespace is gone — there is no deprecated wrapper.packages/cli/src/spec.ts. Single source oftruth for translating an
ask.jsonspec into a library slug.Existing source adapters (
sources/{npm,github,web,llms-txt}.ts) andthe registry resolver are intentionally untouched.
Behavioral changes
ask installbootstraps an emptyask.jsonwhen missing (FR-8)..ask/resolved.json(NFR-1, AC-9).local-intentadapter; the AGENTS.md marker block writer is reused.ignore-filesnow manages.ask/resolved.jsonalongside.ask/docs/.--ref. Thespec originally said "default to
mainwith a warning"; duringimplementation we rejected the silent default because users could
not reliably tell which version they were pinned to. Spec amended.
Test plan
bun run --cwd packages/schema test— 47 passbun run --cwd packages/cli build && lint && test— 219 pass, lint cleanbun run --cwd apps/registry build— cleanask docs add/.ask/config.json/.ask/ask.lockin source, tests, fixtures,README, CLAUDE.md, skills, registry UI, or eval docs. Only
historical-context lines (CLAUDE.md gotchas) retain the legacy
names to document the rename.
bun init+bun add next,run
ask add npm:next && ask install. Verify.ask/docs/next@<ver>/,AGENTS.mdblock,.claude/skills/next-docs/SKILL.mdare produced and versionmatches
bun.lock.github:entry andrun
ask install. Verify.ask/docs/<name>@<ref>/is populated.ask installtwice in a row and confirmthe second run reports "already up to date" with no source
fetches (NFR-1, AC-9).
Track:
pm-driven-install-20260410Summary by cubic
Adds a PM-driven install flow with a root
ask.jsonandask install, making your package manager lockfile the single source of truth for versions. Removesask docs *and the legacy.ask/config.json/.ask/ask.lock, and short-circuits repeat installs via.ask/resolved.json. Implements #51.New Features
ask.jsonentries:npm:<pkg>andgithub:<owner>/<repo>(requires--ref).ask installreadsbun.lock/package-lock.json/pnpm-lock.yaml/yarn.lock/package.json, writes.ask/docs/,AGENTS.md, skills, and caches to.ask/resolved.json; bootstraps emptyask.json; per-entry failures warn with exit code 0.npmEcosystemReader; CLI surface is flat:ask install | add | remove | list(noask docs *).vin GitHub--refso cache keys and.ask/docs/<name>@<ver>/match, preventing unnecessary re-fetches.Migration
ask add npm:<pkg>orask add github:<owner>/<repo> --ref <tag>, then runask install(works well inpostinstall)..ask/docs/and.ask/resolved.jsonare managed by ASK and added to ignore files automatically.Written for commit b6e2395. Summary will update on new commits.