fix(cli): stop downloading full GitHub monorepo for scoped npm specs#39
Merged
Conversation
`ask docs add npm:@mastra/client-js` was extremely slow because three bugs stacked up: 1. `parseEcosystem` bailed on any input containing `/`, so scoped names like `npm:@mastra/client-js` were returned with `ecosystem: undefined` and a garbage spec, producing a `registry/npm/npm:` lookup URL. 2. `fetchRegistryEntry` did not URL-encode segments, so even a correct scoped name would have collided with the server's two-segment slug parser. 3. On a registry miss with `npm:` prefix, the CLI fell back to the npm ecosystem resolver, which extracts the repository URL from npm metadata and hard-codes `source: 'github'`. For `@mastra/client-js` that meant downloading the entire `mastra-ai/mastra` monorepo tarball — exactly the surprise the user hit. Fixes: - `parseEcosystem` now splits on the first colon as long as it appears before any slash, so scoped ecosystem specs parse correctly while `owner/repo` shorthand still bypasses the branch. - `fetchRegistryEntry` encodes both path segments so scoped names survive the round trip to the registry API. - The registry server decodes each catch-all segment, since Nitro only decodes `%40` automatically and leaves `%2F` intact. - On a registry miss with an explicit `npm:` prefix, honor the user's intent and fetch the single npm tarball via `NpmSource` instead of pulling the upstream repo. The library name is slugified the same way the registry server slugifies monorepo entries (`@mastra/client-js` → `mastra-client-js`) and the original package name is preserved on the `package` field so the tarball fetch and `node_modules` local read both resolve correctly. Verified end-to-end against `/tmp/ask-test`: registry lookup now logs the correct spec, the npm tarball path fetches 25 doc files in seconds, and the slugified directory (`mastra-client-js@1.13.2`) is written without the monorepo detour.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Deploying ask-registry with
|
| Latest commit: |
3d3aac1
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://104d3beb.ask-registry.pages.dev |
| Branch Preview URL: | https://fix-scoped-npm-registry-look.ask-registry.pages.dev |
Contributor
There was a problem hiding this comment.
No issues found across 4 files
Requires human review: This PR modifies both CLI source resolution logic and a server-side API route handler, which requires architectural context to ensure no regressions in package fetching.
Architecture diagram
sequenceDiagram
participant User
participant CLI as CLI (packages/cli)
participant Registry as Registry API (Nitro Server)
participant NPM as NPM Registry
participant GH as GitHub API/Tarball
User->>CLI: ask docs add npm:@mastra/client-js
CLI->>CLI: CHANGED: parseEcosystem()<br/>(Identifies 'npm' before the scoped '/')
CLI->>Registry: CHANGED: GET /api/registry/npm/%40mastra%2Fclient-js<br/>(URL-encodes scoped name)
Registry->>Registry: NEW: Decode slug segments<br/>(Handles %2F for scoped names)
alt Registry Hit
Registry-->>CLI: Return package metadata
else Registry Miss (404)
Registry-->>CLI: Not Found
CLI->>CLI: NEW: Check ecosystem == 'npm'
alt Ecosystem is 'npm' (NEW Path)
CLI->>CLI: NEW: Slugify dir name (@mastra/client-js -> mastra-client-js)
CLI->>NPM: Fetch specific package tarball
NPM-->>CLI: Return tarball data
else Other Ecosystem (Fallback)
CLI->>GH: Download full repository source
GH-->>CLI: Return large monorepo tarball
end
end
CLI->>CLI: Extract docs to .ask/docs/<name>
CLI-->>User: Success
This was referenced Apr 9, 2026
Merged
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ask docs add npm:@mastra/client-jswas extremely slow because three bugs stacked up and ultimately caused the CLI to download the entiremastra-ai/mastramonorepo tarball instead of the single scoped package. This PR fixes all three.Root cause
parseEcosystembailed on scoped names. The old guard (!input.includes('/')) madenpm:@mastra/client-jsreturnecosystem: undefined, so the registry lookup ran with a garbage spec (registry/npm/npm:).fetchRegistryEntrydid not URL-encode segments. Even with a correct name,@mastra/client-jswould have collided with the server's two-segment slug parser.source: 'github'. The npm ecosystem resolver extractsrepository.urlfrom npm metadata and hands off toGithubSource— which downloaded the upstream repo in full.The user's
npm:prefix was an explicit request to use npm, so the fallback ignored their intent.Fixes
packages/cli/src/registry.tsparseEcosystemnow splits on the first colon as long as it appears before any slash. Scoped ecosystem specs parse correctly;owner/reposhorthand still bypasses the branch.fetchRegistryEntryURL-encodes both path segments so scoped names survive the round trip.apps/registry/server/api/registry/[...slug].get.ts— decodes each catch-all segment. Nitro only auto-decodes%40, so%2Fhas to be handled explicitly.packages/cli/src/index.ts— on a registry miss with an explicitnpm:prefix, the CLI now fetches viaNpmSource(single tarball +node_moduleslocal-first) instead of the github resolver fallback. The library name is slugified the same way the registry server does (@mastra/client-js→mastra-client-js) and the original package name is preserved on thepackagefield.packages/cli/test/registry.test.ts— 4 regression tests coveringparseEcosystemfor scoped names, simple prefixes, andowner/reposhorthand.Test plan
bun run --cwd packages/cli test— 259 pass (was 255)bun run --cwd packages/cli lint— cleanask docs add npm:@mastra/client-jsin an empty project now fetches 25 doc files from the single npm tarball in seconds and writes.ask/docs/mastra-client-js@1.13.2/. Previously it would try to download the fullmastra-ai/mastramonorepo.Notes
queryCollectiontype diagnostics in[...slug].get.tsare unrelated to this change.Summary by cubic
Fixes CLI handling of scoped npm specs so we fetch a single npm tarball instead of the full GitHub monorepo.
ask docs add npm:@mastra/client-jsis now fast and writes docs for the correct package.Bug Fixes
npm:@mastra/client-jsparse correctly.%2Fin server slug segments to round-trip scoped names reliably.npm:prefix, fetch viaNpmSource, slugify the directory name (@mastra/client-js→mastra-client-js), and keep the originalpackagefield.Tests
parseEcosystemand owner/repo shorthand.Written for commit 3d3aac1. Summary will update on new commits.