feat(copilot): mount the docs corpus in the VFS with path-scoped search_docs - #6389
feat(copilot): mount the docs corpus in the VFS with path-scoped search_docs#6389j15z wants to merge 24 commits into
Conversation
…ocs; serve openapi.json publicly - search_docs server tool: same vector search over docs_embeddings plus an optional docs/documentation/... VFS path prefix mapped onto a source_document scope (covers both <tail>.mdx and <tail>/... layouts); unscoped searches exclude academy/ and api-reference/ rows so the scope is exactly the Documentation tab - @docs chat context repointed to the new tool; display label updated - apps/docs now serves /openapi.json so the mothership can build its docs/api-reference/<tag>.json VFS views from the deployed spec - generated tool catalog/schemas regenerated from the mothership contract Companion: simstudioai/mothership feat/enhance-search-agent Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tool chips
read("docs/documentation/workflows/index.mdx") now renders "Read
Workflows/index" instead of the leaf-only fallback ("Read Index").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"Searched docs" becomes 'Searched docs for "<query>"' (toolTitle/title preferred, query fallback, truncated at 60 chars). Also adds the missing browser_list_sessions display title the catalog regen surfaced. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…search_docs Replaces the mothership's runtime docs corpus (llms.txt + llms-full.txt + openapi.json behind a 15m TTL cache) with a static manifest generated from the docs source, plus live per-page fetches. ~1,000 fewer lines of hand- written code and one repo instead of two. - scripts/sync-docs-manifest.ts walks apps/docs/content/docs/en and emits lib/copilot/generated/docs-manifest.ts. Each entry is simultaneously the docs/ VFS path and the docs.sim.ai URL path, so a read is a plain fetch. Section index pages fold onto their parent (fumadocs serves /workflows, not /workflows/index); academy/ and api-reference/ are excluded — they stay unmounted and unsearchable, reachable only via scrape_page. - docs-manifest:generate / :check, with a CI step so a page added, renamed, or deleted without regenerating fails the build. Content edits don't. - lib/copilot/docs/docs-corpus.ts + tools/handlers/vfs.ts: glob matches the manifest with no network, read fetches the page live, grep takes exactly ONE page (each is a fetch, so there is no corpus-wide grep). Opt-in like uploads/ — only an explicit docs/ prefix ever matches. - search_docs now scopes to the docs/ tree instead of docs/documentation/, validates its path against the manifest (a bad path errors instead of silently returning nothing), and returns the docs/ path with every chunk so search chains into read. Unscoped searches drop rows the agent could not then read: unmounted sections, and pages gone since the last index rebuild. - @docs tagging disabled: its query was the raw user message, a poor embedding query, and the mention UI it fed was already dead code. - Reverts the apps/docs /openapi.json route, added only for the old api-reference VFS views. Companion: simstudioai/mothership feat/enhance-search-agent Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Multi-agent review of the docs/ VFS change. Applied the behavior-preserving fixes plus two agent-facing bugs that made real pages unreadable. - docs read no longer hard-fails on oversized pages. Six+ live integration references exceed the inline cap (github.mdx is 354KB, sportmonks 513KB), so a plain read of them ALWAYS failed and cost a second fetch to recover. Truncate to the largest whole-line prefix that fits, keep the true totalLines, and tell the model how to page. An explicit offset/limit that still overflows is still an error — that one is a caller mistake. - classify docs fetch failures. Everything collapsed to null, so a permanent 404 was reported to the agent as "temporarily unavailable, retry shortly", inviting a retry loop on a page that will never exist. 4xx (except 429) is now permanent and says so; 5xx/429/network/timeout keep the retry wording. - register search_documentation as a transitional alias for search_docs. sim and mothership deploy independently and the rename deleted the old id on both sides, so BOTH deploy orders broke docs lookup for the window between them. Old params are a subset of the new. Remove once both ship. - extract the index-page fold (X/index.mdx <-> X.mdx) into docs-path.ts. It was re-derived in three places — the manifest generator, the source_document reverse mapping, and the search scope filter — which is the hand-synced-duplicate shape that has drifted in this repo before. - grepDocsPage now goes through grepReadResult, the primitive files/ and uploads/ grep already use, instead of calling grep directly. - couldMatchDocsScope delegates to isDocsPath; the bodies were identical. - drop the dead 'docs' member from AgentContextType. Tests: 404-vs-5xx-vs-429 classification, network failure, and a docs-path round-trip asserting one source candidate reproduces every manifest entry. Verified: tsc clean, 932 copilot tests, biome clean, docs-manifest:check, check:utils and check:api-validation:strict both pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The SQL LIMIT is applied before the similarity-threshold and liveness filters, so search_docs can return fewer hits than topK — or none, when every candidate was filtered. An empty array is indistinguishable from "the documentation does not cover this", which sends the agent off to guess instead of rephrasing or falling back to glob. searchDocs now returns the drop counts alongside the results, and the tool attaches a note when anything was dropped: how many candidates the index returned, why they went, and what to try next. Silent on the common path. This does not change which rows are returned or how many — the ordering issue behind the shortfall is a pre-existing bug the deleted search-documentation.ts had too, and pushing the threshold into SQL is its own change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…g search A directory scope matched only `<section>/%`, which covers an overview stored as `<section>/index.mdx` but not one stored as a sibling `<section>.mdx`. Fumadocs accepts both layouts and page scope already handles both via docsSourceCandidates, so a scoped section search could silently omit the overview chunks — and the doc comment claimed it did not. Every section in the tree currently uses the index.mdx layout, so nothing is broken today; this closes the gap before someone adds a sibling overview and gets quietly incomplete results.
The clamp guarded magnitude but not type: Math.min/Math.max propagate NaN, so a non-numeric topK reached the query as `.limit(NaN)`. The `?? DEFAULT` only caught undefined. Nothing enforced this but the generated Ajv schema, and searchDocs is also called directly, so it should not depend on that. Extract clampTopK, which falls back to the default for anything non-finite (NaN, Infinity, a string that slipped through) and clamps the rest to [1, 25]. The clamp was completely untested because the db mock's .limit() stub discarded its argument — the mock now records it. Covers default, cap, floor, truncation, and the non-finite fallback. Worth pinning: staging's search_documentation documented "max 10" and enforced nothing, so this bound is new behavior, not just a bigger number.
Chips read "Searched docs" with no indication of what was searched. The query-aware title existed earlier on this branch and this commit's own predecessor dropped it: removing search_docs from the catalog deleted the display case and its test, and putting the tool back only restored the static map entry. The generic "every visible catalog tool has a title" assertion still passed, because it checks that a title exists, not that it is the useful one. Chips now read: Searching docs for "how to read workflow logs and view executions" -> Searched docs for "...". The gerund flip already preserves the suffix, so the completed state needs no extra handling — the test now pins that too, since it was the part most likely to regress silently.
…h default Two places decide what the docs/ corpus is: the manifest generator (what is readable) and the vector search's unscoped filter (what is findable). They each carried their own copy of the excluded-section list. If they drift, a hit in a section that is indexed but not mounted comes back as a chunk the agent cannot then read — dropped as stale, silently shrinking the result set. UNMOUNTED_DOCS_SECTIONS is now the one list both import. search_docs returns 5 chunks by default instead of 10; raise topK when a pass genuinely comes back thin. A truncated docs page now routes to one more fetch instead of two. grep and read cost the same single uncached fetch of the page, so grep is an alternative to a read here, never a step after one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…erence tool Picks up get_platform_actions' hidden/retired description from mothership. The id stays in the catalog so isKnownTool keeps routing calls from an older build during a mixed deploy; the handler is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two TSDoc blocks sat back to back above DocsSearchOutcome; the first describes DocsSearchScopeError, which had no doc comment of its own. Moved it to the class. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ex drops, oversized-line reads
Review findings applied from the multi-agent pass on this branch:
- glob("docs/") matched no key and silently returned empty; normalize now
strips trailing slashes so it resolves like "docs"
- unscoped search_docs no longer returns root-homepage chunks that would
only be counted against topK and then dropped as stale (the manifest
deliberately omits index.mdx)
- a docs page whose single line exceeds the inline cap now fails with grep
guidance instead of returning an over-cap payload as success
- test coverage for the vfs docs routing (glob/read/grep dispatch,
DocsCorpusError surfacing, truncation paths), the search_docs server
tool's shortfall notes, the empty-embedding outcome, and the inert
@docs context
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The search subagent's task description now tells callers to pass a fully self-contained task — it no longer inherits the conversation (see the companion mothership change). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tions outright, no shims The transitional apparatus is gone: no search_documentation registry alias, no get_platform_actions handler, and the ids are out of the regenerated catalog/schemas. During the deploy window an old Mothership build calling either id gets the recoverable tool-not-found result. The two ids stay in HIDDEN_TOOL_NAMES forever — like load_agent_skill, historical persisted chats contain their tool calls and must replay without rendering chips for retired tools. The alias test is replaced by a dispatch test pinning search_docs's own catalog -> route -> handler chain and the retired ids' gone-but-chip-hidden state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e case The static TOOL_TITLES entry is unreachable for search_docs — the dynamic switch case returns first so it can include the query — so the rename only takes effect there. Tests updated to the new wording. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… parallel Two robustness upgrades to the docs corpus. Page fetches from docs.sim.ai now retry transient failures (5xx, 429, network, timeout) with jittered backoff over three 3s attempts instead of a single 10s attempt, so a momentary stall recovers in seconds instead of failing the tool call. And grep now accepts a docs directory path: it fans out to every manifest page under the directory with bounded concurrency and runs one multi-file grep, replacing the single-page restriction that forced agents into per-page call sweeps. Pages the site no longer serves are skipped; an unreachable page fails the whole grep so a partial result is never mistaken for "not documented". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tool was retired outright with the search_docs replacement; its test outlived the module on staging and no longer resolves. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The vfs handler test still pinned the retired single-page restriction; directory grep now succeeds with a parallel page fan-out, and an invalid path (neither page nor directory) is the remaining rejection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tracts The rebase resolutions carried arc-era generated output missing staging's browser and terminal tools; resync from the regenerated mothership contract so the mirror matches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Staging added docs pages since the manifest was generated; the CI freshness check (docs-manifest:check) catches exactly this drift. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…racts Mirrors the schema fix documenting the docs corpus grep mode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A directory-scoped docs grep now records copilot.vfs.grep.docs_page_count (pages fetched from the live site) on the active tool span, mirroring the new contract attribute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryMedium Risk Overview
CI adds Reviewed by Cursor Bugbot for commit dac3284. Bugbot is set up for automated code reviews on this repo. Configure here. |
Greptile SummaryThe PR mounts public Sim documentation as a lazy, read-only docs/ VFS namespace and replaces the legacy documentation tools with path-scoped semantic search.
Confidence Score: 3/5The PR should not merge until search_docs restores resolved-secret projection before logging, embedding, result, and display boundaries. A secret-bearing documentation query can now pass unchanged to an external embedding provider and additional observable surfaces because the replacement handler ignores the secret registry that its dispatch context supplies. Files Needing Attention: apps/sim/lib/copilot/tools/server/docs/search-docs.ts, apps/sim/lib/copilot/docs/docs-search.ts, apps/sim/lib/copilot/tools/tool-display.ts, scripts/sync-docs-manifest.ts
|
| Filename | Overview |
|---|---|
| apps/sim/lib/copilot/tools/server/docs/search-docs.ts | Registers the replacement semantic-search tool, but drops the prior resolved-secret projection before model and observability boundaries. |
| apps/sim/lib/copilot/docs/docs-search.ts | Implements path-scoped vector search and explicit filtering diagnostics; its raw-query logging and embedding call participate in the secret-disclosure path. |
| apps/sim/lib/copilot/docs/docs-corpus.ts | Implements manifest-backed globbing and resilient live read/grep behavior with bounded concurrency and explicit partial-failure handling. |
| apps/sim/lib/copilot/tools/handlers/vfs.ts | Integrates the docs corpus into existing VFS operations with line paging, output caps, and user-facing corpus errors. |
| scripts/sync-docs-manifest.ts | Generates and checks the folded public docs manifest, but uses relative imports contrary to the repository import convention. |
| apps/sim/lib/copilot/generated/tool-catalog-v1.ts | Updates the shared generated catalog to register search_docs and retire the two legacy documentation tools. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Agent[Copilot agent] --> Search[search_docs]
Search --> Scope[Validate optional docs path]
Scope --> Embed[Generate query embedding]
Embed --> DB[(docs_embeddings)]
DB --> Filter[Threshold and manifest filtering]
Filter --> Results[Paths, snippets, and shortfall outcome]
Agent --> Glob[glob docs/**]
Agent --> Read[read docs/page.mdx]
Agent --> Grep[grep docs/page or directory]
Glob --> Manifest[Generated docs manifest]
Read --> Site[docs.sim.ai markdown]
Grep --> Site
Reviews (1): Last reviewed commit: "improvement(copilot): stamp docs grep fa..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| async execute(params: SearchDocsParams): Promise<SearchDocsOutput> { | ||
| const outcome = await searchDocs(params.query, { path: params.path, topK: params.topK }) |
There was a problem hiding this comment.
When a search_docs query contains a resolved secret, this handler ignores the available secret registry and forwards the plaintext query unchanged, causing it to be logged, sent to the external embedding provider, echoed in the tool result, and rendered in the tool title.
How this was verified: The execution path passes the raw query to searchDocs without establishing either supported projection context before logging or embedding.
Knowledge Base Used: Copilot module
| */ | ||
| import { readdir, readFile, writeFile } from 'node:fs/promises' |
There was a problem hiding this comment.
The new generator imports helpers through ../apps/... and ./... paths instead of the repository-required path aliases, making these imports path-sensitive and inconsistent with the established module-resolution convention.
Rule Used: Use established path alias patterns instead of dee... (source)
Learned From
simstudioai/sim#233
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit dac3284. Configure here.
| totalResults: outcome.results.length, | ||
| ...(note ? { note } : {}), | ||
| } | ||
| }, |
There was a problem hiding this comment.
Docs search skips secret projection
High Severity
search_docs embeds the raw query and never calls projectServerToolModelInput, even though the adapter still supplies resolvedSecretTraceRegistry. The retired search_documentation tool projected the query before generateSearchEmbedding, and sibling tools like knowledge search still do. Plaintext secrets in the query can therefore leave Sim toward the embedding provider.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit dac3284. Configure here.
| const topK = clampTopK(options?.topK) | ||
| const where = scopeCondition(options?.path) | ||
|
|
||
| logger.info('Executing docs search', { query, topK, path: options?.path ?? null }) |
There was a problem hiding this comment.
Docs search logs full query
High Severity
searchDocs logs the full query string at info level. The previous docs search logged only queryLength, and its regression test asserted plaintext never appeared in logger calls. Any secret-bearing query is now persisted in application logs.
Reviewed by Cursor Bugbot for commit dac3284. Configure here.
| ) | ||
| } | ||
| const dir = `${key}/` | ||
| const pages = [...docsKeyView.keys()].filter((pageKey) => pageKey.startsWith(dir)) |
There was a problem hiding this comment.
Directory grep omits section overview
Medium Severity
Directory grepDocs only fans out to keys under `${key}/`, so a scope like docs/workflows never fetches docs/workflows.mdx. search_docs directory scoping intentionally includes that sibling overview because fumadocs folds section index pages there. Grep and search therefore disagree on what a section path covers, and overview-only content is invisible to directory greps.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit dac3284. Configure here.
1 participant
What
Mounts Sim's public documentation as a read-only
docs/tree in the copilot VFS and replacessearch_documentationwith a path-scoped, semantically-searchablesearch_docs— the sim half of a paired change (mothership: simstudioai/mothership#416, to be merged together).docs/VFS corpus: one.mdxper page mirroring docs.sim.ai URLs, built from a generated manifest (docs-manifest:generate/:checkin CI). Opt-in for glob/grep — an unscoped**never drags 400 doc pages into results. Reads fetch live page markdown from the docs site.search_docs: vector search overdocs_embeddingswith optional page/sectionpathscoping, topK clamping, and an explicit outcome contract (candidatesConsidered/droppedBelowThreshold/droppedStale) so an empty result is never mistaken for "not documented".search_documentationandget_platform_actionsoutright (the quick reference now lives in the docs corpus, always current instead of hand-maintained).copilot.vfs.grep.docs_page_count) so dashboards can watch corpus-grep load.Deploy note
Sim and mothership ship independently; during the window between the two deploys, an old mothership build can still offer
search_documentation, whose calls will fail against a sim build that has retired it. This mirrors the deliberate "retire outright, no shims" decision — merge both halves together and deploy normally.Testing
113 tests across the docs corpus, search dispatch, VFS handlers, and chat-content suites; manifest freshness gated in CI; validated end-to-end against a locally rebuilt index (402 pages) including live directory-grep fan-out (~3s for the full corpus, CDN-cached).
🤖 Generated with Claude Code