Add static search to isolated documentation builds#3709
Draft
reakaleek wants to merge 8 commits into
Draft
Conversation
Mpdreamz
added a commit
that referenced
this pull request
Jul 24, 2026
Replace PR #3709's native pagefind binary subprocess approach with the Pagefind.Net and Pagefind.Net.Frontend NuGet packages — pure managed .NET, no native binary, AOT-compatible. The pagefind indexer is now a proper IMarkdownExporter that eagerly indexes each page during ExportAsync (O(1) memory) and flushes the index to disk in FinishExportAsync. It is included in both ExportOptions.Default and ExportOptions.Validation, so it works in isolated builds and docs-builder serve mode automatically. Key changes: - New PagefindMarkdownExporter in Elastic.Markdown/Exporters/Pagefind - New Exporter.Pagefind enum value, registered in both ExporterParsers - Removed native binary pipeline (package-pagefind.mjs, embedded resource, pagefind npm dep, PagefindSearchIndexer subprocess) - Removed static-search feature flag — search is always available for isolated builds and serve mode - Serve mode reads pagefind files from the background build's in-memory filesystem via a route handler - Added .pagefind-net-frontend-version to AllowedHiddenFileNames Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Mpdreamz
added a commit
that referenced
this pull request
Jul 24, 2026
Replace PR #3709's native pagefind binary subprocess approach with the Pagefind.Net and Pagefind.Net.Frontend NuGet packages — pure managed .NET, no native binary, AOT-compatible. The pagefind indexer is now a proper IMarkdownExporter that eagerly indexes each page during ExportAsync (O(1) memory) and flushes the index to disk in FinishExportAsync. It is included in both ExportOptions.Default and ExportOptions.Validation, so it works in isolated builds and docs-builder serve mode automatically. Key changes: - New PagefindMarkdownExporter in Elastic.Markdown/Exporters/Pagefind - New Exporter.Pagefind enum value, registered in both ExporterParsers - Removed native binary pipeline (package-pagefind.mjs, embedded resource, pagefind npm dep, PagefindSearchIndexer subprocess) - Removed static-search feature flag — search is always available for isolated builds and serve mode - Serve mode reads pagefind files from the background build's in-memory filesystem via a route handler - Added .pagefind-net-frontend-version to AllowedHiddenFileNames Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Mpdreamz
added a commit
that referenced
this pull request
Jul 24, 2026
* Emit Mermaid diagrams as external SVG files loaded via img (#3713) Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Fix CopyBrandingResources failure during serve in-memory build (#3718) Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> * Fix overlapping scope roots for nested Codex config paths (#3719) docs-builder 1.29.0 added extension roots from FindGitRoot for Codex commands. When config.yml lives under environments/internal/, the resolved root is nested inside WorkingDirectoryRoot and ScopedFileSystem rejects the overlapping roots. Normalize extension roots before constructing the scoped filesystem so redundant nested paths are dropped. Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> * Simplify scope root normalization for nested extension roots (#3720) Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> * Replace pagefind native binary with Pagefind.Net managed library Replace PR #3709's native pagefind binary subprocess approach with the Pagefind.Net and Pagefind.Net.Frontend NuGet packages — pure managed .NET, no native binary, AOT-compatible. The pagefind indexer is now a proper IMarkdownExporter that eagerly indexes each page during ExportAsync (O(1) memory) and flushes the index to disk in FinishExportAsync. It is included in both ExportOptions.Default and ExportOptions.Validation, so it works in isolated builds and docs-builder serve mode automatically. Key changes: - New PagefindMarkdownExporter in Elastic.Markdown/Exporters/Pagefind - New Exporter.Pagefind enum value, registered in both ExporterParsers - Removed native binary pipeline (package-pagefind.mjs, embedded resource, pagefind npm dep, PagefindSearchIndexer subprocess) - Removed static-search feature flag — search is always available for isolated builds and serve mode - Serve mode reads pagefind files from the background build's in-memory filesystem via a route handler - Added .pagefind-net-frontend-version to AllowedHiddenFileNames Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Jan Calanog <jan.calanog@elastic.co>
Generate a Pagefind index at build time so standalone sites can provide search without requiring a backend service. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Only publish frontend resources from the static asset namespace so the Pagefind indexer remains internal to isolated builds. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Expose structured page breadcrumbs to Pagefind so isolated search results identify their documentation section without a redundant Docs prefix. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Collapse secondary labels and controls on narrow screens so standalone documentation headers remain usable without horizontal overflow. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Reduce the embedded indexer size and preserve useful page titles, breadcrumbs, and section deep links in static search results. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Keep search results focused on canonical pages and avoid changing the global heading DOM solely for section deep links. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Reuse the normalized platform target so packaging validation and executable selection cannot drift. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
* Emit Mermaid diagrams as external SVG files loaded via img (#3713) Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Fix CopyBrandingResources failure during serve in-memory build (#3718) Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> * Fix overlapping scope roots for nested Codex config paths (#3719) docs-builder 1.29.0 added extension roots from FindGitRoot for Codex commands. When config.yml lives under environments/internal/, the resolved root is nested inside WorkingDirectoryRoot and ScopedFileSystem rejects the overlapping roots. Normalize extension roots before constructing the scoped filesystem so redundant nested paths are dropped. Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> * Simplify scope root normalization for nested extension roots (#3720) Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> * Replace pagefind native binary with Pagefind.Net managed library Replace PR #3709's native pagefind binary subprocess approach with the Pagefind.Net and Pagefind.Net.Frontend NuGet packages — pure managed .NET, no native binary, AOT-compatible. The pagefind indexer is now a proper IMarkdownExporter that eagerly indexes each page during ExportAsync (O(1) memory) and flushes the index to disk in FinishExportAsync. It is included in both ExportOptions.Default and ExportOptions.Validation, so it works in isolated builds and docs-builder serve mode automatically. Key changes: - New PagefindMarkdownExporter in Elastic.Markdown/Exporters/Pagefind - New Exporter.Pagefind enum value, registered in both ExporterParsers - Removed native binary pipeline (package-pagefind.mjs, embedded resource, pagefind npm dep, PagefindSearchIndexer subprocess) - Removed static-search feature flag — search is always available for isolated builds and serve mode - Serve mode reads pagefind files from the background build's in-memory filesystem via a route handler - Added .pagefind-net-frontend-version to AllowedHiddenFileNames Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Jan Calanog <jan.calanog@elastic.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Standalone documentation sites currently have no search unless they depend on a backend service. They need a search option that continues to work when the generated frontend is hosted as static files on services such as S3 and CloudFront, or when previewing locally with
docs-builder serve.What
Adds Pagefind-powered client-side search to all isolated builds and
docs-builder servemode. The existing search modal queries the generated client-side index while omitting backend-only Ask AI features.This PR represents combined work:
How
The `PagefindMarkdownExporter` runs inline during the build pipeline alongside other exporters. For each markdown page it extracts plain text via `PlainTextExporter`, builds weighted segments (title, headings, body), and feeds a `PagefindRecord` to `PagefindIndex.AddRecord` which tokenizes and discards the content immediately. At the end, `WriteAsync` flushes the pre-built inverted index to disk and `PagefindFrontend.ExtractToAsync` writes the browser runtime (`pagefind.js` + WASM).
The exporter is gated on `BuildType.Isolated` — assembler and codex builds skip it entirely (they use the Elasticsearch-backed search API). In serve mode, the index resides in the in-memory filesystem and is served via a dedicated route handler.
Made with Cursor