Skip to content

feat(orama): local only database#8836

Merged
avivkeller merged 11 commits intomainfrom
orama-local
Apr 26, 2026
Merged

feat(orama): local only database#8836
avivkeller merged 11 commits intomainfrom
orama-local

Conversation

@avivkeller
Copy link
Copy Markdown
Member

cc @nodejs/web-infra

This PR changes our Orama implementation to use a locally-merged version of our API docs Orama and Learn Orama.

Copilot AI review requested due to automatic review settings April 22, 2026 18:31
@avivkeller avivkeller requested review from a team as code owners April 22, 2026 18:31
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 22, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
nodejs-org Ready Ready Preview Apr 25, 2026 8:48pm

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented Apr 22, 2026

PR Summary

Medium Risk
Search is reworked to load and query client-side Orama databases fetched at runtime, removing Orama Cloud sync/chat paths; this can impact search results, performance, and availability if index fetches fail.

Overview
Replaces the website’s Orama Cloud-based search+chat integration with a local-only search that lazily fetches prebuilt Orama DB JSON indexes (Learn + API docs), prefixes/merges their hrefs, and queries them via @orama/orama (WithSearch).

Removes the Orama Cloud sync workflow/script/tests and deletes the site-level Searchbox/chat components; the navbar now mounts the new WithSearch wrapper.

Updates @node-core/ui-components search to a simplified SearchBox (no chat, suggestions, or mode-specific hit behavior), inlines default Orama query params, bumps the package version, and adjusts dependencies/config/docs accordingly.

Reviewed by Cursor Bugbot for commit 23e4c7b. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
Copy link
Copy Markdown
Contributor

👋 Codeowner Review Request

The following codeowners have been identified for the changed files:

Team reviewers: @nodejs/web-infra @nodejs/nodejs-website

Please review the changes when you have a chance. Thank you! 🙏

Comment thread apps/site/scripts/orama/index.mjs Outdated
Comment thread apps/site/scripts/orama/index.mjs Outdated
Comment thread packages/ui-components/src/hooks/useOrama.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the website search integration away from Orama Cloud toward a locally generated (and locally loaded) Orama database that merges the Learn + API docs indexes.

Changes:

  • Generate a merged orama-db.json during site prebuild and load it client-side for search.
  • Replace the site’s Orama Cloud searchbox + chat integration with the UI-components SearchBox wired to the local DB.
  • Remove Orama Cloud configuration, sync workflow, and related scripts/tests.

Reviewed changes

Copilot reviewed 44 out of 46 changed files in this pull request and generated 13 comments.

Show a summary per file
File Description
pnpm-lock.yaml Updates lockfile for Orama package changes across workspaces.
packages/ui-components/src/hooks/useOrama.ts Adds a hook to create a local Orama DB and lazy-load it from a JSON snapshot.
packages/ui-components/src/Common/Search/index.tsx Adds a new SearchBox component that uses the local Orama hook.
packages/ui-components/src/Common/Search/index.module.css Styles for the new SearchBox layout/footer.
packages/ui-components/src/Common/Search/Suggestions/index.tsx Removes suggestions component (part of prior chat/suggestions UX).
packages/ui-components/src/Common/Search/Suggestions/index.module.css Removes styles for suggestions component.
packages/ui-components/src/Common/Search/Results/index.tsx Hardcodes search params (limit/threshold/boost) rather than receiving them from site.
packages/ui-components/src/Common/Search/Results/Hit/index.tsx Simplifies hit rendering by removing now-unused chat/search mode handling.
packages/ui-components/src/Common/Search/README.md Removes Orama search component README.
packages/ui-components/src/Common/Search/Chat/Trigger/index.tsx Removes chat trigger UI.
packages/ui-components/src/Common/Search/Chat/Trigger/index.module.css Removes chat trigger styles.
packages/ui-components/src/Common/Search/Chat/Input/index.tsx Removes chat input UI.
packages/ui-components/src/Common/Search/Chat/Input/index.module.css Removes chat input styles.
packages/ui-components/src/Common/Search/Chat/Actions/index.tsx Removes chat actions UI.
packages/ui-components/src/Common/Search/Chat/Actions/index.module.css Removes chat actions styles.
packages/ui-components/package.json Swaps dependencies to use @orama/orama and moves @orama/core.
apps/site/turbo.json Removes Orama Cloud env vars from turbo pipeline config.
apps/site/scripts/orama/index.mjs New script that fetches two remote Orama DBs, merges them, and writes public/orama-db.json.
apps/site/scripts/orama/constants.mjs Adds constants for remote Orama DB URLs.
apps/site/scripts/orama-search/sync-orama-cloud.mjs Removes Orama Cloud sync script.
apps/site/scripts/orama-search/process-documents.mjs Removes old document processing for cloud sync.
apps/site/scripts/orama-search/get-documents.mjs Removes old document fetching logic for cloud sync.
apps/site/scripts/orama-search/tests/process-documents.test.mjs Removes tests for deleted processing logic.
apps/site/scripts/orama-search/tests/get-documents.test.mjs Removes tests for deleted fetching logic.
apps/site/package.json Runs Orama DB generation in prebuild/dev/deploy and removes cloud sync script entry.
apps/site/next.constants.mjs Removes Orama Cloud constants and default query/suggestions config.
apps/site/components/withSearch.tsx New wrapper component that mounts the new SearchBox.
apps/site/components/withNavBar.tsx Re-enables search in the navbar via the new wrapper.
apps/site/components/Common/Searchbox/orama-client.ts Removes Orama Cloud client initialization.
apps/site/components/Common/Searchbox/index.tsx Removes old site-specific searchbox (chat + suggestions + i18n wiring).
apps/site/components/Common/Searchbox/index.module.css Removes old searchbox styles.
apps/site/components/Common/Searchbox/SlidingChatPanel/index.tsx Removes sliding chat panel.
apps/site/components/Common/Searchbox/SlidingChatPanel/index.module.css Removes sliding chat panel styles.
apps/site/components/Common/Searchbox/SearchItem/utils.ts Removes old hit formatting + locale/base-path link building utilities.
apps/site/components/Common/Searchbox/SearchItem/index.tsx Removes old adapter from Orama hit docs to UI-components hit rendering.
apps/site/components/Common/Searchbox/Footer/index.tsx Removes old footer (powered-by + shortcuts).
apps/site/components/Common/Searchbox/Footer/index.module.css Removes old footer styles.
apps/site/components/Common/Searchbox/DocumentLink/index.tsx Removes locale-aware next/link wrapper for search/chat sources.
apps/site/components/Common/Searchbox/ChatSources/index.tsx Removes chat sources UI.
apps/site/components/Common/Searchbox/ChatSources/index.module.css Removes chat sources styles.
apps/site/components/Common/Searchbox/ChatMessage/index.tsx Removes chat message renderer.
apps/site/components/Common/Searchbox/ChatMessage/index.module.css Removes chat message styles.
apps/site/components/Common/Searchbox/ChatInteractions/index.tsx Removes chat interactions container.
apps/site/components/Common/Searchbox/ChatInteractions/index.module.css Removes chat interactions styles.
.gitignore Ignores generated apps/site/public/orama-db.json.
.github/workflows/sync-orama.yml Removes workflow that synced Orama Cloud.
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/site/package.json
Comment thread apps/site/scripts/orama/index.mjs Outdated
Comment thread packages/ui-components/src/hooks/useOrama.ts Outdated
Comment thread packages/ui-components/src/Common/Search/Results/index.tsx
Comment thread apps/site/scripts/orama/index.mjs Outdated
Comment thread packages/ui-components/src/hooks/useOrama.ts Outdated
Comment thread packages/ui-components/src/hooks/useOrama.ts Outdated
Comment thread packages/ui-components/src/hooks/useOrama.ts Outdated
Comment thread packages/ui-components/package.json
Comment thread apps/site/package.json Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.18%. Comparing base (524e64b) to head (23e4c7b).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8836      +/-   ##
==========================================
- Coverage   73.88%   73.18%   -0.71%     
==========================================
  Files         105      102       -3     
  Lines        8889     8623     -266     
  Branches      326      314      -12     
==========================================
- Hits         6568     6311     -257     
+ Misses       2320     2311       -9     
  Partials        1        1              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@avivkeller avivkeller marked this pull request as ready for review April 22, 2026 21:34
@avivkeller
Copy link
Copy Markdown
Member Author

@dario-piotrowicz

[WebServer] ✘ [ERROR] Asset too large.
[WebServer] 
[WebServer]   Cloudflare Workers supports assets with sizes of up to 25 MiB. We found a file /home/runner/work/nodejs.org/nodejs.org/apps/site/.open-next/assets/orama-db.json with a size of 28.9 MiB.
[WebServer]   Ensure all assets in your assets directory "/home/runner/work/nodejs.org/nodejs.org/apps/site/.open-next/assets" conform with the Workers maximum size requirement.
[WebServer] 
[WebServer] 
[WebServer] If you think this is a bug then please create an issue at https://github.com/cloudflare/workers-sdk/issues/new/choose
[WebServer] Note that there is a newer version of Wrangler available (4.84.1). Consider checking whether upgrading resolves this error.
[WebServer] ? Would you like to report this error to Cloudflare? Wrangler's output and the error details will be shared with the Wrangler team to help us diagnose and fix the issue.
[WebServer] 🤖 Using fallback value in non-interactive context: no
[WebServer] 🪵  Logs were written to "/home/runner/.config/.wrangler/logs/wrangler-2026-04-22_21-36-59_766.log"

@avivkeller
Copy link
Copy Markdown
Member Author

Perhaps I should GZip the file?

@MattIPv4
Copy link
Copy Markdown
Member

MattIPv4 commented Apr 22, 2026

I imagine you're going to want to chunk that file, GZiping would only be a temporary fix and would bite us again as the site grows.

@ovflowd
Copy link
Copy Markdown
Member

ovflowd commented Apr 23, 2026

Did a functional pass.
Question: does anyone else wish we had this documented within https://github.com/nodejs/nodejs.org/blob/main/docs/technologies.md ? I was surprised to see no mention of Orama in there.

Wait, really? Interesting. We should probably document it now, can also be done in a follow-up PR if needed, @avivkeller!

Comment thread packages/ui-components/src/Common/Search/index.module.css Outdated
Comment thread packages/ui-components/src/Common/Search/index.module.css Outdated
Comment thread packages/ui-components/src/Common/Search/index.module.css Outdated
@avivkeller avivkeller linked an issue Apr 24, 2026 that may be closed by this pull request
Signed-off-by: Aviv Keller <me@aviv.sh>
Comment thread apps/site/components/withSearch.tsx
@avivkeller avivkeller requested a review from canerakdas April 24, 2026 20:20
Comment thread packages/ui-components/src/Common/Search/index.tsx Outdated
Comment thread packages/ui-components/src/hooks/useOrama.ts Outdated
Comment thread packages/ui-components/package.json
@araujogui
Copy link
Copy Markdown
Member

@avivkeller Not sure if you’re aware, but API docs results are redirecting to 404 pages

Copy link
Copy Markdown
Member

@canerakdas canerakdas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@avivkeller
Copy link
Copy Markdown
Member Author

@araujogui Yes, that's because they are on the preview URL, on the main site, that will not happen

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 67a346f. Configure here.

Comment thread apps/site/components/withSearch.tsx
@araujogui
Copy link
Copy Markdown
Member

araujogui commented Apr 25, 2026

@araujogui Yes, that's because they are on the preview URL, on the main site, that will not happen

Understood! but it would be nice if this works on preview urls but I don't know how much complicated would be.

@avivkeller
Copy link
Copy Markdown
Member Author

@araujogui Better?

@avivkeller avivkeller added this pull request to the merge queue Apr 26, 2026
Merged via the queue into main with commit 60aeb4b Apr 26, 2026
15 checks passed
@avivkeller avivkeller deleted the orama-local branch April 26, 2026 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The Orama logo is returning a 404

8 participants