Improve Documentation Search System

[aymericzip]

Description:
Currently, our doc search combines two approaches:

1. Local fuzzy search (Fuse.js):

   • Uses metadata only (title, description, keywords) with weighted fields.
   • Very fast but order of words matters.
   • Does not search in full doc content (to avoid loading everything client-side).

const fuseOptions: IFuseOptions<DocMetadata> = {
  keys: [
    { name: 'title', weight: 0.8 },
    { name: 'description', weight: 0.1 },
    { name: 'keywords', weight: 0.1 },
  ],
  threshold: 0.02, // Defines how fuzzy the matching should be (lower is more strict)
};

2. Backend embeddings search:

   • Runs in parallel.
   • Adds results to the top list when response arrives.

Why:

• Search is critical for discovering docs, dictionaries, etc.
• Needs iterations and refinements to balance speed, accuracy, and usability.
• Goal: avoid dependency on Algolia or similar external services.

How contributors can help:

• Experiment with better Fuse.js configs (threshold, tokenization, word order).
• Propose efficient ways to include more content context without loading full docs client-side.
• Optimize embeddings search and result merging strategy.
• Benchmark performance and relevance.

Relevant files:

• apps/website/src/components/DocPage/Search/SearchView.tsx
• apps/backend/src/controllers/search.controller.ts

[dfordp]

Gonna try working on this