Fix search indexing configuration (ENG-791)

[devin-ai-integration]

Fix search indexing configuration (ENG-791)

Changes

• Updated Nextra configuration to use contentDirBasePath: '/content' to match getPageMap('/content') in layout.tsx
• Simplified Nextra search configuration to avoid conflicts with Pagefind
• Enabled static HTML export in Next.js configuration to generate HTML files for all routes
• Enhanced Pagefind configuration to target the correct directories and improve content indexing:
  • Updated site path to target static export files in 'out' directory
  • Added --glob "**/*.html" to target all HTML files
  • Added --exclude-selectors "nav, footer, [data-nextra-hide]" to exclude navigation elements
  • Added --force-language "en" to ensure proper language detection

Problem

The docs site's search was only indexing a tiny fraction of content (3 pages, 11 words) despite having 109 MDX files available.

Approach

The key issue was that Next.js wasn't generating enough static HTML files for Pagefind to index. Our solution:

1. Enable static HTML export with output: 'export' in next.config.mjs to generate HTML files for all routes
2. Update Pagefind to target the 'out' directory where static exports are generated
3. Align contentDirBasePath with the actual content location used in layout.tsx
4. Enhance Pagefind configuration with additional parameters for better indexing

These changes should ensure Pagefind properly indexes all relevant content by targeting the static HTML files generated during the build process.

Testing

• Pushed changes to GitHub for CI verification
• The changes should be verified by checking the Pagefind output during build
• Vercel deployment at https://ai-git-devin-eng-791-1745914560.dev.driv.ly can be used to verify search functionality

Link to Devin run: https://app.devin.ai/sessions/938cae89a83f4f18a9f44c05229b6d64
User: Nathan Clevenger (nateclev@gmail.com)

[linear]

ENG-791 Fix Search Indexes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
ai	❌ Failed (Inspect)			Apr 29, 2025 2:22pm

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[github-actions]

📦 Next.js Bundle Analysis for do

This analysis was generated by the Next.js Bundle Analysis action. 🤖

This PR introduced no changes to the JavaScript bundle! 🙌

[devin-ai-integration]

Closing due to inactivity for more than 7 days.

[devin-ai-integration]

Devin is archived and cannot be woken up. Please unarchive Devin if you want to continue using it.