Search

[perber]

Implement full-text search across all wiki pages.

[perber]

Concept: Full-Text Search for Leafwiki:

I'd like to implement full-text search as an independent module within LeafWiki. The goal is to provide searchable content indexing that works separately from the existing pagestore logic.

---

Technical Basis

• SQLite with FTS5 as the backend
  • Lightweight and embedded
  • No external services required
  • Fully ARM-compatible (e.g., works on Raspberry Pi)
  • Stores index in a local .sqlite file

1. search Module

• Own Go package responsible for:
  • Managing the FTS5-enabled SQLite database
  • Indexing Markdown page contents
  • Handling update/insertion on file changes

2. Initial Indexing Process

• Scans all Markdown files under the data content directory
• Populates the SQLite FTS table
• Runs at startup
• Indexing Status exposed via HTTP API

3. File Change Monitoring

• Detects changes to .md files via:
  • fsnotify
  • Or periodic directory scan with stat/hash comparison

4. HTTP Endpoints

• GET api/search/status
  Returns info such as:

  {
      "initial_indexing": false,
      "indexed_files": 124,
      "last_updated": "2025-06-07T12:00:00Z"
  }

• GET /search?q=term
  Returns a list of matches including:
  • title
  • path
  • pageID
  • snippet / content preview

---

Features & Goals

• Decoupled from core logic – keeps things modular and testable
• Works on all platforms LeafWiki supports (including ARM)
• Lightweight and embedded – no external search servers
• Extensible – allows for future improvements (e.g. tag-based filtering, modified-date sort, etc.)

---

Planned Steps

• Create search module
• Initialize SQLite with FTS5 schema
• Implement initial full scan/index
• Integrate file change detection
• The title should be ranked as highest
• Expose /search and /search/status endpoints

Feedback welcome before I get started!