feat: migrate website search and embeddings to crawler service#595

Merged

larryro merged 9 commits into

mainfrom

feat/migrate-search-to-crawler

Feb 27, 2026

Commits on Feb 27, 2026

feat: migrate website search and embeddings from Convex to crawler service
larryro
committed
feat: add search UI, chunk inspection, and async website registration
larryro
committed
feat(crawler): replace custom chunker with semantic-text-splitter MarkdownSplitter
larryro
committed
refactor(db): rename search database from tale_crawler_search to tale_search
larryro
committed
fix(crawler): harden embedding config, error handling, and website sync
larryro
committed
fix(crawler): pin embedding column dimensions at startup for HNSW index reliability
larryro
committed
fix: resolve CI lint, typecheck, and knip failures
larryro
committed
fix: format services/platform/package.json
larryro
committed
fix(crawler): use paradedb.match() for FTS to handle special characters in queries
larryro
committed