Skip to content

v1.1.0 - Semantic Search

Choose a tag to compare

View all tags
@PoisonedYouth PoisonedYouth released this 15 Nov 12:20
· 10 commits to main since this release
v1.1.0
7d5373e

What's New

Semantic Search with Ollama

This release adds powerful semantic search capabilities to ClaudeCodeHistory, enabling you to search conversations by meaning and concept, not just keywords.

Key Features

  • 🔍 Vector Embeddings

    • Generate embeddings for conversations using Ollama (nomic-embed-text model)
    • 768-dimensional embeddings for accurate semantic matching
    • Automatic normalization and storage in SQLite

  • ⚡ Three Search Modes

    • KEYWORD: Traditional FTS5 full-text search (fast, exact matching)
    • SEMANTIC: Vector similarity search (finds similar concepts)
    • HYBRID: Combines both using Reciprocal Rank Fusion (default, best results)

  • 🎯 Find Similar Conversations

    • Discover related conversations based on semantic similarity
    • Cosine similarity for vector comparison
    • Automatic fallback to keyword search if Ollama is unavailable

  • ⚙️ Embedding Management UI

    • New Settings screen section for Ollama integration
    • Real-time Ollama status indicator
    • Batch embedding generation with progress tracking
    • "Generate Missing Embeddings" button

Search Algorithm

Reciprocal Rank Fusion (RRF):

  • Combines FTS5 keyword search and vector similarity search
  • Formula: Score(doc) = 1/(k + rank_keyword) + 1/(k + rank_vector)
  • Constant k=60 for optimal result combination
  • Provides both precision of keyword matching and flexibility of semantic understanding

Technical Details

New Components:

  • VectorUtils.kt - Vector operations (normalization, cosine similarity, byte conversion)
  • SearchMode.kt - Enum for search modes (KEYWORD, SEMANTIC, HYBRID)
  • OllamaClient.kt - HTTP client for Ollama API integration
  • EmbeddingService.kt - Embedding generation and management

Updated Components:

  • ConversationRepository.kt - Added vectorSearch() and hybridSearch() methods
  • SearchService.kt - Updated with mode parameter and findSimilar() method
  • MainScreen.kt - Wired up OllamaClient and EmbeddingService

Database Schema:

  • New embeddings table with conversation_id, embedding (BLOB), and model fields
  • Each embedding: ~3KB (768 floats)
  • Indexed for fast joins with conversations table

Performance:

  • Keyword search: ~1-5ms (FTS5 only)
  • Semantic search: ~50-200ms (depending on conversation count)
  • Hybrid search: ~100-300ms (combined)
  • Embedding generation: ~20ms per conversation

Documentation

See SEMANTIC_SEARCH.md for:

  • Installation instructions for Ollama
  • How it works (embeddings, similarity, RRF)
  • API reference for all new components
  • Performance benchmarks
  • Troubleshooting guide
  • Future UI enhancements

Getting Started

  1. Install Ollama:

    curl -fsSL https://ollama.com/install.sh | sh

  2. Pull the embedding model:

    ollama pull nomic-embed-text

  3. Launch the application and go to Settings

  4. Click "Generate Missing Embeddings" to create embeddings for your conversations

  5. Start searching with semantic understanding!

Installation

Download the appropriate installer for your platform from the assets below.

Upgrading from v1.0.0

Simply install the new version. Your existing database and conversations will work seamlessly. Visit the Settings screen to generate embeddings for historical conversations.

Other Additions

  • TODO.md with 80+ future feature ideas organized by category
  • Highlighted semantic search as a quick win feature

Full Changelog: v1.0.0...v1.1.0

Assets 5
Loading