Skip to content

Search

Jump to bottom
Temp edited this page Oct 4, 2025 · 19 revisions

Search Guide

Complete guide to the triple search system in Memory Journal.

Triple Search System

Memory Journal provides three complementary search methods:

  1. Full-Text Search - SQLite FTS5 with highlighting
  2. Date Range Search - Time-based filtering with tags
  3. Semantic Search - ML-powered similarity (optional)

Full-Text Search

search_entries Tool

Fast keyword-based search with result highlighting.

Basic Usage:

search_entries({
  query: "performance optimization",
  limit: 10
})

With filtering:

search_entries({
  query: "lazy loading startup",
  limit: 5,
  is_personal: false
})

Output:

Found 3 entries:

#42 (technical_achievement) - 2025-10-04 16:45:30
Personal: False
Snippet: Implemented **lazy** **loading** for ML dependencies - **startup** improved...

#38 (development_note) - 2025-10-03 14:20:15
Personal: False
Snippet: Researching **lazy** initialization patterns for **performance**...

How It Works

SQLite FTS5:

  • Full-text index on entry content
  • Automatic tokenization
  • Stop word filtering
  • Rank-ordered results
  • Query term highlighting

Search Features:

  • Boolean operators: AND, OR, NOT
  • Phrase search: "exact phrase"
  • Wildcards: optim* matches optimization, optimized, etc.

Examples:

// Multiple terms (AND)
search_entries({ query: "database performance" })

// Exact phrase
search_entries({ query: '"lazy loading"' })

// Wildcard
search_entries({ query: "optim*" })

// Boolean
search_entries({ query: "performance AND (optimization OR tuning)" })

Best Practices

Good queries:

  • ✅ "performance optimization" - Specific
  • ✅ lazy loading startup - Multiple keywords
  • ✅ bug fix database - Related terms

Poor queries:

  • ❌ the - Stop word
  • ❌ a - Too common
  • ❌ Single letter searches

Date Range Search

search_by_date_range Tool

Filter entries by date with additional criteria.

Basic Usage:

search_by_date_range({
  start_date: "2025-10-01",
  end_date: "2025-10-31"
})

With filters:

search_by_date_range({
  start_date: "2025-10-01",
  end_date: "2025-10-31",
  tags: ["performance", "optimization"],
  entry_type: "technical_achievement",
  is_personal: false
})

Output:

📅 Found 5 entries between 2025-10-01 and 2025-10-31:

**Entry #42** (technical_achievement) - 2025-10-04 16:45:30
Implemented lazy loading for ML dependencies...

**Entry #38** (development_note) - 2025-10-03 14:20:15
Researching lazy initialization patterns...

Use Cases

Sprint reviews:

search_by_date_range({
  start_date: "2025-10-01",
  end_date: "2025-10-14",
  tags: ["sprint-15"]
})

Monthly summaries:

search_by_date_range({
  start_date: "2025-10-01",
  end_date: "2025-10-31",
  is_personal: false
})

Milestone tracking:

search_by_date_range({
  start_date: "2025-01-01",
  end_date: "2025-12-31",
  entry_type: "milestone"
})

Semantic Search

semantic_search Tool

Find conceptually similar entries using ML embeddings.

Requirements:

pip install sentence-transformers faiss-cpu

Basic Usage:

semantic_search({
  query: "improving application startup time",
  limit: 5
})

With threshold:

semantic_search({
  query: "performance optimization strategies",
  limit: 3,
  similarity_threshold: 0.5,
  is_personal: false
})

Output:

🔍 Semantic Search Results for: 'improving application startup time'
Found 3 semantically similar entries:

**Entry #42** (similarity: 0.687)
Type: technical_achievement | Personal: False
Content: Implemented lazy loading for ML dependencies...

**Entry #38** (similarity: 0.521)
Type: development_note | Personal: False
Content: Researching lazy initialization patterns...

How It Works

Vector Embeddings:

  • Model: all-MiniLM-L6-v2
  • Dimensions: 384
  • Storage: SQLite BLOB + FAISS index
  • Similarity: Cosine similarity

First Use:

  • Loads model (~100MB download)
  • Takes ~15 seconds first time
  • Subsequent searches: <1 second

Similarity Scores:

  • 1.0 = Identical
  • 0.7+ = Very similar
  • 0.5-0.7 = Moderately similar
  • 0.3-0.5 = Somewhat similar
  • <0.3 = Not similar (filtered out)

Use Cases

Concept-based search:

// Find entries about performance even if they don't use that word
semantic_search({
  query: "making things faster and more efficient"
})

Find related work:

// Find similar implementations
semantic_search({
  query: "lazy loading pattern for improving startup performance"
})

Discover connections:

// Find entries you might have forgotten about
semantic_search({
  query: "debugging concurrent database access issues"
})

Search Comparison

Feature Full-Text Date Range Semantic
Speed Fast Fast Medium
Precision High High Variable
Use Case Keywords Time-based Concepts
Highlighting Yes No No
Dependencies None None Optional ML
Filters Personal All filters Personal

Search Strategies

Strategy 1: Start with Full-Text

// 1. Try keywords first
search_entries({ query: "performance optimization" })

// 2. If too many results, add filters
search_entries({
  query: "performance optimization",
  is_personal: false
})

// 3. If still too many, use date range
search_by_date_range({
  start_date: "2025-10-01",
  end_date: "2025-10-31",
  tags: ["performance"]
})

Strategy 2: Use Semantic for Discovery

// When you're not sure of exact keywords
semantic_search({
  query: "strategies for reducing initialization overhead",
  limit: 10
})

// Then use full-text on specific entries found

Strategy 3: Combine Multiple Searches

// 1. Date range to narrow down
const oct_entries = search_by_date_range({
  start_date: "2025-10-01",
  end_date: "2025-10-31"
})

// 2. Full-text within that period
search_entries({ query: "performance" })

// 3. Semantic to find related concepts
semantic_search({ query: "optimization techniques" })

Advanced Search Techniques

Boolean Search

// AND (both terms required)
search_entries({ query: "performance AND optimization" })

// OR (either term)
search_entries({ query: "bug OR issue" })

// NOT (exclude term)
search_entries({ query: "performance NOT testing" })

// Combination
search_entries({
  query: "(lazy OR eager) AND loading NOT testing"
})

Phrase Search

// Exact phrase
search_entries({ query: '"lazy loading"' })

// Multiple phrases
search_entries({ query: '"lazy loading" OR "eager initialization"' })

Wildcard Search

// Prefix wildcard
search_entries({ query: "optim*" })
// Matches: optimize, optimization, optimized, optimal

// Multiple wildcards
search_entries({ query: "perform* optim*" })

Search Tips

Improve Full-Text Results

Use specific terms:

  • ✅ "lazy loading pattern"
  • ❌ "loading"

Multiple keywords:

  • ✅ "database connection pool leak"
  • ❌ "problem"

Use tags:

// More effective than text search for categorization
search_entries({ query: "optimization", is_personal: false })

Improve Semantic Results

Be descriptive:

  • ✅ "techniques for reducing application startup latency"
  • ❌ "faster startup"

Use context:

  • ✅ "debugging concurrent database locking in multi-threaded Python applications"
  • ❌ "database locks"

Adjust threshold:

// Strict (high quality, fewer results)
semantic_search({ query: "...", similarity_threshold: 0.6 })

// Loose (more results, lower quality)
semantic_search({ query: "...", similarity_threshold: 0.2 })

Troubleshooting

No Results

Check:

  • Spelling of search terms
  • Try broader terms
  • Remove filters
  • Check date range is correct
  • Verify entries exist

Solutions:

  • Use wildcards: optim*
  • Try semantic search
  • Browse with get_recent_entries

Too Many Results

Solutions:

  • Add more specific keywords
  • Use filters (is_personal, entry_type)
  • Use date range
  • Increase similarity threshold (semantic)

Semantic Search Not Working

Check:

  • Dependencies installed
  • Server restarted after install
  • Check logs for errors
  • Try with test_simple first

Performance

Search Speed

Full-Text (FTS5):

  • Small DB (<1000 entries): <10ms
  • Medium DB (1000-10000): <50ms
  • Large DB (>10000): <200ms

Date Range:

  • Fast (indexed by timestamp)
  • <10ms for most queries

Semantic:

  • First search: 15s (model load)
  • Subsequent: 50-200ms
  • Depends on index size

Optimization Tips

  1. Use specific queries - Faster than broad searches
  2. Limit results - Default 10 is good
  3. Filter early - Use is_personal to reduce search space
  4. Tag-based filtering - Very fast with indexes

Next: Explore Examples or learn about Tools.

🏠 Home

Getting Started

Core Features

Knowledge Graph

GitHub Integration

Workflows

Deployment

Architecture

Clone this wiki locally