fix: Phase 3 core fixes — DB paths, date filtering, search metadata

[EtanHey]

Summary

• DB path fix: Centralized path resolution in paths.py — all 8 modules now resolve to the correct ~/.local/share/zikaron/zikaron.db instead of the empty brainlayer.db
• Date filtering: Added created_at column with date_from/date_to search params in MCP tool, backfill script that searches archives
• Search metadata: Results now include created_at timestamps and source (claude_code/whatsapp/youtube)
• Project normalization: Decode Claude Code path encoding (-Users-etanheyman-Gits-golems → golems) for cleaner filtering
• Chunker fix: Sentence-aware splitting for oversized paragraphs instead of mid-sentence truncation
• Data documentation: Comprehensive docs/data-locations.md — where all data lives, archive strategy, migration history

Backfill Results

Ran scripts/backfill-created-at.py — 107,935/268,864 chunks (40.1%) now have created_at:

• 35,339 from metadata timestamps (WhatsApp/YouTube)
• 72,596 from archived JSONL session files
• 160,929 remaining are pre-archiver sessions whose JSONL files no longer exist

Test plan

• Verify MCP search works with date_from/date_to params
• Verify project normalization in search results
• Verify BRAINLAYER_DB env var override works
• Run backfill script on fresh DB to verify archive path scanning

🤖 Generated with Claude Code

---

Note

Medium Risk
Touches core persistence/search paths and extends the SQLite schema with a new indexed created_at field, which can affect query results and indexing behavior across multiple entrypoints. Risk is mitigated by additive schema changes and backfill tooling, but rollout should verify searches still hit the intended DB and filters behave as expected.

Overview
Centralizes DB path resolution via new src/brainlayer/paths.py (supports BRAINLAYER_DB override and prefers the legacy ~/.local/share/zikaron/zikaron.db when present), and updates daemon/dashboard/MCP/pipelines/scripts to import DEFAULT_DB_PATH instead of hardcoding paths.

Adds temporal metadata and filtering by introducing a created_at column + index on chunks, populating it on new ingests (index_new.py), surfacing it (and source) in search metadata, and extending MCP brainlayer_search with date_from/date_to that flow through VectorStore.search/hybrid_search.

Adds a new scripts/backfill-created-at.py to backfill timestamps from existing metadata, session JSONL files (including archives/manifests), file mtimes, and a final rowid-based estimate pass; also improves text chunking to split oversized paragraphs on sentence boundaries and documents data/archive locations in docs/data-locations.md.

^{Written by Cursor Bugbot for commit 68869bb. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

[cursor]

Filesystem fallback truncates multi-segment project names

Medium Severity

normalize_project_name falls back to remaining[0] (only the first segment after "Gits") when the os.path.isdir() filesystem lookup fails. For multi-segment project names like rudy-monorepo, this returns "rudy" instead of "rudy-monorepo" when the original directory path no longer exists. Since the PR documents that repos moved from ~/Desktop/Gits/ to ~/Gits/, old encoded paths will fail the lookup against the old gits_dir and produce truncated names, breaking project filtering.

 

[cursor]

Date-only date_to excludes entries from that day

Medium Severity

The date_to filter uses created_at <= ? with string comparison, but the MCP tool schema encourages date-only values like '2026-02-19'. Since created_at stores full ISO 8601 timestamps (e.g. "2026-02-19T10:30:00+00:00"), and in lexicographic ordering "2026-02-19T..." > "2026-02-19", all entries actually on the date_to date are excluded. date_from is unaffected because the inequality goes the other direction.

Additional Locations (2)

• src/brainlayer/vector_store.py#L427-L429
• src/brainlayer/vector_store.py#L608-L611

 

[cursor]

Duplicate project normalization diverges from CLI implementation

Low Severity

normalize_project_name in mcp/__init__.py reimplements project name normalization that already exists in cli/__init__.py (_clean_project_name + _normalize_project_name). The two implementations differ: the CLI version handles additional path markers (Desktop, projects, config), supports monorepo package mappings and project aliases, and has no filesystem dependency. Having two divergent implementations risks inconsistent behavior between indexing (CLI) and searching (MCP).