fix: search_brain SQL bug, health check timeout, LiteLLM IP migration - #4
Merged
Merged
Conversation
feat: Open Brain v1.0 -- unified semantic knowledge backend for PAI
feat: Open Brain v1.0 -- semantic knowledge server
* docs(phase-7): research data curation domain for planning Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(07-data-curation): create phase plan for data curation 3 plans across 2 waves: - 07-01 (wave 1): schema migration, permission system, archived filtering - 07-02 (wave 2): archive_entry, list_recent, update_entry, rate_entry tools - 07-03 (wave 2): usage-weighted search, curation script (LLM-as-judge) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(v1.1): phase 7 data curation plans -- researched, planned, reviewed 3 plans in 2 waves for v1.1 Data Curation milestone: - 07-01 (wave 1): Schema migration, delete permissions, archived filtering - 07-02 (wave 2): 4 new tools (archive, list, update, rate) - 07-03 (wave 2): Usage-weighted search + LLM-as-judge curation script Adversarial review caught 2 critical + 5 warnings, all resolved: - Added CUR-01..07 requirements to REQUIREMENTS.md - Descoped brain_stats explicitly - Added archived_at guards on rate/update tools - Fixed O(n^2) duplicate detection to HNSW O(n log n) - Fixed update_entry idempotentHint to false Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(07-01): add curation migration and delete permission - 002_curation.sql: archived_at, access_count, last_accessed_at, usefulness_score on all 5 tables - Partial indexes for active-only queries on all 5 tables - Sessions updated_at column + update trigger - Permission type extended with "delete" - canDelete() function, admin and n8n get RWD, other roles unchanged Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(07-01): filter archived rows from all read paths - search_brain: AND archived_at IS NULL in every table CTE - session_load: archived_at IS NULL in both project and global queries - find_person: archived_at IS NULL in both name and semantic search - New test verifying archived_at filtering in search_brain SQL - Updated session-load test for global query WHERE clause change Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(07-01): complete curation foundation plan - 07-01-SUMMARY.md with task results, deviations, self-check - STATE.md: plan 1/3 complete, 33% progress - ROADMAP.md: 07-01 checked off, phase 7 in progress - REQUIREMENTS.md: CUR-01 marked complete Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(07-02): implement archive_entry and list_recent tools with tests - archive_entry soft-deletes via archived_at, enforces canDelete, idempotent on re-archive - list_recent returns chronological entries with configurable date range, limit, table filter, archived toggle - Both tools follow established registration pattern with full MCP protocol tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(07-03): add failing tests for usage tracking and usefulness-weighted ranking - Usage tracking: verify fire-and-forget UPDATE for access_count/last_accessed_at - Usage tracking: verify no tracking on empty results or embed failure - Usefulness ranking: verify composite ORDER BY with distance+usefulness weights - Usefulness ranking: verify CTE includes usefulness_score column - Updated existing ORDER BY assertion to check composite formula components Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(07-03): add usage tracking and usefulness-weighted ranking to search_brain - Fire-and-forget access_count/last_accessed_at tracking after successful search - Composite ORDER BY: 80% vector distance + 20% usefulness score - CTE SELECT includes COALESCE(usefulness_score, 0.5) for neutral default - LABEL_TO_TABLE reverse map for grouping returned rows by source table - Tracking errors logged at warn level, never block search response - Updated existing tests for new query patterns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(07-02): implement update_entry and rate_entry tools, wire all four into registerAllTools - update_entry handles per-table field validation, re-embedding, hash collision detection, archived guard - rate_entry sets usefulness_score with canWrite enforcement and archived_at IS NULL guard - All four curation tools (archive, list_recent, update, rate) registered in registerAllTools - 206 tests pass, TypeScript compiles cleanly Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(07-03): create curation script with duplicate, stale, and vague detection - HNSW nearest-neighbor duplicate detection (O(n log n), not cross-join) - LLM-as-judge for stale entry evaluation (KEEP/ARCHIVE/DOWNGRADE) - LLM-as-judge for vague content quality scoring (0.0-1.0) - --dry-run flag for safe previewing without mutations - Idempotent: archived_at IS NULL guards prevent re-processing - Direct SQL operations (no MCP tools) for performance - Added "curate" script entry to package.json Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(07-02): complete curation tools plan - SUMMARY.md with task results, deviations, self-check - STATE.md updated to 2/3 plans, 67% progress - ROADMAP.md marks 07-02 complete - REQUIREMENTS.md marks CUR-02, CUR-04, CUR-05 complete Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(07-03): complete usage-weighted search + curation script plan - SUMMARY.md with task details, deviations, self-check passed - STATE.md: 3/3 plans complete, 100% progress, phase complete - ROADMAP.md: 07-03 marked complete, progress table updated - REQUIREMENTS.md: CUR-03, CUR-06, CUR-07 marked complete Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: add migration and curate dry-run to CI pipeline CI already has a pgvector container but wasn't using it for migration verification. Now runs migrations against the test DB and verifies the curate script starts cleanly in dry-run mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: add Claude Code review workflow and upgrade CI to Node 24 - New claude-code-review.yml runs on self-hosted runner (LXC 106) with Claude Code 2.1.76 via LiteLLM proxy - CI stays on ubuntu-latest (needs Docker for pgvector container) - Bump actions/checkout to v6, add FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: remove curate dry-run from CI, covered by typecheck The curate script needs real data and working pgvector types to run meaningfully. CI's pgvector type registration fails at pool connect time because the extension hasn't been created yet (chicken-and-egg with migrations). Typecheck already validates the script compiles. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: add concurrency group to claude review workflow Cancels in-progress reviews when new commits are pushed to the same PR, preventing stale reviews from wasting runner time. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…tion - Remove ORDER BY/LIMIT from individual CTEs in search_brain and wrap UNION ALL in subquery -- fixes "invalid UNION/INTERSECT/EXCEPT ORDER BY" - Switch health check from /health to /health/liveliness -- avoids 3s timeout on LiteLLM's full model health sweep (33 endpoints) - Update LiteLLM IPs from 10.71.20.53 to 10.71.1.33/10.71.20.33 - Fix health test to mock unreachable LiteLLM instead of assuming env unset Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2 tasks
rodaddy
added a commit
that referenced
this pull request
Mar 16, 2026
…#4) (#5) * feat: Open Brain v1.1 -- Data Curation tools (#3) * docs(phase-7): research data curation domain for planning * docs(07-data-curation): create phase plan for data curation 3 plans across 2 waves: - 07-01 (wave 1): schema migration, permission system, archived filtering - 07-02 (wave 2): archive_entry, list_recent, update_entry, rate_entry tools - 07-03 (wave 2): usage-weighted search, curation script (LLM-as-judge) * docs(v1.1): phase 7 data curation plans -- researched, planned, reviewed 3 plans in 2 waves for v1.1 Data Curation milestone: - 07-01 (wave 1): Schema migration, delete permissions, archived filtering - 07-02 (wave 2): 4 new tools (archive, list, update, rate) - 07-03 (wave 2): Usage-weighted search + LLM-as-judge curation script Adversarial review caught 2 critical + 5 warnings, all resolved: - Added CUR-01..07 requirements to REQUIREMENTS.md - Descoped brain_stats explicitly - Added archived_at guards on rate/update tools - Fixed O(n^2) duplicate detection to HNSW O(n log n) - Fixed update_entry idempotentHint to false * feat(07-01): add curation migration and delete permission - 002_curation.sql: archived_at, access_count, last_accessed_at, usefulness_score on all 5 tables - Partial indexes for active-only queries on all 5 tables - Sessions updated_at column + update trigger - Permission type extended with "delete" - canDelete() function, admin and n8n get RWD, other roles unchanged * feat(07-01): filter archived rows from all read paths - search_brain: AND archived_at IS NULL in every table CTE - session_load: archived_at IS NULL in both project and global queries - find_person: archived_at IS NULL in both name and semantic search - New test verifying archived_at filtering in search_brain SQL - Updated session-load test for global query WHERE clause change * docs(07-01): complete curation foundation plan - 07-01-SUMMARY.md with task results, deviations, self-check - STATE.md: plan 1/3 complete, 33% progress - ROADMAP.md: 07-01 checked off, phase 7 in progress - REQUIREMENTS.md: CUR-01 marked complete * feat(07-02): implement archive_entry and list_recent tools with tests - archive_entry soft-deletes via archived_at, enforces canDelete, idempotent on re-archive - list_recent returns chronological entries with configurable date range, limit, table filter, archived toggle - Both tools follow established registration pattern with full MCP protocol tests * test(07-03): add failing tests for usage tracking and usefulness-weighted ranking - Usage tracking: verify fire-and-forget UPDATE for access_count/last_accessed_at - Usage tracking: verify no tracking on empty results or embed failure - Usefulness ranking: verify composite ORDER BY with distance+usefulness weights - Usefulness ranking: verify CTE includes usefulness_score column - Updated existing ORDER BY assertion to check composite formula components * feat(07-03): add usage tracking and usefulness-weighted ranking to search_brain - Fire-and-forget access_count/last_accessed_at tracking after successful search - Composite ORDER BY: 80% vector distance + 20% usefulness score - CTE SELECT includes COALESCE(usefulness_score, 0.5) for neutral default - LABEL_TO_TABLE reverse map for grouping returned rows by source table - Tracking errors logged at warn level, never block search response - Updated existing tests for new query patterns * feat(07-02): implement update_entry and rate_entry tools, wire all four into registerAllTools - update_entry handles per-table field validation, re-embedding, hash collision detection, archived guard - rate_entry sets usefulness_score with canWrite enforcement and archived_at IS NULL guard - All four curation tools (archive, list_recent, update, rate) registered in registerAllTools - 206 tests pass, TypeScript compiles cleanly * feat(07-03): create curation script with duplicate, stale, and vague detection - HNSW nearest-neighbor duplicate detection (O(n log n), not cross-join) - LLM-as-judge for stale entry evaluation (KEEP/ARCHIVE/DOWNGRADE) - LLM-as-judge for vague content quality scoring (0.0-1.0) - --dry-run flag for safe previewing without mutations - Idempotent: archived_at IS NULL guards prevent re-processing - Direct SQL operations (no MCP tools) for performance - Added "curate" script entry to package.json * docs(07-02): complete curation tools plan - SUMMARY.md with task results, deviations, self-check - STATE.md updated to 2/3 plans, 67% progress - ROADMAP.md marks 07-02 complete - REQUIREMENTS.md marks CUR-02, CUR-04, CUR-05 complete * docs(07-03): complete usage-weighted search + curation script plan - SUMMARY.md with task details, deviations, self-check passed - STATE.md: 3/3 plans complete, 100% progress, phase complete - ROADMAP.md: 07-03 marked complete, progress table updated - REQUIREMENTS.md: CUR-03, CUR-06, CUR-07 marked complete * ci: add migration and curate dry-run to CI pipeline CI already has a pgvector container but wasn't using it for migration verification. Now runs migrations against the test DB and verifies the curate script starts cleanly in dry-run mode. * ci: add Claude Code review workflow and upgrade CI to Node 24 - New claude-code-review.yml runs on self-hosted runner (LXC 106) with Claude Code 2.1.76 via LiteLLM proxy - CI stays on ubuntu-latest (needs Docker for pgvector container) - Bump actions/checkout to v6, add FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 * ci: remove curate dry-run from CI, covered by typecheck The curate script needs real data and working pgvector types to run meaningfully. CI's pgvector type registration fails at pool connect time because the extension hasn't been created yet (chicken-and-egg with migrations). Typecheck already validates the script compiles. * ci: add concurrency group to claude review workflow Cancels in-progress reviews when new commits are pushed to the same PR, preventing stale reviews from wasting runner time. --------- * fix: search_brain SQL bug, health check timeout, and LiteLLM IP migration - Remove ORDER BY/LIMIT from individual CTEs in search_brain and wrap UNION ALL in subquery -- fixes "invalid UNION/INTERSECT/EXCEPT ORDER BY" - Switch health check from /health to /health/liveliness -- avoids 3s timeout on LiteLLM's full model health sweep (33 endpoints) - Update LiteLLM IPs from 10.71.20.53 to 10.71.1.33/10.71.20.33 - Fix health test to mock unreachable LiteLLM instead of assuming env unset --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
search_brainSQL error ("invalid UNION/INTERSECT/EXCEPT ORDER BY clause") by removing ORDER BY/LIMIT from individual CTEs and wrapping UNION ALL in a subquery/healthto/health/livelinessto avoid 3s timeout on LiteLLM's full 33-model health sweepTest plan
mcp2cli open-brain search_brainreturns ranked results🤖 Generated with Claude Code