Skip to content

[Perf] Graph search index endpoint loads full record bodies for all entity types and daily logs #3507

Description

@atomantic

Problem

GET /api/brain/graph/search-index (server/routes/brainGraph.js:25-28) invokes getBrainGraphSearchIndex() (server/services/brainGraph.js:106-109), which calls loadNodes() (server/services/brainGraph.js:52-103). loadNodes() calls brainStorage.getAll(type) for 5 entity types (people, projects, ideas, admin, memories), plus getGoals(), plus brainStorage.getAll('journals'). This reads and JSON-parses full record bodies for every entity and daily log entry on disk, only for getBrainGraphSearchIndex() to discard all body/summary/tag fields and return { id, label, brainType }.

Impact

Opening or focusing the brain graph search input forces a complete disk walk and JSON parse of every brain entity and daily log file. As the store grows, this lightweight search dropdown causes significant disk I/O latency and memory overhead.

Fix

Optimize getBrainGraphSearchIndex() or loadNodes() to fetch only the minimal { id, label, brainType } projection (e.g., using collectionStore.listIds() or lightweight header metadata), avoiding reading full record content files from disk.

Files touched:

  • server/services/brainGraph.js
  • server/routes/brainGraph.js

Acceptance criteria

  • GET /api/brain/graph/search-index does not load full record content bodies from disk for all brain entity types and daily log entries.
  • The search index endpoint returns accurate { nodes: [{ id, label, brainType }] } projections with low latency.
  • Unit tests in server/services/brainGraph.test.js pass.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:brainBrain notes/memories/goals/knowledge graphbugSomething isn't workingplanTracked by /do:replan

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions