Commit 8de2525
feat: Add database schema for document-aware chunking (Phase 1)
This commit introduces the foundational schema and types needed to support
document chunking while maintaining backward compatibility with existing data.
Schema Changes:
- Added `is_chunk`, `parent_document_id`, `chunk_order` columns to nodes table
- Added `edge_type` column to edges table (semantic, parent-child, sequential, manual)
- All new columns have appropriate defaults for backward compatibility
Type Updates:
- NodeRecord now includes chunking metadata fields
- EdgeRecord now includes edgeType discriminator
- Created parseNodeRow() and parseEdgeRow() helper functions
Core Infrastructure:
- src/core/import.ts - Document import with chunking and structural edges
- src/lib/chunking.ts - Header-based and size-based chunking strategies
Updated all edge/node creation sites to include new required fields:
- Semantic edges marked as 'semantic' (auto-linking)
- Manual edges marked as 'manual' (user-created links)
- Parent-child edges marked as 'parent-child' (document structure)
- Sequential edges marked as 'sequential' (chunk ordering)
- All nodes marked with isChunk=false unless explicitly chunked
Next phases will add:
- Document reconstruction utilities
- UX updates to hide chunks and show parent documents
- forest import command for explicit document ingestion
🤖 Generated with Claude Code (https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>1 parent 80ce3e4 commit 8de2525
10 files changed
Lines changed: 1014 additions & 74 deletions
File tree
- src
- cli
- commands
- core
- lib
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
| 93 | + | |
93 | 94 | | |
94 | 95 | | |
95 | 96 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
130 | 130 | | |
131 | 131 | | |
132 | 132 | | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
133 | 136 | | |
134 | 137 | | |
135 | 138 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
769 | 769 | | |
770 | 770 | | |
771 | 771 | | |
| 772 | + | |
772 | 773 | | |
773 | 774 | | |
774 | 775 | | |
| |||
786 | 787 | | |
787 | 788 | | |
788 | 789 | | |
| 790 | + | |
789 | 791 | | |
790 | 792 | | |
791 | 793 | | |
| |||
0 commit comments