fix: critical bug fixes for indexing, chunking, and search stability

[runonthespot]

Summary

This PR addresses 5 critical and high-priority bugs that affect core stability and correctness across indexing, chunking, and search functionality.

High Priority Bug Fixes

🔧 Model Propagation (ck-index/src/lib.rs:1072)

• Problem: index_single_file always called ck_chunk::chunk_text without the resolved model name, causing striding to use default 8k token ceiling instead of model-specific limits (e.g., bge-small's 512-token window)
• Solution: Added model_name() method to Embedder trait and updated index_single_file_with_progress to use chunk_text_with_model with the embedder's model name
• Impact: Chunking now respects embedder token limits, preventing fastembed from silently truncating oversized functions

🔧 UTF-8 Safe Striding (ck-chunk/src/lib.rs:476-509)

• Problem: stride_large_chunk used floating-point estimates to slice UTF-8 strings directly, causing runtime panics on multi-byte character boundaries
• Solution: Converted to character-based indexing using char_indices() and safe byte position mapping
• Impact: Eliminates runtime panics when striding text with Unicode characters, emoji, or CJK text

🔧 CRLF Line Offset Correction (ck-chunk/src/lib.rs:216-249)

• Problem: Generic chunker assumed 1-byte newlines, causing byte offset errors with CRLF files where spans undershoot by the number of \r characters
• Solution: Implemented proper line ending detection (CRLF=2 bytes, LF=1 byte, CR=1 byte) in byte offset calculations
• Impact: Accurate spans and previews for Windows/mixed line ending files

🔧 Deterministic Reranking (ck-engine/src/semantic_v3.rs:217-236)

• Problem: Reranking matched by preview text, causing duplicate previews to miss score updates and wrong resort ordering
• Solution: Used document matching with HashMap-based duplicate handling for deterministic score reassignment
• Impact: Consistent reranking results regardless of duplicate content

🔧 Haskell Type Synonym Fix (ck-chunk/src/lib.rs:331-338)

• Problem: Tree-sitter filter had typo type_synomym instead of type_synonym
• Solution: Fixed the typo to type_synonym
• Impact: Haskell type synonym declarations now properly tagged as semantic chunks

Technical Improvements

• Enhanced Embedder trait with model_name() method for proper model propagation
• Updated FastEmbedder and DummyEmbedder to store and expose model names
• Fixed non-fastembed build compilation by correctly passing model parameter
• Updated test embedders to implement new model_name() method
• Ensured byte spans align with original file structure using proper line offset calculations

Testing & Quality Assurance

• ✅ All 130+ tests passing across all crates
• ✅ No clippy warnings
• ✅ Code properly formatted
• ✅ Full backward compatibility maintained
• ✅ Comprehensive regression tests for each fix

Impact

These fixes address fundamental stability and correctness issues that could cause:

• Runtime panics on Unicode text
• Incorrect chunking for small-context embedders
• Wrong byte offsets on Windows files
• Inconsistent reranking results
• Missing Haskell type declarations in search

The changes ensure robust text processing across different file formats, character encodings, and embedding models.

🤖 Generated with Claude Code