Skip to content

New: Dry run feature and semantic chunking

Choose a tag to compare

View all tags
@duriantaco duriantaco released this 22 Feb 15:38
· 58 commits to main since this release
v0.0.4
3e02fcc
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

[0.0.4] - 2025-03-02

Added

  • AST-based semantic chunking for Python files:
    • Use --semantic-chunking on the CLI or semantic_chunking=True in the Python API
    • Splits .py files by top-level functions and classes
    • Falls back to a single chunk if a file has invalid syntax
    • Non-Python files remain chunked by the existing size-based logic
  • Updated tests to cover semantic chunking scenarios, including syntax-error edge cases
  • Dry Run before chunking: Added dry-run to see what files were added/removed, no actual chunking is carried out
Assets 2
Loading