v1.9.0

1.9.0 (2025-03-28)

Features

• Chunking can now be configured using embed_sources in segment_processing.{segment_type} configuration and allows the choice of pre-configured tokenizers or any huggingface tokenizer by setting the tokenizer field in chunk_processing (#420) (d88ac64)

Bug Fixes

• Updated hashmap to lru for caching (d868c76)