v0.3.0
Pipeline validation and Tracer Overhaul
What's Changed
✨ Features
- feat(pipelines): tokenizer resource nodes — chunk_size counts real tokens by @Neeeser in #108
- feat(ingestion): never-fail embedding token guard with ingestion warnings by @Neeeser in #109
- refactor(pipelines): move tokenizer selection onto chunker config by @Neeeser in #112
- feat(dev): use ParadeDB for local development by @Neeeser in #114
- feat(traces): preserve result traceability across pipeline nodes by @Neeeser in #113
- feat(traces): streamline retrieval diagnosis by @Neeeser in #115
🐛 Bug fixes
Full Changelog: v0.2.1...v0.3.0