Skip to content

v0.2.2: Cascading chunker, core optimizations, and stability fixes

Latest
Latest

Choose a tag to compare

View all tags
@finktech-dev finktech-dev released this 10 Jun 17:55
v0.2.2
53028b5

v0.2.0

This release finalizes the stabilization phase, focusing on processing reliability and CI/CD robustness.

Changelog

  • fix(core): Implemented a 4-level cascading chunker (Paragraphs → Sentences → Lines → Sliding Window) to prevent silent BERT truncation on dense prose.
  • perf(core): Optimized token counting by grouping encodings and caching tiktoken objects, reducing latency in batch requests.
  • fix(types): Performed a comprehensive refactor of type annotations across the API, CLI, and Pricing modules to satisfy strict CI/CD pipelines.
  • feat(ci): Added GitHub Actions workflows for automated linting, type checking, and PyPI releases.
  • feat(i18n): Added truncation warning keys in five languages: English (en), Spanish (es), Portuguese (pt), Chinese (zh), and Japanese (ja).
  • chore(ci): Initialized default environment configuration and installed inference dependencies for GitHub Actions runners.
  • fix(tests): Marked API integration tests with the integration marker to prevent out-of-memory (OOM) errors in memory-constrained environments (Win32).
Assets 4
Loading