v0.6.3

Sentences Chunker v0.6.3

A cutting-edge tool that intelligently splits text documents into optimally-sized chunks while preserving sentence boundaries and semantic integrity. Powered by WTPSplit neural models supporting 85+ languages, ideal for retrieval-augmented generation (RAG), embeddings generation, and LLM preprocessing.

Key Features

Neural sentence segmentation with configurable token limits and overlap
GPU acceleration with automatic CPU fallback and persistent model caching
Strict mode with intelligent parameter suggestions for constraint violations
REST API with FastAPI and comprehensive metadata
Prerequisites

Docker and Docker Compose (for Docker deployment)
NVIDIA GPU with CUDA support (recommended for performance)
NVIDIA Container Toolkit (for GPU passthrough in Docker)
Python 3.10-3.12 (for local installation)
Changelog

Torch library to 2.6.0
Docker compose fix
Bugs fix
Revised workflow for creating Docker images in the Github repository.