Skip to content

release: v0.3.0

Choose a tag to compare

@finktech-dev finktech-dev released this 14 Jun 05:11
· 19 commits to main since this release
  • feat: implement split architecture (Alpine API + Debian Models)
  • feat: proxy file conversion to models container in split mode
  • feat: batch BERT inference for 3–5× lower latency on large inputs
  • feat: persist pricing data to disk cache (~/.llmzip/prices.json)
  • feat: offline-first tiktoken support in Docker (pre-downloaded encodings)
  • fix: resolve permission error for logging directory under non-root user
  • fix: correct original/compressed token tracking in batch responses
  • fix: resolve duplicate LiteLLM fetches under concurrent load
  • test: expand test suite covering pricing, i18n, and chunker stress

BREAKING CHANGES

  • resolve_prices() and fetch_prices() now return:
    tuple[dict[str, PriceEntry], dict[str, str]]
  • The default Dockerfile now builds the lightweight Alpine API image and no longer contains ML dependencies. To run the monolith architecture, you must build from Dockerfile.models or pull the -models image variant.