Skip to content

Infinity v3.0.0-alpha.8

Pre-release
Pre-release

Choose a tag to compare

View all tags
@michaelfeil michaelfeil released this 20 Oct 17:56
· 11 commits to main since this release
v3.0.0-alpha.8
d5f7456
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Changes since alpha.7:

  • Count each distinct OneJev block-attention prefix once per request, plus every question suffix, in JSON input usage and billing headers. A 1,000-token shared prefix plus ten 100-token suffixes reports 2,000 input tokens (#732).
  • Replace INFINITY_TORCH_PREFIX_CACHE_BYTES with INFINITY_TORCH_PREFIX_CACHE_SIZE, using explicit units such as 4GiB or 4GB. 0 disables the cache; bare nonzero values fail startup (#731).

CPU and NVIDIA image builds run asynchronously. Check the workflow for publication and startup-check results:

docker pull ghcr.io/michaelfeil/infinity_v3:nvidia-3.0.0-alpha.8
docker pull ghcr.io/michaelfeil/infinity_v3:cpu-3.0.0-alpha.8

Image builds · Licensing · Full changelog

Assets 2
Loading