Skip to content

v6.1.2

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 12 Aug 16:42
v6.1.2
35a4f80

Install

Prebuilt binaries (linux amd64/arm64, macOS Apple Silicon) are
attached below, e.g.:

curl -fsSL https://github.com/weka/wekai/releases/download/v6.1.2/wekai_v6.1.2_$(uname -s | tr A-Z a-z)_$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/').tar.gz | tar -xz

Benchmark

Replay benchmark against your endpoint (see chart README for all values).
This image embeds the replay artifact, so no volume or fetch step is needed:

helm upgrade --install my-replay oci://quay.io/weka.io/helm/wekai \
  --version v6.1.2 \
  --set endpoint=http://YOUR-LLM-HOST:8000

docker pull quay.io/weka.io/wekai:v6.1.2

Router

Model-aware LLM router with prefix-cache affinity. Separate,
replay-less image — same binary, without the multi-GB replay layer a
router never reads.

Point it at your fleet by pod label. Each discovered pod contributes
its own port, so several DaemonSets on different ports stay one pool:

helm upgrade --install my-router oci://quay.io/weka.io/helm/wekai-router \
  --version v6.1.2 \
  --set-string 'router.backends[0].pods.app=vllm' \
  --set-string 'router.backends[0].port=http'

A static list works the same way, and the two can be mixed:

  --set-string 'router.backends[0]=http://vllm-a:8000' \
  --set-string 'router.backends[1]=http://vllm-b:8000'

Per-model routes, a hosted fallback and credentials are in
docs/router.md.

docker pull quay.io/weka.io/wekai-router:v6.1.2

CLI (binary is named wekai):

go install github.com/weka/wekai@v6.1.2

Full Changelog: v6.1.1...v6.1.2

Assets 5
Loading