Skip to content

v6.0.4

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Aug 21:32
· 14 commits to main since this release
v6.0.4
e44e62c

Install

Prebuilt binaries (linux amd64/arm64, macOS Apple Silicon) are
attached below, e.g.:

curl -fsSL https://github.com/weka/wekai/releases/download/v6.0.4/wekai_v6.0.4_$(uname -s | tr A-Z a-z)_$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/').tar.gz | tar -xz

Benchmark

Replay benchmark against your endpoint (see chart README for all values).
This image embeds the replay artifact, so no volume or fetch step is needed:

helm upgrade --install my-replay oci://quay.io/weka.io/helm/wekai \
  --version v6.0.4 \
  --set endpoint=http://YOUR-LLM-HOST:8000

docker pull quay.io/weka.io/wekai:v6.0.4

Router

Model-aware LLM router with prefix-cache affinity. Separate,
replay-less image — same binary, without the multi-GB replay layer a
router never reads:

helm upgrade --install my-router oci://quay.io/weka.io/helm/wekai-router \
  --version v6.0.4 \
  --set router.routes[0]="* => http://vllm-a:8000|http://vllm-b:8000"

docker pull quay.io/weka.io/wekai-router:v6.0.4

CLI (binary is named wekai):

go install github.com/weka/wekai@v6.0.4

Full Changelog: v6.0.3...v6.0.4

Assets 5
Loading