Skip to content

v5.0.2

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Aug 15:34
· 28 commits to main since this release
v5.0.2
60a2bc4

Install

Prebuilt binaries (linux amd64/arm64, macOS Apple Silicon) are
attached below, e.g.:

curl -fsSL https://github.com/weka/wekai/releases/download/v5.0.2/wekai_v5.0.2_$(uname -s | tr A-Z a-z)_$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/').tar.gz | tar -xz

Benchmark

Replay benchmark against your endpoint (see chart README for all values).
This image embeds the replay artifact, so no volume or fetch step is needed:

helm upgrade --install my-replay oci://quay.io/weka.io/helm/wekai \
  --version v5.0.2 \
  --set endpoint=http://YOUR-LLM-HOST:8000

docker pull quay.io/weka.io/wekai:v5.0.2

Router

Model-aware LLM router with prefix-cache affinity. Separate,
replay-less image — same binary, without the multi-GB replay layer a
router never reads:

helm upgrade --install my-router oci://quay.io/weka.io/helm/wekai-router \
  --version v5.0.2 \
  --set router.routes[0]="* => http://vllm-a:8000|http://vllm-b:8000"

docker pull quay.io/weka.io/wekai-router:v5.0.2

CLI (binary is named wekai):

go install github.com/weka/wekai@v5.0.2

Full Changelog: v5.0.1...v5.0.2

Assets 5
Loading