Repository navigation
Releases
Inference Stack 0.1.2
Fixes Kubernetes setup selecting LiteLLM's database migration pod instead of its proxy, which could close the launcher and dashboard connections during startup.
If you already installed v0.1.1, rerun the Kubernetes installer to reconnect. It preserves your credentials, models, and Helm settings:
curl -fsSL https://github.com/BerriAI/inference-stack/releases/download/v0.1.2/install.sh | bash -s -- kubernetesThis is a preview release. Kubernetes deployment, routing, recovery, restricted-permission commands, and reconnects were tested with simulated models. Real GPU inference remains unverified.
Inference Stack 0.1.1 preview
This preview fixes the first Deploy click after changing model settings.
The README now has a shorter quickstart, an updated stack animation, and the blue LiteLLM logo in both themes. LiteLLM routes to models; llm-d selects a vLLM replica.
Set up Inference Stack · Update from v0.1.0
Docker and Kubernetes flows were tested with simulated models. Native amd64 and arm64 launcher smoke tests passed. Real-GPU model execution remains unverified; simulation returns canned responses.
Release assets and container images are built and signed by the private release pipeline. The manifest records the source commit and image digest.
Inference Stack 0.1.0
Everything you need to own your inference. Deploy with one command.
This first preview release brings together vLLM, LiteLLM, llm-d and Grafana behind a minimal launcher. Search Hugging Face, deploy a model, send a request, and inspect routing and metrics.
Docker Compose
curl -fsSL https://github.com/BerriAI/inference-stack/releases/download/v0.1.0/install.sh | bash -s -- dockerKubernetes
curl -fsSL https://github.com/BerriAI/inference-stack/releases/download/v0.1.0/install.sh | bash -s -- kubernetesSee setup and prerequisites. The installer generates credentials and tells you where to find them. Existing credentials and installations are preserved.
Included
- Native amd64 and arm64 launcher image, pinned by digest in the install bundle.
- Stack and model Helm charts with bundled dependencies.
- Installer, install bundle, source/image manifest, checksums, and a signing bundle.
- MIT-licensed original code, with upstream notices retained.
The private release pipeline smoke-tests both launcher architectures and signs the image and checksums. Local Compose and Kubernetes flows have been exercised with simulated models. Real GPU model execution remains unverified while test capacity is pending; simulation returns canned responses.
To verify the downloaded artifacts with cosign:
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
--certificate-identity https://github.com/BerriAI/inference-stack-releaser/.github/workflows/release.yml@refs/heads/main \
--certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
sha256sum --check SHA256SUMS