Skip to content

Inference Stack 0.1.0

Pre-release
Pre-release

Choose a tag to compare

@moe-berri moe-berri released this 22 Nov 00:51
8ea01e5

Everything you need to own your inference. Deploy with one command.

This first preview release brings together vLLM, LiteLLM, llm-d and Grafana behind a minimal launcher. Search Hugging Face, deploy a model, send a request, and inspect routing and metrics.

Docker Compose

curl -fsSL https://github.com/BerriAI/inference-stack/releases/download/v0.1.0/install.sh | bash -s -- docker

Kubernetes

curl -fsSL https://github.com/BerriAI/inference-stack/releases/download/v0.1.0/install.sh | bash -s -- kubernetes

See setup and prerequisites. The installer generates credentials and tells you where to find them. Existing credentials and installations are preserved.

Included

  • Native amd64 and arm64 launcher image, pinned by digest in the install bundle.
  • Stack and model Helm charts with bundled dependencies.
  • Installer, install bundle, source/image manifest, checksums, and a signing bundle.
  • MIT-licensed original code, with upstream notices retained.

The private release pipeline smoke-tests both launcher architectures and signs the image and checksums. Local Compose and Kubernetes flows have been exercised with simulated models. Real GPU model execution remains unverified while test capacity is pending; simulation returns canned responses.

To verify the downloaded artifacts with cosign:

cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/BerriAI/inference-stack-releaser/.github/workflows/release.yml@refs/heads/main \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
sha256sum --check SHA256SUMS