Skip to content

Foretoken 0.0.6

Choose a tag to compare

View all tags
@shiweijiezero shiweijiezero released this 24 Oct 02:28
v0.0.6
d26867a

What's changed

Cold installation and model startup

  • Model preparation and serving startup now receive their full spec.timeouts.startup budget after the first model container starts. Scheduling, runtime-image downloads and init-container setup no longer consume that budget. Preparation remains bounded by Kubernetes' native Job deadline, including Pod retries.
  • Preparation failures retain Kubernetes' original deadline reason while Pods terminate, instead of being reported as an unspecified container failure.
  • Platform installation prefers an accessible official OCI registry. Public proxies remain available when the origin cannot be reached, and explicit registry or image settings are preserved. A faster manifest response alone no longer moves an installation to another runtime-image source.
  • Source and release builds normalize native dependency input permissions, so equivalent checkouts reuse Mooncake compilation even when shared directories use a different umask. Source files keep their local permissions.

Benchmark startup and recording

  • foretoken perf parses help and arguments before loading measurement backends. Fixed prompts and local data avoid initializing EvalScope's unrelated benchmark registry and Hugging Face dataset readers; random and remote-dataset workloads still use their existing native implementations.
  • Video help also avoids dataset initialization. HTTP, trace, SLO and video execution retain their existing commands and measurement clocks, and the normal installation still includes the complete dependency set.
  • Diagnostics now respect SDK loggers that intentionally keep their records private. This fixes W&B recording failures during shutdown while retaining both Foretoken's run.log and W&B's native diagnostics and recordings.

Artifacts

This release provides foretoken==0.0.6 on PyPI, Helm Chart 0.0.6, foretoken-applications-0.0.6-linux-amd64.tar.gz, and these Linux amd64 environments:

ghcr.io/shiweijiezero/foretoken/control-plane-environment:0.0.6
ghcr.io/shiweijiezero/foretoken/frontend-environment:0.0.6
ghcr.io/shiweijiezero/foretoken/model-server-environment:0.0.6

Matching source-build caches are published with the environments. The NVIDIA inference environment retains vLLM 0.31 and the complete Mooncake package from 0.0.5. The MetaX runtime image remains deferred; source deployment is available.

Upgrading

Upgrade the CLI in its active Python environment:

uv pip install --upgrade foretoken==0.0.6  # Or: pip install --upgrade foretoken==0.0.6

The platform administrator then runs foretoken install, retaining any existing custom values or registry options. This updates the platform template and the controller permission needed to set preparation deadlines. Redeploy each instance that should adopt the new controller and defaults, using its existing configuration and instance name.

For source installations, update to the matching source and use foretoken install -e ., followed by foretoken deploy PATH --name INSTANCE -e .. Existing deployments retain their selected applications until explicitly redeployed.

See the 0.0.6 Quick Start for a new installation.

Known issues

Workload replacement can inherit an incorrect startup deadline. Source builds can repeat runtime downloads or Mooncake compilation even when compatible content is available. These issues are fixed in 0.0.7.

Thanks

  • @shiweijiezero — cold-start lifecycle fixes, benchmark startup and logging improvements, and release validation.

Full changelog

Changes from v0.0.5 to v0.0.6

Assets 6
Loading