Skip to content

v0.4.0

Choose a tag to compare

View all tags
@vdparikh vdparikh released this 04 Sep 16:57
· 140 commits to main since this release
v0.4.0
595ef09

pantry v0.4.0

Major feature and hardening release with embeddings, worker process isolation, LaunchAgent service management, exact token usage, tool calling, and expanded curated catalog.

What's New in v0.4.0

  • Exact Token Usage (usage): Real prompt_tokens, completion_tokens, and total_tokens returned in /v1/chat/completions non-streaming JSON and terminal streaming SSE chunks.
  • Embeddings Endpoint (POST /v1/embeddings): OpenAI-compatible vector embeddings with unit normalization and Echo/MLX embed runtimes.
  • Worker Subprocess Isolation (--worker-isolation): Runs inference in an isolated worker subprocess (spawn), guaranteeing 100% Metal cache and Mach virtual memory reclaim upon /v1/unload.
  • Service Management CLI (pantry service): First-class macOS LaunchAgent management (pantry service install, start, stop, status, uninstall) configured with UIElement accessory activation policy (status bar only, clean Dock).
  • Tool / Function Calling: Host-owned <tools> prompting and <tool_call> extraction conforming to OpenAI tool call schemas.
  • Expanded Curated Catalog: Starter manifests for Llama 3.2 1B & 3B, Qwen 2.5 Coder 1.5B, DeepSeek-R1 Distill Qwen 1.5B, and embeddings, plus remote sync via pantry catalog update.

Install or Upgrade

brew tap vdplabs/tap
brew update && brew upgrade pantry

Or via pip:

pip install --upgrade "git+https://github.com/vdplabs/pantry.git#egg=pantry[mac]"
Assets 2
Loading