Skip to content

fix(deps): make macOS installs work without a Rust toolchain - #3199

Merged
nicoloboschi merged 1 commit into
mainfrom
fix/litellm-macos-install
Aug 6, 2026
Merged

fix(deps): make macOS installs work without a Rust toolchain#3199
nicoloboschi merged 1 commit into
mainfrom
fix/litellm-macos-install

Conversation

@nicoloboschi

Copy link
Copy Markdown
Collaborator

Problem

A stock macOS install of the published packages is currently broken: @vectorize-io/hindsight-all (and any uvx hindsight-api / hindsight-embed install) fails during daemon startup with:

× Failed to build `litellm==1.95.0`
error: rustc 1.93.1 is not supported by the following packages:
  aws-config@1.9.0 requires rustc 1.94.1
  ...

Two stacked causes, both in litellm's packaging:

  1. litellm publishes no macOS wheels for any release ≥ 1.92.0 (only manylinux + win_amd64; ≤ 1.91.4 shipped pure-python py3-none-any wheels). Every fresh macOS install therefore compiles the sdist's Rust/PyO3 bridge — silently requiring a Rust toolchain most users don't have.
  2. 1.95.0 newly vendors the aws-smithy crates, which require rustc ≥ 1.94.1 — so since its release, even Macs with a reasonably current Rust toolchain fail. Our floor litellm>=1.93.0 resolves straight to it.

Linux and Windows are unaffected (wheels exist for cp310–cp314).

Fix

Split the requirement with platform markers in hindsight-api-slim/pyproject.toml:

  • sys_platform != 'darwin' — keep the existing >=1.93.0 floor (unchanged resolution; CI still locks 1.93.0).
  • sys_platform == 'darwin' — pin to >=1.91.3,<1.92, the last line shipping pure-python wheels, so a plain install needs no compiler at all. The 1.93.0 floor's Python 3.14 rationale (cp314 platform wheels) doesn't apply to pure wheels.

The 1.91.x line is only ~3 weeks older than 1.95.0 (1.91.4 published 2026-07-19), and every litellm API hindsight uses (acompletion, Router, aembedding, arerank, drop_params, get_max_tokens, litellm.exceptions) is present and stable there.

Revisit when litellm ships macOS wheels — tracked upstream in BerriAI/litellm#31261.

Verification (macOS arm64)

  • Reproduced the failure with the published @vectorize-io/hindsight-all@0.8.6 → daemon start dies building litellm 1.95.0.
  • Confirmed the sdist regression point: 1.93.0/1.94.0/1.94.1 vendor no aws crates and build with rustc 1.93.1; 1.95.0 requires 1.94.1.
  • With this change: fresh workspace resolve picks litellm 1.91.4 (pure wheel, no build step); the embedded daemon boots via the hindsight-all npm lifecycle manager (embedPackagePath → this branch), health check passes, and a real synchronous retain + recall ran LLM fact extraction through litellm (Gemini) successfully.
  • Non-darwin lock entries are byte-identical to before (litellm stays 1.93.0), so Linux CI exercises the same versions as main.

Once merged, the next core release makes this reach users automatically: hindsight-embed@X always launches hindsight-api@X, and the npm package defaults to hindsight-embed@latest.

Interim workaround for users on current releases: UV_CONSTRAINT=<file with "litellm<1.95.0"> (uvx honors it), or rustup update.

litellm publishes no macOS wheels for any release >= 1.92.0, so every
macOS install of the published hindsight-api compiles litellm's sdist
Rust/PyO3 bridge. That silently required a Rust toolchain, and litellm
1.95.0 raised the bar further (vendored aws-smithy crates need
rustc >= 1.94.1), breaking even machines with a recent-but-not-newest
rustc. A stock 'uvx hindsight-api' / hindsight-all install on macOS
failed during daemon startup.

Pin litellm to the 1.91.x line on darwin only - the last releases that
ship pure-python py3-none-any wheels - so installs need no compiler at
all. Linux and Windows keep the existing >= 1.93.0 floor (litellm
publishes manylinux/win_amd64 wheels there, including cp314).

Verified on macOS arm64: fresh workspace resolve picks litellm 1.91.4
(pure wheel), the embedded daemon boots via @vectorize-io/hindsight-all,
and retain/recall run real LLM extraction through litellm successfully.

Revisit when litellm ships macOS wheels (BerriAI/litellm#31261).
@nicoloboschi
nicoloboschi merged commit 8de576b into main Aug 6, 2026
103 of 104 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant