Skip to content

v1.8.5

Latest

Choose a tag to compare

@corcillo corcillo released this 11 Nov 05:36
· 16 commits to main since this release
v1.8.5
82149a2

Major changes

  • Memory escalation now recovers from pod-level OOM kills. When a worker is lost whole (for example an out-of-memory kill that takes the pod down before any usage is reported), the scheduler now steps the action's memory reservation up on re-dispatch once the same action has lost escalate_after_worker_losses workers (default 2, 0 disables), instead of re-dispatching at the same size until the loss budget fails it. Workers also report each running action's peak memory on the keepalive, so an action that was observed over its reservation escalates on the first loss, and a memory-kill escalation now steps past the sampled peak rather than just past the old reservation. (#3168, #3170 by @amankrx)
  • macOS builds of the source tree work again. The Linux-only caps dependency introduced in #3166 is now gated to Linux, restoring cargo and Bazel builds on macOS and the aarch64-apple-darwin release artifact. (#3174 by @corcillo)

Upgrade and rollback

  • Loss-driven memory escalation is on by default. Actions that repeatedly lose workers for non-memory reasons (spot preemption, node drains) inflate their reservations too; set escalate_after_worker_losses: 0 under memory_escalation for the v1.8.4 behavior. Older binaries, including v1.8.4, reject a config that names the new field — keep it out of shared configs until every consumer runs v1.8.5, and remove it before rolling back.
  • A memory-kill escalation steps past the sampled peak, so a single escalation can jump several size classes at once (still bounded by the fleet ceiling).
  • The keepalive memory report (RunningOperations.memory) is safe in mixed fleets in both directions: an older worker never reports and the scheduler falls back to the loss-count rule; an older scheduler ignores the field.
  • Unknown instance names and missing request fields now return INVALID_ARGUMENT instead of INTERNAL, so clients fail fast instead of retrying: a mistyped --remote_instance_name errors immediately rather than after Bazel's retry backoff.

⛰️ Features

  • Report each running action's memory peak on the keepalive and escalate a lost worker's action that was over its reservation by @amankrx in (#3170) - (5ca5732)
  • Escalate memory on the second worker lost under an action by @amankrx in (#3168) - (a51db80)

🐛 Bug Fixes

🧪 Testing & CI

⚙️ Miscellaneous

  • Pricing page rewrite, comparison table, trial request, and ranked type and radius scales by @modernmedici in (#3167) - (af1646c)

⬆️ Bumps & Version Updates

Full Changelog: v1.8.4...v1.8.5