Skip to content

v0.6.2 — GPU-contention fix: vision calls wait for generation jobs instead of failing

Choose a tag to compare

View all tags
@dmmdea dmmdea released this 04 Jul 21:54
· 846 commits to main since this release
v0.6.2
d88096b

v0.6.2

Fixes the worst production failure mode on single-GPU machines: vision calls (vqa/ocr/assess_image/video_describe) mass-failed with 5xx whenever an image/video generation job occupied the GPU — every failure fell back to your cloud model, burning exactly the tokens this harness exists to save (in one observed incident: 295 defers in a single hour).

  • Vision calls now detect the generation GPU lock and wait (bounded, configurable vision_gpu_wait_sec, default 90s) instead of failing
  • Distinct defer reason + ledger class gpu_busy so contention is visible in ledger reports
  • One automatic retry after 3s on transient 5xx
  • New config: gpu_lock_path, vision_gpu_wait_sec (see regenerated config.example.json)

Upgrade: go install github.com/dmmdea/offload-harness@v0.6.2 or grab the Windows binary below.

Assets 3
Loading