Skip to content

v0.1.1

Choose a tag to compare

@hlin99 hlin99 released this 05 Apr 03:56
· 17 commits to main since this release

Bug fixes for issues #48-#65:

  • render_dummy_text cycles tokens for any max_tokens value
  • Streaming requests logged to JSONL
  • max_model_len enforced in all code paths
  • prefill/app.py: /ping and /metrics endpoints
  • Warmup penalty works with scheduling
  • /v1/completions routes through scheduler
  • Scheduler sync-safe (no event loop dependency)
  • CLI arg priority fixed (_TrackAction subparser bug)
  • Batch metrics always present in /metrics
  • /debug/batch returns zero-state when scheduling disabled
  • Prefill batch FIFO ordering fixed
  • PyPI publish workflow added