Skip to content

hotpath 0.3.0

Choose a tag to compare

@alityb alityb released this 06 Apr 00:18
· 10 commits to master since this release

hotpath 0.3.0 focuses on serving-timing correctness and the local profiling flow.

Highlights:

  • serve-profile now honors configured concurrency and stops dispatching new requests when the requested duration window closes.
  • client TTFT is now measured from the first streamed token chunk instead of HTTP first-byte timing.
  • local vLLM DEBUG log autodiscovery now follows the real live listener stdout/stderr files instead of stale .hotpath/video-server/* logs.
  • vLLM v1 parsing now merges exact request IDs from Added request ... lines with anonymous Running batch ... BatchDescriptor(...) lines.
  • vLLM v1 server queue, prefill, and decode timings are now refined from Prometheus when the raw DEBUG timestamps are only second-resolution, and the report explicitly discloses that.
  • serve-report no longer renders missing timing as fake 0.0 ms values.
  • the live serve-profile dashboard redraw path was hardened for Ghostty-style terminals so updates happen in place instead of stacking duplicate frames.
  • logs.md is now included in the repo with a detailed UTC log of this pass.

Focused verification run before release:

  • cmake --build build --parallel
  • ctest --test-dir build -R 'test_log_parser|test_serve_profiler|test_serve_report|test_traffic_replayer|test_cli|test_audit' --output-on-failure
  • live local smoke against Qwen/Qwen3.5-4B on localhost:8000