Update metrics command

[amrit110]

PR Type

Feature/Fix

Short Description

• Fix the metrics command to use the metrics endpoint which is supported by vllm

When the server is not yet ready and if the command is used:

If server is available, then it looks like:

Note that the prompt and throughput rates are calculated by us and not provided by the metrics endpoint.

Computed metrics:

1. Prompt throughput (tokens/s)
2. Generation throughput (tokens/s)
3. Avg. Request latency (s)
4. Total prompt tokens (this is a sum from all requests)
5. Total generation tokens (this is a sum from all requests)
6. Successful requests

Note that the throughput is calculated over every second when the fetch_metrics() method is called while refreshing the TUI. It then displays the last computed value of throughout instead of reverting to 0.0 even when no requests are being served. If new requests get served, then the throughput is re-computed and updated.

Tests Added

...

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 77.27273% with 25 lines in your changes missing coverage. Please review.

Project coverage is 79.50%. Comparing base (ecb6d88) to head (a1c07d3).
Report is 24 commits behind head on develop.

Files with missing lines	Patch %	Lines
vec_inf/cli/_helper.py	73.95%	25 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop      #53      +/-   ##
===========================================
+ Coverage    74.83%   79.50%   +4.67%     
===========================================
  Files            4        4              
  Lines          445      522      +77     
===========================================
+ Hits           333      415      +82     
+ Misses         112      107       -5     

Files with missing lines	Coverage Δ
vec_inf/cli/_cli.py	82.41% <100.00%> (+19.91%)	⬆️
vec_inf/cli/_utils.py	86.51% <ø> (+0.10%)	⬆️
vec_inf/cli/_helper.py	74.84% <73.95%> (+3.28%)	⬆️

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.