Am the only one hitting bug after bug after bug? #941
Unanswered
dragetd
asked this question in
User Support
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Providing an api key breaks the csv renderer, the provided podman/docker command requires user permissions that shoudl be set via -e user=guidellm, the HTML output is ALWAYS broken, the console-table is lacking width and showing almost all info with an ellipsis...
And I sort of expected it to benchmark ALL models after it queries 'models', but instead if only benchmarks the first.
I do not really see the point of this utility.
I wanted a simple time-to-first-token, tps, prefix benchmark for all my models behind a local endpoint.
guidlellm is probably the completely wrong tool for this...?
All reactions