2 Pull requests merged by 2 people

• Drop Entrypoint and Improve Containerfile

  #250 merged Jul 30, 2025

• Add CLI options for backend args (like headers and verify)

  #230 merged Jul 29, 2025

4 Pull requests opened by 2 people

• Add --version flag to guidellm CLI

  #240 opened Jul 24, 2025

• Add consistent synthetic data flag

  #241 opened Jul 25, 2025

• add custom dict camelize logic

  #246 opened Jul 28, 2025

• update ui readme

  #247 opened Jul 28, 2025

1 Issue closed by 1 person

• Requests Latency in (ms) --> (s)?

  #245 closed Jul 28, 2025

4 Issues opened by 2 people

• UI latency reported units as ms- should be sec

  #248 opened Jul 30, 2025

• Margin Of Error (MOE) stopping

  #244 opened Jul 27, 2025

• Margin Of Errors (MOE) in output report

  #243 opened Jul 27, 2025

• Over-Saturation stopping

  #242 opened Jul 27, 2025

46 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Improve error handling around invalid tokenizer names

  #205 commented on Jul 30, 2025 • 0 new comments

• Long running test spawn too many threads

  #196 commented on Jul 30, 2025 • 0 new comments

• Sweep mode with --max-seconds fails ungracefully if calculated RPS is too low

  #115 commented on Jul 30, 2025 • 0 new comments

• INFO log level is too verbose

  #97 commented on Jul 30, 2025 • 0 new comments

• [Feature Request] Need for ability to save partial results

  #227 commented on Jul 30, 2025 • 0 new comments

• [Feature Request] Record server config from /server_info endpoint when vLLM 0.9.0 lands

  #166 commented on Jul 30, 2025 • 0 new comments

• [Bug] Possibly incorrect output dump ? (The actual data appears to be fine)

  #114 commented on Jul 30, 2025 • 0 new comments

• [Feature Request] Testing with defined prefix lengths

  #104 commented on Jul 30, 2025 • 0 new comments

• Different prompts generated for different concurrency levels despite same seed/config

  #222 commented on Jul 30, 2025 • 0 new comments

• Bug: Negative TTFT values when benchmarking Ollama at high concurrency

  #216 commented on Jul 30, 2025 • 0 new comments

• Multi-turn benchmarking

  #199 commented on Jul 30, 2025 • 0 new comments

• Simplifying the readme towards expected use cases and examples; expand out docs particularly for configuration options

  #198 commented on Jul 30, 2025 • 0 new comments

• Improve performance of synthetic dataset prompt creation to match a given number of tokens

  #187 commented on Jul 30, 2025 • 0 new comments

• Ensure prefixes cannot be cached for synthetic datasets

  #186 commented on Jul 30, 2025 • 0 new comments

• Map out issues around potential max and convergence issues for rates

  #180 commented on Jul 30, 2025 • 0 new comments

• Add support for a Python vLLM backend

  #178 commented on Jul 30, 2025 • 0 new comments

• Standardized Sweep Numbers

  #156 commented on Jul 30, 2025 • 0 new comments

• Sweep Mode Improvements - get closer to approaching peak-throughput

  #155 commented on Jul 30, 2025 • 0 new comments

• Standardize dataset profiles for reasoning benchmarking

  #154 commented on Jul 30, 2025 • 0 new comments

• Support Audio to text multi-modal benchmarking in GuideLLM

  #153 commented on Jul 30, 2025 • 0 new comments

• Support VL Model Benchmarking

  #152 commented on Jul 30, 2025 • 0 new comments

• Enable Batch Inferencing Benchmarking Support

  #102 commented on Jul 30, 2025 • 0 new comments

• Automatic performance stabilizization - so no need to set max-requests or max-seconds

  #101 commented on Jul 30, 2025 • 0 new comments

• Land GuideLLM UI in Upstream Repository

  #98 commented on Jul 30, 2025 • 0 new comments

• Have GuideLLM kick off a vLLM server automatically to avoid having the user install vLLM and assign the target themselves

  #95 commented on Jul 30, 2025 • 0 new comments

• Integrate lm-eval into GuideLLM

  #90 commented on Jul 30, 2025 • 0 new comments

• Enable vLLM scheduler specific Metrics

  #86 commented on Jul 30, 2025 • 0 new comments

• [Scoping] Enable Hardware Accelerator Monitoring metrics to be captured in benchmarks

  #85 commented on Jul 30, 2025 • 0 new comments

• Community-oriented audio dataset profile creation

  #84 commented on Jul 30, 2025 • 0 new comments

• Scope out and propose Community-oriented text dataset profiles for model validation

  #83 commented on Jul 30, 2025 • 0 new comments

• Text-dataset performance profile creation - decide what the profiles are.

  #82 commented on Jul 30, 2025 • 0 new comments

• End-to-End Testing Enablement / Expansion

  #55 commented on Jul 30, 2025 • 0 new comments

• Integration Testing Enablement / Expansion

  #54 commented on Jul 30, 2025 • 0 new comments

• [FeatureReuqest] Auto-sweep with model saturation detection

  #108 commented on Jul 30, 2025 • 0 new comments

• [Feature Request] TTFT & ITL Timeouts

  #107 commented on Jul 30, 2025 • 0 new comments

• [Feature Request] Max error rate parameter

  #105 commented on Jul 30, 2025 • 0 new comments

• Add new benchmark mode to search for peak goodput under an SLO

  #197 commented on Jul 30, 2025 • 0 new comments

• Max RPS in sweep mode could be higher

  #93 commented on Jul 30, 2025 • 0 new comments

• Enable dynamic Prefix Handling

  #234 commented on Jul 30, 2025 • 0 new comments

• [Feature Request] Consider having groups of queries with multiple system prompts

  #232 commented on Jul 30, 2025 • 0 new comments

• Token Efficiency Benchmarking

  #179 commented on Jul 30, 2025 • 0 new comments

• WIP: Multiturn Benchmarking Support

  #211 commented on Jul 30, 2025 • 0 new comments

• Unify RPS and Concurrent Scheduler Paths

  #233 commented on Jul 30, 2025 • 0 new comments

• Add dataset-from-file command to extract datasets from benchmark reports

  #235 commented on Jul 26, 2025 • 0 new comments

• Feat/max error rate - continued

  #238 commented on Jul 28, 2025 • 0 new comments

• WIP: Various Small Tooling Fixes/Features

  #239 commented on Jul 30, 2025 • 0 new comments