30 Pull requests merged by 9 people

• Bump google.golang.org/protobuf from 1.36.5 to 1.36.6

  #568 merged Mar 25, 2025

• Removing unsafe lib by switching to atomic.Pointer

  #567 merged Mar 25, 2025

• Allow partial metric updates

  #561 merged Mar 24, 2025

• Update boilerplate template

  #566 merged Mar 24, 2025

• Swapping out flow image

  #562 merged Mar 24, 2025

• remove controller-runtime dependency from API

  #565 merged Mar 24, 2025

• Allow bodyless requests to passthrough EPP

  #555 merged Mar 21, 2025

• Adding deprecation notice of BUFFERED mode on patch policy.

  #560 merged Mar 21, 2025

• Add makefile configs for bbr helm chart

  #553 merged Mar 21, 2025

• Initial helm chart for bbr

  #546 merged Mar 20, 2025

• Default to streaming mode

  #552 merged Mar 20, 2025

• Tag the main version of the helm chart with v0

  #547 merged Mar 20, 2025

• Add some more unit tests for BBR

  #545 merged Mar 20, 2025

• [Metrics] Handle vLLM streaming response in streaming server

  #518 merged Mar 20, 2025

• Bug fix: Initialize RequestReceivedTimestamp

  #539 merged Mar 20, 2025

• Simplify body streaming for BBR

  #544 merged Mar 20, 2025

• setting gotoolchain to auto

  #543 merged Mar 20, 2025

• Updated the image used for cloudbuild

  #542 merged Mar 20, 2025

• Add inferencepool chart push mechanics

  #540 merged Mar 19, 2025

• integration test stability improvements

  #541 merged Mar 19, 2025

• Simplifying EPP-side buffer

  #538 merged Mar 19, 2025

• Support full duplex streaming in body-based routing extension

  #463 merged Mar 19, 2025

• fixed rbac in helm chart

  #531 merged Mar 19, 2025

• Move benchmark under tools

  #534 merged Mar 19, 2025

• Bump golang.org/x/net from 0.35.0 to 0.36.0

  #529 merged Mar 19, 2025

• removed hf token from cpu based example

  #464 merged Mar 19, 2025

• bump vllm-cpu image to latest

  #530 merged Mar 19, 2025

• add helm template

  #416 merged Mar 19, 2025

• Add instructions to run benchmarks

  #480 merged Mar 18, 2025

• Docs: Uses tabs for quickstart model server options

  #527 merged Mar 18, 2025

8 Pull requests opened by 5 people

• [WIP] Groundwork to support OpenAI API endpoints that vLLM supports

  #526 opened Mar 18, 2025

• Document model server compatibility and config options

  #537 opened Mar 19, 2025

• Configure the vllm deployment with best practices for startup

  #550 opened Mar 20, 2025

• update benchmarking guide with latest results with vllm v1

  #559 opened Mar 21, 2025

• Add benchmark automation tool

  #563 opened Mar 24, 2025

• Bump github.com/onsi/gomega from 1.36.2 to 1.36.3

  #569 opened Mar 24, 2025

• Bump sigs.k8s.io/controller-runtime from 0.20.3 to 0.20.4

  #570 opened Mar 24, 2025

• Bump github.com/onsi/ginkgo/v2 from 2.23.0 to 2.23.3

  #571 opened Mar 24, 2025

6 Issues closed by 2 people

• Remove Controller-Runtime Dependencies from API Types

  #564 closed Mar 24, 2025

• Refactor the vllm specific code to become model server agnostic

  #383 closed Mar 23, 2025

• Flaky streaming integration tests

  #532 closed Mar 19, 2025

• move the benchmark folder under tools

  #533 closed Mar 19, 2025

• Add metrics & observability to body-based routing extension

  #439 closed Mar 19, 2025

• Add helm chart to simplify creating an InferencePool + EPP deployment

  #381 closed Mar 19, 2025

10 Issues opened by 4 people

• Improve vLLM upstream health checks to only pass when models are servable

  #558 opened Mar 21, 2025

• During hitless rollout testing, on average one early request to vLLM times out.

  #557 opened Mar 21, 2025

• Configure x-request-id support in the default ootb examples

  #556 opened Mar 21, 2025

• Improve metric capture on error

  #554 opened Mar 20, 2025

• Helm chart for BBR should also configuring ports

  #551 opened Mar 20, 2025

• We should encourage all InferencePool deployments to gracefully rollout and drain

  #549 opened Mar 20, 2025

• EPP should gracefully rollout and drain

  #548 opened Mar 20, 2025

• Add usage examples for BBR

  #536 opened Mar 19, 2025

• Remove k8s dependency from BBR

  #535 opened Mar 19, 2025

• Add a helm chart to parameterize the benchmark guide so users don't need to fork the repo

  #528 opened Mar 18, 2025

7 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Simplify InferencePool Ownership

  #489 commented on Mar 19, 2025 • 0 new comments

• Consider backend augmentation vs new backend type

  #521 commented on Mar 19, 2025 • 0 new comments

• Extension Auto-Provisioning

  #507 commented on Mar 20, 2025 • 0 new comments

• EPP protocol: proxy<->epp should use request/response body streaming

  #496 commented on Mar 20, 2025 • 0 new comments

• v0.3.0 Release Tracker

  #493 commented on Mar 23, 2025 • 0 new comments

• Prefix Cache Aware Proposal

  #498 commented on Mar 24, 2025 • 0 new comments

• Expose baseline algorithm parameters as configurable

  #16 commented on Mar 24, 2025 • 0 new comments