Roadmap follow-up issues for maintainer feedback

[Aamod007]

I reviewed the current roadmap against the codebase and opened a focused set of follow-up issues for the areas that look incomplete or that need maintainer direction before contributors start implementation.

The intent is not to force scope, but to make each roadmap item concrete enough for maintainers to approve, reshape, prioritize, or close as duplicate/out-of-scope.

Summary of proposed issues

Workloads

• workloads: Support additional inference engines beyond vLLM and SGLang #1209: Support additional inference engines beyond vLLM and SGLang

  • Current state: vLLM and SGLang already have API/runtime/router support.
  • Main question: should future engines be added as enum values, plugin-style adapters, or a runtime profile CRD?

• workloads: Add migration tooling for existing LeaderWorkerSet deployments #1210: Add migration tooling for existing LeaderWorkerSet deployments

  • Current state: LWS integration exists and converts LWS resources into ModelServing-managed workloads.
  • Main question: should migration support be docs-only first, a CLI command, or a validation/conversion tool?

Autoscaler

• autoscaler: Support multi-dimensional scaling across node, hyperNode, ServingGroup, and role levels #1211: Support multi-dimensional scaling across node, hyperNode, ServingGroup, and role levels

  • Current state: ServingGroup, Role, sub-targets, and heterogeneous targets exist.
  • Main question: what exactly should hyperNode mean in Kthena, and how should it compose with role/ServingGroup scaling?

• autoscaler: Generate autoscaling configuration from user SLA goals #1212: Generate autoscaling configuration from user SLA goals

  • Current state: policies are expressive, but users still map SLA goals to low-level thresholds manually.
  • Main question: should SLA generation live in the API, CLI, ModelBooster, or docs/tooling first?

• autoscaler: Research and implement dynamic PD ratio scaling #1213: Research and implement dynamic PD ratio scaling

  • Current state: PD-disaggregated routing and role modeling exist.
  • Main question: should dynamic PD ratio work start as a research proposal/benchmark, or directly in autoscaler logic?

Operations and performance

• observability: Define and export controller and autoscaler metrics #1214: Define and export controller and autoscaler metrics

  • Current state: router observability is strong; controller/autoscaler metrics need clearer coverage.
  • Main question: which metrics should be prioritized and what label conventions should be enforced?

• controller: Improve fault rapid-recovery with gang scheduling and network topology awareness #1215: Improve fault rapid-recovery with gang scheduling and network topology awareness

  • Current state: GangPolicy, NetworkTopology, PodGroup integration, and recovery policies exist.
  • Main question: should rapid recovery prioritize fastest replacement or strict topology preservation when both cannot be satisfied immediately?

• performance: Optimize fast start and stop paths for vLLM and SGLang workloads #1216: Optimize fast start and stop paths for vLLM and SGLang workloads

  • Current state: graceful shutdown and runtime support exist, but dedicated cold-start/fast-stop optimization is not clearly complete.
  • Main question: should the first step be measurement/docs or direct lifecycle implementation?

Existing issue not duplicated

I did not open a new session-affinity issue because #310 already tracks sticky/session routing work.

Maintainer questions

@hzxuzhonghu could you please review the issue set and share guidance on these points?

1. Do these issues correctly reflect the intended roadmap direction, or should any be closed as duplicate/out-of-scope?
2. Which 2-3 issues should contributors prioritize first?
3. Are there any architecture preferences we should follow before implementation, especially for engine extensibility, LWS migration tooling, and autoscaler dimensions?
4. Should any issue be merged/split before contributors pick it up?
5. Are there specific maintainers or area owners who should be tagged for workloads, router, autoscaler, or controller topics?

Once these points are clear, contributors can pick up the issues with better implementation boundaries.