Skip to content

[Feature]: AutoDeploy: Integrate with CppMambaHybridCacheManager #14320

Open
Open
[Feature]: AutoDeploy: Integrate with CppMambaHybridCacheManager#14320
Assignees
govind-ramnarayan
Labels
AutoDeploy<NV> AutoDeploy BackendDisaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.feature requestNew feature or request. This includes new model, dtype, functionality support
@govind-ramnarayan

Description

@govind-ramnarayan
govind-ramnarayan
opened on May 19, 2026

🚀 The feature, motivation and pitch

Currently disaggregated serving with Mamba models is only supported through the CppMambaHybridCacheManager. So to support them in AutoDeploy, we need to integrate with this cache manager.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Metadata

Metadata

Assignees

Labels

AutoDeploy<NV> AutoDeploy BackendDisaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.feature requestNew feature or request. This includes new model, dtype, functionality support

Type

No type

Fields

No fields configured for issues without a type.

Projects

AutoDeploy Board

Status

Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions