Skip to content

Conversation

@propanone1006
Copy link
Contributor

Purpose

This PR integrates support for Mooncake Store as a unified cache backend in unified-cache-management.

It introduces a new UcmMooncakeStore connector that wraps MooncakeDistributedStore, enabling seamless dump/load/lookup operations for KV cache tensors in vLLM and related systems.

This provides:

  • Improved extensibility for distributed cache offloading
  • Full async event loop support and task scheduling
  • Compatibility with safetensors-based serialization
  • A consistent interface aligned with UcmKVStoreBase

Modifications

This PR adds the following files:

  • unifiedcache/ucm_connector/ucm_mooncake.py: Mooncake connector implementation
  • test/test_mooncake.py: Unit tests for dump/load/lookup logic
  • test/mooncake.json: Example config file for testing

User-Facing Changes

✅ Yes. This PR introduces a new backend interface UcmMooncakeStore, which can now be selected for external KV cache operations.

No breaking changes to existing APIs.

Test

This patch was tested via:

  • ✅ Unit tests in test/test_mooncake.py:
    • test_lookup_not_found
    • test_lookup_found
    • test_dump_once
    • test_dump_repeated
    • test_load_existing_data
    • test_load_non_existent_data

@propanone1006 propanone1006 merged commit 385325b into develop Aug 13, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants