[Feature] Add DRAM Connector for uc_connector

[harrisonyhq]

Prupose

Add a DRAM Connector to enable offloading the KV cache from GPU HBM to CPU DRAM, helping reduce GPU memory pressure and support larger models or batch sizes.

Modifications

• [Feat] Add ucm dram connector and test
• [Feat] Add some arguments in config for ucm_dram
• [Doc] Add doc for dram connector

Test

Unit Test

Passes unit test in test/test_ucm_dram.py

python test/test_ucm_dram.py

Performance Test

Using llmperf with following command:

python token_benchmark_ray.py  --model "/home/models/QwQ-32B" --mean-input-tokens 16000 --mean-output-tokens 1 --max-num-completed-requests 10 --num-concurrent-requests 1

to evaluate the connector TTFT performance with a series of input token lengths, using the model QwQ-32B, got the following result:

Token length	Local Disk Connector	vllm with disabled prefix cache	DRAM Connector
8K	0.88s	2.16s	0.65s
16K	1.79s	5.01s	1.26s
32K	3.51s	12.54s	2.21s