[cite_start]TriLab is a systems research prototype exploring whether operating-system concepts—such as namespaces, snapshotting, process isolation, and context switching—can be applied to multi-tenant adapter serving for large language models[cite: 5, 59, 110, 111].
[cite_start]The goal is not to replace high-throughput inference engines like vLLM or SGLang[cite: 7, 63, 112]. [cite_start]Instead, TriLab investigates a complementary runtime abstraction strictly focused on adapter lifecycle management, fault isolation, and microsecond-scale state recovery[cite: 7, 113].
TriLab is built as a modular platform, separating the OS-level logic from the underlying hardware execution:
- [cite_start]TriRuntime: A prototype runtime implementing namespace-based adapter management via memory pointer manipulation[cite: 6, 115].
- [cite_start]TriBench: A reproducible benchmark suite for evaluating routing overhead, lifecycle operations, isolation, and recovery[cite: 6, 116].
triruntime/
├── core/ # OS Logic (Namespaces, Snapshots, Routing)
├── backends/ # Hardware translation (PyTorch, vLLM, TensorRT)
├── adapters/ # Model modifications (LoRA, Prefix)
└── api/ # Developer interface
Empirical Microkernel MeasurementsCurrent prototype evaluation has been performed across four transformer families using 10x statistical iterations: Qwen 2.5 (7B), Mistral (7B), Yi 1.5 (9B), and Llama 3.1 (8B). OS PrimitiveMetric EvaluatedStable PerformanceProvisioningNamespace Creation Latency~0.068 ms Context SwitchPointer Routing Latency (P50)~0.46 μs SnapshotState Rollback Latency~0.05 ms MetadataKernel Metadata Overhead392 - 448 KB / tenant
$git clone [https://github.com/irenee28/Trilab.git$](https://github.com/irenee28/Trilab.git$) cd Trilab
$pip install -r requirements.txt$ python tribench_os_stats.py