-
Notifications
You must be signed in to change notification settings - Fork 0
Memory System
This page explains OSAI's memory model for CPU-only AI workloads.
CPU-only AI is often memory-bound. Effective performance depends on placement, sharing, page-fault behavior, and avoiding duplicate model copies.
OSAI should use explicit arenas for:
- shared read-only model weights;
- private KV/cache state;
- source-code indexes;
- tokenizer state;
- network buffers;
- build output;
- logs and telemetry.
No swap should be used for AI memory.
Model arenas should use hugepages or large pages where supported. Hot arenas should be prefaulted before service READY.
Target invariant:
post-warmup page faults on hot AI paths = 0
On Intel Xeon, a latency-sensitive AI Cell should remain inside one NUMA node by default. Model weights should be NUMA-local or deliberately replicated.
On ARM/NVIDIA N1X-compatible systems, memory policy must account for LPDDR or unified memory behavior and cluster topology.
OSAI should refuse to start or scale an AI Cell when memory bandwidth pressure would violate existing service contracts.
This page defines the GitHub Wiki navigation sidebar.
- Architecture
- AI Cells
- CPU AI Runtime
- App Agents
- Memory System
- Networking
- Scheduler and Core Isolation
- Filesystem and Storage
- Driver Model
- Security Model
- Build System
- Build System
- Project Tracker
- Implementation Plan
- QEMU Full OS Core Workdown
- QEMU 100 Completion Plan
- Example Apps
- Codex Work Packages
- Testing and Benchmarking