Show & Tell: Inside the Compute Module — Fair-Share Scheduling, 6 Preemption Policies, and Multi-Provider Orchestration

[web3guru888]

The compute module is ASI:BUILD's infrastructure backbone — a Kenny AGI Compute Resource Pooling System that abstracts over local CPUs, Kubernetes clusters, and SLURM HPC environments behind a unified scheduling interface. At 11,698 LOC across 25 files, it's one of the more operationally complete modules in the framework.

---

What it does

The module solves a real problem: when you're running 29 cognitive modules simultaneously, you need something to decide who gets compute resources, when jobs get preempted, and how failures get recovered. The compute module handles all three.

Core components:

• ComputePoolManager — top-level orchestrator managing up to 1,000 concurrent jobs across providers (local, kubernetes, slurm)
• ResourceAllocator — bin-packing / best-fit allocation for CPU, GPU, memory, network, storage
• JobScheduler — priority queue with 10 priority levels and deadline-aware scheduling
• FairShareManager — four fair-share algorithms: proportional, lottery, weighted fair queuing (WFQ), and deficit round-robin (DRR)
• PreemptionManager — six preemption policies with live migration support

---

The fair-share scheduler

This is the most interesting piece. Rather than simple FIFO or priority, the module implements a decaying usage history for each agent account:

def effective_priority(self, current_time: float, decay_rate: float = 0.9) -> float:
    decayed_usage = 0.0
    for timestamp, usage in self.usage_history:
        age_hours = (current_time - timestamp) / 3600.0
        decay_factor = decay_rate ** age_hours
        decayed_usage += usage * decay_factor
    
    usage_ratio = (decayed_usage + self.current_usage) / self.shares
    return self.priority_boost / max(usage_ratio, 0.1)

Agents that have been hogging compute get deprioritized. Agents that haven't run recently get a natural priority boost. This is the same mechanism SLURM clusters use — it's good to see it replicated here for multi-agent cognitive systems.

---

Preemption: six policies

When resources are scarce, PreemptionManager decides which jobs to interrupt:

Policy	When it's appropriate
PRIORITY_BASED	Strict hierarchy — high-priority jobs always win
FAIR_SHARE	Account for historical usage; prevents starvation
DEADLINE_AWARE	Protect jobs with tight completion deadlines
RESOURCE_BASED	Preempt highest resource consumers first
COST_BASED	Minimize total economic cost of preemption
HYBRID	Combine all signals with tunable weights

Migration types include CHECKPOINT_RESTART, LIVE_MIGRATION, CONTAINER_MIGRATION, and PROCESS_MIGRATION — so a preempted job doesn't necessarily lose state.

---

Resource managers

Five resource-specific managers operate under the pool:

Manager	Capabilities
CPUPoolManager	Thread affinity, NUMA awareness, hyper-threading
GPUPoolManager	MIG slicing, multi-instance, PCIe topology
MemoryPoolManager	Huge pages, NUMA pinning, swap policies
NetworkManager	Bandwidth reservation, QoS, RDMA path selection
StoragePoolManager	Tiered storage (NVMe → SSD → HDD), distributed FS

---

Fault tolerance

CheckpointManager snapshots jobs at configurable intervals (default: 60s). RecoveryManager handles restart-from-checkpoint when nodes fail. This is important for the long-running cognitive modules (IIT Φ computation, VQE optimization) that can take minutes to complete.

---

Open question for the community

How should the compute module integrate with the Cognitive Blackboard?

Issue #70 tracks adding ComputeBlackboardAdapter, but the design question is interesting: should resource allocation decisions themselves be visible on the Blackboard? There are two options:

Option A — Observability only: Publish ComputeMetricsEntry (utilization, queue depth, job status) to the Blackboard as read-only telemetry. Modules can see compute pressure but can't directly control scheduling.

Option B — Bidirectional control: Modules publish ComputeRequestEntry items to the Blackboard; the adapter reads them and submits jobs to the pool. This makes the Blackboard the control plane for all compute dispatching.

Option B is more powerful but risks creating a scheduling bottleneck through the Blackboard's lock. Option A keeps concerns separated but means modules can't adapt to resource pressure.

Which design would you prefer to see implemented? Is there a third option I'm missing — perhaps an async priority queue that decouples Blackboard reads from actual scheduling decisions?