Local and Remote Collectors

Local and Remote Collectors

Relevant source files

The following files were used as context for generating this wiki page:

• hack/apple-foundation-models.swift
• internal/execution/observations_test.go
• internal/execution/rusage_other.go
• internal/execution/rusage_unix.go
• internal/facts/apple_foundation_models_source.go
• internal/facts/collectors_test.go
• internal/facts/local.go
• internal/facts/network_test.go
• internal/facts/parsers_test.go
• internal/facts/pressure_topology.go
• internal/facts/remote.go
• internal/facts/resident_models_test.go
• internal/facts/storage_thermal_test.go
• internal/facts/tools.go
• internal/facts/turboquant.go
• internal/facts/turboquant_test.go
• internal/models/types.go
• internal/placement/empirical.go
• internal/placement/empirical_test.go
• internal/placement/warmth_test.go
• internal/turboexec/plan.go
• internal/turboexec/plan_test.go

The Fact Plane (Layer 1) relies on two primary entities to gather system telemetry: the LocalCollector and the RemoteCollector. These components perform deep inspection of hardware and software state to populate the NodeFacts data model. They handle the heterogeneity of the cluster—ranging from macOS workstations with Unified Memory to Linux servers with discrete NVIDIA GPUs—by using a combination of OS-native probes and specialized discovery scripts.

Collector Architecture and Data Flow

AXIS uses a unified collection interface. Whether a node is the local machine or a remote server accessed via SSH, the resulting NodeFacts structure is identical, ensuring the Placement Engine can treat the cluster as a homogenous pool of resources.

Collector Types and Interfaces

Entity	File	Responsibility
LocalCollector	internal/facts/local.go:22-25	Gathers facts from the machine currently running the AXIS binary using direct syscalls and local command execution.
RemoteCollector	internal/facts/remote.go:20-25	Connects to nodes over SSH via the transport.Executor interface and executes probes remotely.

Collection Data Flow

The following diagram illustrates how the collectors interact with the underlying system to produce structured facts.

Diagram: Fact Collection Pipeline

graph TD
    subgraph "Collector Space"
        LC["LocalCollector"]
        RC["RemoteCollector"]
    end

    subgraph "Probing Logic"
        OS["OS Probes (uname, sw_vers)"]
        RES["Resource Probes (vm_stat, /proc/meminfo)"]
        GPU["GPU Telemetry (nvidia-smi, ioreg)"]
        TOOLS["Tool Discovery (DiscoverTools)"]
        MODELS["Model Discovery (Ollama/MLX/Llama)"]
    end

    subgraph "Code Entities"
        NF["models.NodeFacts"]
        RI["models.Resources"]
        TI["models.ToolInfo"]
        RM["models.ResidentModel"]
    end

    LC --> OS
    LC --> RES
    RC --> OS
    RC --> RES
    
    OS --> NF
    RES --> RI
    RI --> NF
    TOOLS --> TI
    TI --> NF
    MODELS --> RM
    RM --> NF

Loading

Sources: internal/facts/local.go:52-143, internal/facts/remote.go:36-139, internal/models/types.go:56-107

---

Hardware and Resource Probing

The collectors perform multi-stage probing to determine the hardware capabilities of a node.

Memory Topology and Classification

AXIS distinguishes between standard and unified memory architectures. This is critical for LLM workloads where VRAM and System RAM may share the same physical pool.

• Unified Memory: Detected on Apple Silicon (darwin/arm64). AXIS assigns a MemoryClass (e.g., Class 4 for M3 Max) to help the ranker prioritize high-bandwidth SOCs internal/facts/parsers_test.go:212-228.
• Standard Memory: Detected on Linux/AMD64 systems where CPU and GPU memory are distinct internal/facts/parsers_test.go:221-227.

Storage Class Resolution

The system resolves the root disk's performance characteristics (nvme, ssd, or hdd).

• macOS: Uses diskutil info to parse the Protocol and Solid State fields internal/facts/storage_thermal_test.go:24-38.
• Linux: Performs a complex tree-walk using lsblk. It resolves LVM mappers and LUKS containers back to the physical "slaves" to determine the rotation (ROTA) status of the backing hardware internal/facts/storage_thermal_test.go:147-196.

GPU Telemetry

• NVIDIA: Probed via nvidia-smi to extract VRAM and model name.
• Apple: Probed via system_profiler SPDisplaysDataType and ioreg to determine Metal capabilities and integrated GPU specs internal/facts/remote.go:245-250.

Sources: internal/models/types.go:46-52, internal/facts/remote.go:164-165, internal/facts/storage_thermal_test.go:61-125

---

AI Tool and Resident Model Discovery

A key differentiator of AXIS is its ability to detect "warm" models—LLMs already loaded into memory.

The Discovery Scripts

The collectors execute embedded Bash scripts to discover the state of inference backends:

1. Ollama: Uses OllamaDiscoveryScript to check ollama ps for loaded models and lsof for port 11434 liveness internal/facts/tools.go:23-102.
2. Llama-server: Uses LlamaServerDiscoveryScript to pgrep for llama-server and parse the --model flag from the command line internal/facts/tools.go:111-142.
3. MLX: Uses MLXDiscoveryScript to query the mlx_lm.server API for resident models internal/facts/local.go:41-43.

Apple Foundation Models (AFM) Detection

On macOS 15+ (reported as Darwin 26+), AXIS detects the presence of Apple Foundation Models.

• Validation: It attempts to compile a small Swift helper (hack/apple-foundation-models.swift) using swiftc.
• Verification: The helper is executed to verify that the CoreML and LanguageModel frameworks can actually load the system models internal/facts/local.go:150-186.

Diagram: Resident Model Discovery Mapping

graph LR
    subgraph "Host Runtime"
        OP["Ollama Process"]
        LP["llama-server Process"]
        MP["mlx_lm.server"]
    end

    subgraph "Discovery Logic"
        ODS["OllamaDiscoveryScript"]
        LDS["LlamaServerDiscoveryScript"]
        MDS["MLXDiscoveryScript"]
    end

    subgraph "Fact Model"
        RM["models.ResidentModel"]
        WS["WarmthScore"]
    end

    OP --> ODS
    LP --> LDS
    MP --> MDS
    
    ODS --> RM
    LDS --> RM
    MDS --> RM
    RM --> WS

Loading

Sources: internal/facts/tools.go:21-142, internal/facts/local.go:126-140, internal/facts/resident_models_test.go:134-143

---

TurboQuant and Backend Probing

The TurboQuantInfo struct identifies nodes capable of high-performance quantized inference.

Inference vs. Verification

The discovery happens in two stages defined in internal/facts/turboquant.go:

1. Inference (inferTurboQuantSupport): Checks if tools like mlx_lm or llama-cli are present in the $PATH internal/facts/turboquant.go:13-65.
2. Verification (detectTurboQuantSupport): Executes the tool with --help to probe for specific capability flags like --flash-attn, --ctx-size, and --n-gpu-layers internal/facts/turboquant.go:69-83.

Capability Mapping

Capability	Detection Logic
long-context	Presence of mlx_lm or llama-server.
flash-attn-flag	Found flash-attn in tool help output.
gpu-layers-flag	Found n-gpu-layers in tool help output.
apple-silicon	OS is Darwin and Arch is ARM64.

Sources: internal/facts/turboquant.go:157-207, internal/facts/turboquant_test.go:55-78

---

Precision Failure Mapping

The collectors are designed to be "fault-tolerant" to ensure that a single failing probe doesn't invalidate the entire node.

• StatusComplete: All probes (OS, Resources, Tools, Models) succeeded internal/models/types.go:20.
• StatusPartial: The node was reachable, but some probes (e.g., nvidia-smi failed or df returned an error) failed. The node is still eligible for placement if requirements are met internal/models/types.go:21.
• StatusUnreachable: (Remote only) The SSH connection could not be established. No facts are collected internal/models/types.go:22.

Sources: internal/facts/remote.go:33-36, internal/facts/local.go:51-52, internal/models/types.go:12-24

---