Fact Collection and Discovery

Fact Collection and Discovery

Relevant source files

The following files were used as context for generating this wiki page:

• LICENSE
• internal/facts/collector.go
• internal/facts/local.go
• internal/facts/parsers_test.go
• internal/facts/remote.go
• internal/models/types.go

The Fact Plane (Layer 1) is the foundation of the AXIS architecture. It is responsible for discovering nodes in the cluster and performing deep telemetry collection to understand the hardware and software capabilities of each member. This layer transforms raw machine states into structured NodeFacts, which the rest of the system uses for scheduling and execution decisions.

Overview of Layer 1

Fact collection is governed by the Collector interface internal/facts/collector.go:13-15, which abstracts the method of gathering telemetry. AXIS employs a "push-pull" hybrid approach: it can discover nodes via static configuration or network beacons and then pull facts using either local system calls or remote SSH execution.

AXIS prioritizes high-fidelity hardware data, including:

• Compute: CPU models, core counts, and GPU specifications (VRAM, capabilities like CUDA/Metal).
• Memory: Total vs. Free RAM, memory topology (Unified vs. Standard), and memory pressure levels.
• Software: Installed AI tools (Ollama, Llama.cpp), resident models currently in VRAM, and OS-specific features like Apple Foundation Models.

NodeFacts Data Model

The NodeFacts struct is the primary data entity produced by this layer. It captures a point-in-time snapshot of a node's resources and "epistemic state" (what the node knows it is currently doing).

For a detailed reference of the fields, including RAM metrics, GPU info, and thermal states, see [NodeFacts Data Model (#3.1)].

Code Entity Mapping: Telemetry Structs

graph TD
    subgraph "Code Entity Space"
        NF["struct NodeFacts"]
        RES["struct Resources"]
        GPU["struct GPUInfo"]
        RM["struct ResidentModel"]
    end

    subgraph "Natural Language Space"
        M1["Node Metadata"] --> NF
        M2["Hardware Telemetry"] --> RES
        M3["Accelerator Data"] --> GPU
        M4["In-Memory LLMs"] --> RM
    end

    NF --> RES
    RES --> GPU
    NF --> RM
    
    style NF stroke-dasharray: 5 5

Loading

Sources: internal/models/types.go:53-107, internal/models/types.go:143-150

Local and Remote Collectors

AXIS implements two primary collectors to handle different deployment scenarios:

1. LocalCollector: Used by the daemon or CLI to probe the machine it is currently running on internal/facts/local.go:22-25. It uses direct syscalls, runtime package inspections, and local CLI probes (e.g., nvidia-smi, system_profiler).
2. RemoteCollector: Used to gather facts from other nodes over the network internal/facts/remote.go:20-25. It leverages the transport.Executor to run commands via SSH and parse the output.

Both collectors perform "Tool Discovery" to identify available runtimes and "Resident Model Discovery" to see what LLMs are currently "warm" in memory.

For details on OS-specific probes and tool detection, see [Local and Remote Collectors (#3.2)].

Discovery Logic Flow

sequenceDiagram
    participant C as Collector (Local/Remote)
    participant OS as Operating System
    participant AI as AI Runtimes (Ollama/MLX)
    
    C->>OS: Execute "uname", "sysctl", "df"
    OS-->>C: Raw Hardware Stats
    C->>AI: Probe API/CLI (e.g., "ollama ps")
    AI-->>C: Resident Models & Version
    Note over C: Populate NodeFacts
    C->>C: PopulateMemoryMetrics()

Loading

Sources: internal/facts/local.go:52-143, internal/facts/remote.go:36-139

Node Discovery and Transport

Before facts can be collected, AXIS must know which nodes exist. This is handled through:

• Static Configuration: Parsing nodes.yaml for a list of known hosts.
• Dynamic Discovery: UDP beacons and mesh gossip (handled in the Snapshot Plane).
• Transport: The SSHExecutor manages the lifecycle of connections to remote nodes, handling host-key verification and multipath probing to find the fastest route to a node.

For details on the network discovery patterns and SSH lifecycle, see [Node Discovery and Transport (#3.3)].

Fact Collection Status

Collection is designed to be resilient. If a node is reachable but a specific probe (like GPU telemetry) fails, the node is marked with StatusPartial rather than being discarded internal/models/types.go:21.

Status	Meaning
complete	All facts collected successfully internal/models/types.go:20
partial	Node reachable, but some telemetry probes failed internal/models/types.go:21
unreachable	SSH or connection failure internal/models/types.go:22
error	Internal parsing or collector-specific failure internal/models/types.go:23

Sources: internal/models/types.go:12-24, internal/facts/remote.go:36-53

---