-
Notifications
You must be signed in to change notification settings - Fork 0
Cluster Inspection Commands
Relevant source files
The following files were used as context for generating this wiki page:
- .github/copilot-instructions.md
- cmd/axis/dashboard.go
- cmd/axis/dashboard_test.go
- cmd/axis/doctor.go
- cmd/axis/doctor_test.go
- cmd/axis/facts.go
- cmd/axis/mcp.go
- cmd/axis/mesh.go
- cmd/axis/mesh_test.go
- cmd/axis/status.go
- cmd/axis/status_test.go
- cmd/axis/summary_test.go
- cmd/axis/task_command_test.go
- cmd/axis/task_place_test.go
- cmd/axis/testdata/summary_daemon_cache.golden
- cmd/axis/testdata/summary_with_nodes.golden
The AXIS CLI provides a comprehensive read-only diagnostic surface for inspecting the state of individual nodes and the collective cluster. These commands leverage the Layer 1 (Fact Plane) and Layer 2 (Snapshot Plane) to provide deterministic visibility into hardware telemetry, software backends, and connectivity health.
Inspection commands primarily interact with two data sources:
- Live Discovery: Direct fan-out probing of nodes via SSH or local execution cmd/axis/status.go:54-62.
-
Daemon Cache: A high-performance snapshot stored by the
axis daemon, accessible via a Unix socket or TCP API cmd/axis/status.go:114-116.
| Command | Purpose | Primary Data Source |
|---|---|---|
axis facts |
Detailed telemetry for the local node | LocalCollector |
axis status |
Tabular overview of all cluster nodes | ClusterSnapshot |
axis summary |
Visual dashboard of cluster health/resources |
ClusterSnapshot + daemon.Metadata
|
axis doctor |
Connectivity and configuration validation | SSH Probes + API Health Checks |
axis task context |
Explains placement logic for a hypothetical task |
PlacementEngine + ClusterSnapshot
|
The facts command executes a local collection cycle to display granular hardware and software details about the current machine.
The command initializes a facts.NewLocalCollector cmd/axis/facts.go:17-19. It gathers:
- Identity: Stable IDs and hostname cmd/axis/facts.go:67-74.
- Resources: CPU models, RAM (total/free/reserved), disk space, and memory topology cmd/axis/facts.go:79-124.
- Accelerators: GPU models, VRAM capacity, and capabilities (e.g., Metal, CUDA) cmd/axis/facts.go:97-110.
- Backends: Detection of Ollama, llama.cpp, and MLX runtimes cmd/axis/facts.go:138-145.
Sources: cmd/axis/facts.go:1-172
The status command provides a cluster-wide view, aggregating NodeFacts from all configured members into a ClusterSnapshot.
The command uses collectStatusSnapshot to resolve which data source to use based on CLI flags:
-
--cached: Attempts to hit the daemon API; falls back to live discovery if the daemon is unreachable cmd/axis/status.go:114. -
--cached-only: Fails if the daemon is unreachable cmd/axis/status.go:115. -
Live (Default): Performs a full network fan-out to all nodes defined in
nodes.yaml.
"Code Entity Space" mapping for axis status:
graph TD
subgraph "CLI Entry"
A["axis status"] --> B["collectStatusSnapshot"]
end
subgraph "Data Sources"
B --> C["daemon.FetchSnapshot"]
B --> D["discoverLiveSnapshot"]
end
subgraph "Formatting"
C --> E["printStatusText"]
D --> E
E --> F["RenderNodeTable"]
E --> G["printResidentModelsSection"]
end
subgraph "Output"
F --> H["Stdout"]
G --> H
end
Sources: cmd/axis/status.go:40-110, cmd/axis/status_test.go:12-43
The status command includes a specialized section for active LLMs. It groups models by their runtime (e.g., ollama, mlx) and truncates long lists to maintain terminal readability cmd/axis/status.go:188-199.
Sources: cmd/axis/status.go:188-240, cmd/axis/status_test.go:110-245
The summary command provides a high-level visual dashboard. Unlike the tabular status, summary calculates aggregate cluster metrics.
- Resource Aggregates: Total vs. Allocatable RAM across the entire mesh cmd/axis/dashboard.go:117-120.
- Health Distribution: Count of Healthy, Degraded, and Unreachable nodes cmd/axis/dashboard.go:129-137.
- Cache Metadata: Displays the age of the snapshot and "Stale" markers if the daemon hasn't refreshed recently cmd/axis/dashboard.go:121-123.
The command uses renderBar to create ASCII progress bars for RAM utilization cmd/axis/dashboard.go:209-214.
Sources: cmd/axis/dashboard.go:22-146, cmd/axis/testdata/summary_with_nodes.golden:1-21
The doctor command is the primary tool for troubleshooting cluster connectivity and environment parity.
-
Config Validation: Checks for the existence and schema validity of
nodes.yamlcmd/axis/doctor.go:120-128. -
SSH Connectivity: Performs an actual handshake with every remote node using
transport.NewSSHExecutorto verify credentials and network paths cmd/axis/doctor.go:194-212. - Daemon Health: Probes the local AXIS API socket to ensure the background process is responsive cmd/axis/doctor.go:216-230.
-
Local Backend Probes: Executes discovery scripts (e.g.,
facts.OllamaDiscoveryScript) to verify if AI runtimes are not just installed, but actually running and responsive cmd/axis/doctor.go:43-100.
"Natural Language Space" to "Code Entity Space":
graph TD
subgraph "Check: Configuration"
Config["'Configuration File'"] --> Load["config.Load()"]
end
subgraph "Check: Connectivity"
SSH["'SSH: {node}'"] --> Exec["transport.NewSSHExecutor"]
Exec --> Conn["sshExec.Connect()"]
end
subgraph "Check: Daemon"
Daemon["'Local Daemon'"] --> Fetch["daemon.FetchSnapshot"]
end
subgraph "Check: Backends"
Backend["'Backend Probes'"] --> Probe["runBackendProbeScript"]
Probe --> Script["facts.LlamaServerDiscoveryScript"]
end
Sources: cmd/axis/doctor.go:115-230, cmd/axis/doctor_test.go:17-86
While technically part of the task management suite, axis task context serves as a read-only "explainable AI" surface for the placement engine.
It accepts a natural language task description and performs a "Dry Run" of the placement pipeline. It outputs:
- Inferred Requirements: What RAM/Tooling the system thinks the task needs.
- Placement Reasoning: A step-by-step log of why specific nodes were filtered out or ranked lower cmd/axis/task_command_test.go:12-44.
-
Snapshot Source: Indicates if the decision was based on
liveordaemon-cachedata cmd/axis/task_place_test.go:12-45.
Sources: cmd/axis/task_command_test.go:137-168, cmd/axis/task_place_test.go:47-79