Safety Evaluation System

Safety Evaluation System

Relevant source files

The following files were used as context for generating this wiki page:

• docs/integrations/agent-hooks-and-mcp.md
• internal/execution/guarded.go
• internal/execution/guarded_test.go
• internal/safety/blocker.go
• internal/safety/blocker_test.go
• internal/safety/structured.go
• internal/safety/structured_bench_test.go
• internal/safety/structured_test.go

The Safety Evaluation System (Layer 4) is the primary gatekeeper for the AXIS execution engine. It provides a structured, multi-tier validation pipeline that evaluates commands against a set of risk categories, applies cluster-aware resource constraints, and produces a deterministic verdict (Allow, Deny, or Prompt). This system ensures that both human-initiated and agent-driven tasks adhere to safety boundaries before any local or remote execution occurs.

System Architecture

The safety system is composed of three primary components: the Evaluator, the RuleSet, and the Check function. It operates as a subordinate to observed state, meaning it prioritizes real-world cluster telemetry over static rules when evaluating resource-heavy operations.

Core Components

• Evaluator: The engine that parses raw command strings and matches them against active rules internal/safety/structured.go:17-18.
• RuleSet: A collection of Rule objects defining program patterns, argument globs, and their associated risk categories internal/safety/structured.go:68-71.
• Check: The high-level entry point that integrates the structured evaluator with live cluster knowledge (RAM/GPU availability) and historical failure data internal/safety/blocker.go:20-22.

Data Flow: Natural Language to Execution Verdict

The following diagram illustrates how a command string (Natural Language/CLI Space) is transformed into a Decision object (Code Space) through the safety pipeline.

Safety Evaluation Pipeline

graph TD
    subgraph "Natural Language / CLI Space"
        Input["'rm -rf /'"]
    end

    subgraph "Code Entity Space (internal/safety)"
        Parser["parseCommand()"]
        Eval["Evaluator.Evaluate()"]
        Rules["DefaultRuleSet()"]
        Blocker["safety.Check()"]
        DecisionObj["Decision {Verdict, Category, Reasons}"]
    end

    subgraph "Execution Plane (internal/execution)"
        Guarded["RunGuarded()"]
    end

    Input --> Parser
    Parser --> Eval
    Rules -.-> Eval
    Eval --> Blocker
    Blocker --> DecisionObj
    DecisionObj --> Guarded

Loading

Sources: internal/safety/structured.go:19-52, internal/safety/blocker.go:20-41, internal/execution/guarded.go:61-62

---

Risk Classification (Categories and Verdicts)

AXIS uses a standardized set of categories to classify the risk level of any given command. These categories determine the final Verdict.

Category	Description	Typical Verdict
CategorySafe	Purely informational (e.g., uname, date)	VerdictAllow
CategoryReadOnly	Reads state without mutation (e.g., ls, ps)	VerdictAllow
CategoryModify	Changes state but reversible (e.g., git add, make)	VerdictAllow / VerdictPrompt
CategoryNetworkMutating	External network calls (e.g., curl -X POST)	VerdictPrompt
CategoryDestructive	Permanent data loss (e.g., rm -rf, shred)	VerdictDeny
CategoryPrivilegeEscalate	Identity elevation (e.g., sudo, chmod 777)	VerdictDeny
CategorySystemCritical	Hardware/OS mutation (e.g., mkfs, fdisk)	VerdictDeny

Sources: internal/safety/structured.go:22-40, internal/safety/structured.go:77-118

---

Cluster-Aware Safety Checks

Beyond static command analysis, the system performs "Live" checks using ClusterKnowledge. This prevents execution on nodes that lack the physical resources to support the task, even if the command itself is considered "Safe".

RAM and GPU Gating

The Check function performs the following logic:

1. Historical Failure: If the exact command string is found in the failure history, it is blocked with a score of 92 internal/safety/blocker.go:25-27.
2. Heavy Model Heuristic: Commands containing "model", "inference", or "large" are blocked if the total cluster allocatable RAM is < 4096 MB internal/safety/blocker.go:51-59.
3. GPU Availability: Commands containing "gpu" are blocked if no nodes in the snapshot report available GPU hardware internal/safety/blocker.go:72-74.
4. Node-Specific Pressure: If the best-fit node has < 1024 MB of free RAM and the command is flagged as "large", the task is blocked internal/safety/blocker.go:61-70.

Sources: internal/safety/blocker.go:42-75, internal/safety/blocker_test.go:41-96

---

Implementation Detail: Evaluator.Evaluate

The Evaluator is the core engine that processes the DefaultRuleSet. It uses a priority-based matching system where higher priority rules are evaluated first.

Command Evaluation Logic

graph TD
    Start["Evaluate(rawCmd, surface)"] --> Parse["parseCommand(rawCmd)"]
    Parse --> Loop["Iterate Rules (Sorted by Priority)"]
    Loop --> MatchProg{"Program Match?"}
    MatchProg -- Yes --> MatchArgs{"Args/Raw Match?"}
    MatchProg -- No --> Loop
    MatchArgs -- Yes --> MatchSurface{"Surface Match?"}
    MatchArgs -- No --> Loop
    MatchSurface -- Yes --> SetVerdict["Set Decision.Verdict"]
    MatchSurface -- No --> Loop
    SetVerdict --> Return["Return Decision"]
    Loop -- "No Matches" --> Default["VerdictPrompt (Unknown)"]
    Default --> Return

Loading

Key Functions

• parseCommand: Extracts the base program name and arguments, handling environment variable prefixes (e.g., DEBUG=1 python3 ...) and stripping paths internal/safety/structured.go:153-155.
• globMatch: Performs pattern matching for both program names and argument strings internal/safety/structured.go:183-185.
• NewEvaluator: Constructs the evaluator and sorts rules by Priority in descending order to ensure the most specific/dangerous rules trigger first internal/safety/structured.go:132-138.

Sources: internal/safety/structured.go:121-150, internal/safety/structured_test.go:107-133

---

Integration with Guarded Execution

The safety system is integrated into the GuardedExecutionPipeline via RunGuarded. Before any resource reservation or placement occurs, the pipeline calls safety.Check.

1. Pre-flight: RunGuarded receives a GuardedExecutionRequest internal/execution/guarded.go:72-88.
2. Safety Gate: safety.Check is invoked. If Blocked is true, the execution terminates immediately with a GuardedExecutionResult containing the block reason internal/execution/guarded.go:103-104.
3. Verification: If the verdict is VerdictPrompt, the system requires the Confirm field to match ConfirmWord ("YES") internal/execution/guarded.go:40.

Sources: internal/execution/guarded.go:61-62, internal/safety/blocker.go:19-20, internal/execution/guarded_test.go:128-147

---