Filter Rank and Select Pipeline

Filter, Rank, and Select Pipeline

Relevant source files

The following files were used as context for generating this wiki page:

• cmd/axis/testdata/task_place_turboquant_json.golden
• cmd/axis/turboquant_contract_test.go
• internal/agent/cortex_memory.go
• internal/agent/cortex_memory_test.go
• internal/models/types_test.go
• internal/placement/explain.go
• internal/placement/infer.go
• internal/placement/modelname.go
• internal/placement/modelname_test.go
• internal/placement/placement_test.go
• internal/placement/ranker.go
• internal/placement/selector.go
• internal/snapshot/snapshot.go
• internal/snapshot/snapshot_test.go
• internal/state/observations.go
• internal/state/observations_test.go

The Placement Engine (Layer 3) is responsible for the deterministic selection of a node from the cluster to execute a specific task. This process transforms high-level TaskRequirements into a concrete PlacementDecision through a three-stage pipeline: Filter, Rank, and Select.

Pipeline Architecture

The pipeline ensures that placement is not just a "best guess" but a verifiable decision based on hard constraints, empirical performance history, and cluster-wide resource availability.

Placement Pipeline Flow

graph TD
    subgraph "Natural Language Space"
        A["Task Description"]
    end

    subgraph "Code Entity Space: internal/placement"
        B["InferRequirements()"]
        C["SelectBestNode()"]
        D["FilterCandidates()"]
        E["RankCandidates()"]
        F["ComputeTaskFitScore()"]
    end

    A --> B
    B --> C
    C --> D
    D -- "Eligible Nodes" --> E
    E -- "Sorted Candidates" --> F
    F -- "PlacementDecision" --> G["Execution Engine"]
    
    style B stroke-dasharray: 5 5
    style C stroke-width: 4px

Loading

Sources: internal/placement/selector.go:11-27, internal/placement/infer.go:14-16

---

1. Filtering (Hard Constraints)

The FilterCandidates function identifies nodes that meet the absolute minimum requirements for a task. If a node fails any filter, it is excluded from further consideration.

• Status Check: Nodes must have a StatusComplete or be specifically allowed if incomplete based on tool availability internal/placement/ranker.go:34-37, internal/placement/explain.go:43-45.
• Resource Guards:
  • RAM Floor: Nodes must have RAMAllocatableMB greater than the task's memory request internal/placement/explain.go:84-93.
  • Runtime Pressure: Nodes are excluded if they exhibit critical memory pressure via Linux PSI (avg10 > 70.0) or Darwin VM pressure internal/placement/explain.go:47-59.
  • Environmental Guards: Nodes with critically low battery (< 20%) or thermal throttling are filtered out internal/placement/explain.go:61-71.
• Tool Requirements: Every tool specified in RequiredTools must be present in the node's ToolInfo list internal/placement/ranker.go:36, internal/placement/explain.go:111-114.
• Failure Memory: Nodes previously marked with blocking failure classes (e.g., FailureExecCrash, FailureThermal) for a specific workload are pruned internal/placement/ranker.go:47-53, internal/placement/explain.go:73-82.

Sources: internal/placement/ranker.go:32-45, internal/placement/explain.go:31-117

---

2. Ranking (10-Level Deterministic Sort)

Once a set of eligible candidates is found, RankCandidates performs a stable, deterministic sort. This ensures that two identical requests on the same cluster state always result in the same node selection.

Priority	Criteria	Logic
1	Allocatable RAM	Highest allocatableRAM(n) wins internal/placement/ranker.go:143-145.
2	Empirical History	Prefers nodes with fresh ExecutionObservation for the specific scope internal/placement/ranker.go:147-149.
3	Model Locality	Ranks nodes higher if the requested LLM model is already resident internal/placement/ranker.go:151-153.
4	Backend Preference	Favors nodes supporting preferred backends (e.g., MLX, CUDA) internal/placement/ranker.go:155-157.
5	GPU Score	Higher performance/count of GPUs internal/placement/ranker.go:159-161.
6	Effective Headroom	Highest (free RAM - penalty - requirement) internal/placement/ranker.go:163-165.
7	TurboQuant/Unified	Suitability for long-context or unified memory tasks internal/placement/ranker.go:167-175.
8	RAM Pressure	Tie-break favoring lower pressure (None < Low < Medium < High) internal/placement/ranker.go:177-179.
9	Reservation Ratio	Prefers nodes with lower current commitment relative to capacity internal/placement/ranker.go:185-191.
10	Alphabetical	Final stable tie-break using Node.Name internal/placement/ranker.go:193.

Sources: internal/placement/ranker.go:55-202

---

3. Selection and Fit Scoring

SelectBestNode picks the top-ranked node and generates a PlacementDecision. A key component of this decision is the Fit Score, a 0-100 metric computed by ComputeTaskFitScore.

Fit Score Components

The score is weighted based on the workload class and node capabilities:

• LLM Inference: Heavily weights Unified Memory (Apple Silicon) and TurboQuant support for long-context tasks internal/placement/selector.go:55-78.
• Resource Availability: Calculates the ratio of allocatable memory against total capacity internal/placement/selector.go:100-110.
• Locality Bonus: Provides a bonus if the node is the LocalNode, avoiding SSH overhead for latency-sensitive tasks internal/placement/selector.go:93-94.

Sources: internal/placement/selector.go:11-35, cmd/axis/turboquant_contract_test.go:113-143

---

4. Diagnostic Reasoning Pipeline

AXIS provides detailed explanations for why a node was chosen (or why a request failed). This is exposed via the axis placement explain command.

Reasoning Logic

The system populates PlacementDecision.Reasoning with human-readable strings explaining:

1. Workload Inference: Why a specific class (e.g., long-context-inference) was assigned internal/placement/selector.go:37-43.
2. Hardware Advantages: Detection of Unified Memory or specific GPU models internal/placement/selector.go:55-62.
3. Empirical Evidence: Citations of past execution times or peak RAM usage internal/placement/selector.go:79-82.
4. Runner-up Comparison: If multiple nodes were eligible, it explains why the winner was superior to the second-best candidate (e.g., "lower cluster reservation share favored: 10% vs runner-up 45%") internal/placement/selector.go:151-175.

Data Mapping: Facts to Reasoning

graph LR
    subgraph "NodeFacts (models)"
        F1["Resources.RAMFreeMB"]
        F2["TurboQuant.Supported"]
        F3["Ollama.Models"]
    end

    subgraph "PlacementDecision (models)"
        R1["Reasoning: '8192MB allocatable'"]
        R2["Reasoning: 'turboquant-aware backend'"]
        R3["Reasoning: 'resident model locality'"]
    end

    F1 --> R1
    F2 --> R2
    F3 --> R3

    style F1 stroke-width:2px
    style R1 stroke-dasharray: 3 3

Loading

Sources: internal/placement/selector.go:29-190, cmd/axis/testdata/task_place_turboquant_json.golden:15-28

---