Troubleshooting

Troubleshooting: Diagnosing and Resolving Compitum Route Hiccups

Quick-Scan Infobox

Aspect	Details
Target User	Educators debugging class simulations, developers tracing certificate anomalies; CLI basics assumed.
Core Goal	Step-by-step diagnostics—from environment checks to trace-driven audits—for quick recoveries in bounded loops.
Time to First Fix	5 minutes: run --trace, grep for "feasible": false → loosen b by 10–20% or patch inputs.
Key Analogy	Trailhead checklist: pack the virtualenv, confirm paths, then step through signals—route resumes smoothly.
Compitum Tie	Traces emitted by trace.py; flags feed boundary.py, energy.py, and SRMF control logic.
Known Limitation	Traces redact prompts manually; Windows paths with spaces require quoting.

---

1) 60‑Second Sanity Check

• Activate venv: source .venv/bin/activate (Linux/macOS) or .\.venv\Scripts\Activate.ps1 (Windows PowerShell)
• Pin config paths: prefer absolute: --defaults /full/path/router.yaml
• Verify CLI in PATH: which compitum (or Get-Command compitum)
• Clear stale cache (if behavior seems “stuck”): rm -rf ~/.compitum/cache
• Single‑thread BLAS (first pass):
  • Linux/macOS: export OMP_NUM_THREADS=1 MKL_NUM_THREADS=1
  • Windows PowerShell: $env:OMP_NUM_THREADS='1'; $env:MKL_NUM_THREADS='1'

---

2) Triage by Symptom (choose one)

flowchart LR
  A{Symptom} -->|Empty/Crash| B[Environment]
  A -->|Weird Output| C[Certificate Analysis]
  A -->|Slow/Variable| D[Performance]
  B --> E[Venv / Paths / Deps]
  C --> F[Trace: Feasible / Boundary / Drift]
  D --> G[Threads / BLAS / Config]

Loading

Caption: Pick a branch based on the initial symptom.
Alt text: Flowchart branching from “Symptom” to Environment, Certificate, or Performance, then to concrete checks.

---

3) Minimal Repro: One‑Command Diagnosis

Save as diag.sh (Linux/macOS) or diag.ps1 (Windows). These scripts only read CLI output and a local trace file.

Bash (Linux/macOS)

#!/usr/bin/env bash
set -euo pipefail
PROMPT="Quadratic query about rectangles."
TRACE="${1:-diag_trace.json}"

compitum route --prompt "$PROMPT" --trace > "$TRACE"

echo "━━━━━━━━ DIAGNOSTIC REPORT ━━━━━━━━"
jq -r '.constraints.feasible        as $f
       | .utility                    as $u
       | .constraints.shadow_prices  as $sp
       | .boundary.is_boundary       as $b
       | .drift.drift_ema            as $d
       | "feasible=\($f)\nutility=\($u)\nmax_shadow=\((($sp//[])|max) // 0)\nboundary=\($b)\ndrift_ema=\($d)"' "$TRACE"

# Hints
if [[ "$(jq -r '.constraints.feasible' "$TRACE")" == "false" ]]; then
  echo "HINT: Infeasible constraints. Consider scaling b by 1.1–1.2 in constraints or fix inputs."
fi

if [[ "$(jq -r '.boundary.is_boundary' "$TRACE")" == "true" ]]; then
  echo "HINT: Boundary active. Raise gap δ (e.g., 0.1) or increase exploration temperature τ≈0.5."
fi

if jq -e '.. | tostring? | select(test("nan|inf"; "i"))' "$TRACE" >/dev/null; then
  echo "HINT: Found NaN/Inf. Check predictors and enable value clipping."
fi

PowerShell (Windows)

Param(
  [string]$Prompt = "Quadratic query about rectangles.",
  [string]$Trace = "diag_trace.json"
)

$cmd = "compitum route --prompt `"$Prompt`" --trace"
$stdout = Invoke-Expression $cmd
$stdout | Out-File -Encoding UTF8 $Trace

Write-Host "━━━━━━━━ DIAGNOSTIC REPORT ━━━━━━━━"
$feas = (Get-Content $Trace | ConvertFrom-Json).constraints.feasible
$util = (Get-Content $Trace | ConvertFrom-Json).utility
$shadow = ((Get-Content $Trace | ConvertFrom-Json).constraints.shadow_prices | Measure-Object -Maximum).Maximum
$bound = (Get-Content $Trace | ConvertFrom-Json).boundary.is_boundary
$drift = (Get-Content $Trace | ConvertFrom-Json).drift.drift_ema
"`nfeasible=$feas`nutility=$util`nmax_shadow=$shadow`nboundary=$bound`ndrift_ema=$drift" | Write-Host

# Hints
if ($feas -eq $false) { Write-Host "HINT: Infeasible constraints. Scale b by 1.1–1.2 or fix inputs." }
if ($bound -eq $true) { Write-Host "HINT: Boundary active. Raise gap δ≈0.1 or increase exploration τ≈0.5." }
if (Select-String -Path $Trace -Pattern 'nan|inf' -SimpleMatch) { Write-Host "HINT: Found NaN/Inf. Check predictors and value clipping." }

Privacy: remove sensitive text before sharing traces:

jq 'del(.prompt, .raw_tokens, .attachments)' diag_trace.json > redacted_trace.json

---

4) Trace Anatomy (what to read first)

Field (JSON path)	Meaning	Quick Action
.constraints.feasible	Whether (A x \le b) holds	If false: loosen or repair inputs; consider scaling b by 10–20% for a retry (temporary)
.constraints.shadow_prices[]	Shadow price per active constraint	Large values → relax/high‑impact candidates first
.boundary.is_boundary / .boundary.gap	Route is hugging a boundary; gap measures buffer	Raise gap δ (e.g., to 0.1); increase temperature τ to reduce over‑confidence
.drift.drift_ema	Exponential‑moving‑average drift	If > 0.1: smooth inputs, clip extremes, or lower step size
.utility	Route utility / free energy proxy	Combine with feasibility to check “good but illegal” vs “legal but poor”

---

5) Fast Fixes by Symptom

Symptom	Likely Cause	Fix (first)	Verify
Empty output / crash	Inactive venv, bad path, missing deps	Activate venv; pass absolute --defaults; reinstall pip install -e ".[dev]"	CLI runs; no import errors
Nonsensical certificate	Infeasible constraints or untrained predictions	Inspect trace; if infeasible, temper constraints or retrain; if NaN, add clipping	feasible=true, no NaN in trace
Highly variable runtime	BLAS thread thrash	Force single thread; later tune	Wall time stabilizes within 10–20%
Stalls (r_t → 0)	Noisy signals / drift	Reduce step size; enable EMA on ΔE; check .drift_ema	Drift < 0.1 and progress resumes
Overflagged boundary	δ/τ too strict	Raise δ to 0.1; set τ≈0.5	.boundary.is_boundary=false on typical runs

---

6) Performance Tuning (after correctness)

1. Threads: Start with OMP_NUM_THREADS=1 MKL_NUM_THREADS=1.
2. I/O: Prefer local datasets over network mounts for trace‑heavy sessions.
3. Metric math: Keep SPD rank modest (e.g., r=4) and regularization δ≥1e‑3 to maintain a well‑conditioned metric.
4. Trust control: Use adaptive radii—expand on good steps, shrink on poor; cap maximum radius to avoid overshoot.
5. Cache: Clear ~/.compitum/cache when configs change substantially.

---

7) Windows Notes

• Quote any path containing spaces: "C:\Program Files\..."
• Prefer PowerShell for UTF‑8 handling; ensure files are saved with UTF‑8 encoding.

---

8) Educator’s Corner (optional)

Use a trace as a teachable artifact: assign pairs to diagnose a redacted trace and propose parameter edits. Focus on reading shadow prices, drift, and boundary gap as evidence.

sequenceDiagram
  participant T as Teacher
  participant S as Students
  participant Tr as Trace JSON
  T->>S: Provide redacted trace
  S->>Tr: Inspect feasibility, shadows, drift
  S->>T: Propose edits (δ, τ, step size)
  T->>S: Rerun route and reflect

Loading

Caption: Classroom loop: inspect → hypothesize → adjust → verify.
Alt text: Sequence diagram showing teacher and students iterating on a trace.

---

9) FAQ

Q: Should I relax constraints or fix data?
A: Prefer fixing data. Temporary relaxations (e.g., scaling b) are for quick re‑tests only.

Q: When do I increase exploration?
A: If boundary is frequently active and utility remains low, raise δ or τ and monitor drift.

Q: How do I know the SPD metric is stable?
A: Keep the condition number under ~1e6 with sufficient regularization; refit low‑rank factors periodically.

---

10) Final Checklist (copy/paste)

• Venv active; CLI on PATH
• Absolute --defaults path
• Trace collected; redacted if needed
• feasible=true; no NaN/Inf
• Boundary not over‑triggering (reasonable gap)
• Drift EMA ≤ 0.1
• Single‑thread BLAS (baseline), then tune
• Metric well‑conditioned (regularization in place)
• Trust radius adjusted (no runaway expansion)

---