Observability

Observability & Logging

CortexPrism ships with a structured logging and observability stack including Prometheus metrics, OpenTelemetry tracing, and Langfuse LLM observability. All components are opt-in and configurable.

Log Levels

Level	Description
trace	Extremely verbose — every round, chunk, span
debug	Internal state: turn start, tool calls, metadata
info	Operational events: server start, plugin load
warn	Recoverable problems: policy denials, retries
error	Failures that need attention
silent	No output whatsoever

Configuration

{
  "logging": {
    "level": "info",
    "fileEnabled": true,
    "fileMaxBytes": 10485760,
    "fileMaxFiles": 5,
    "otlp": {
      "endpoint": "http://localhost:4318"
    },
    "langfuse": {
      "publicKey": "pk-lf-...",
      "secretKey": "sk-lf-...",
      "baseUrl": "https://cloud.langfuse.com"
    }
  }
}

Or via environment variable:

CORTEX_LOG_LEVEL=debug cortex chat

Log file: ~/.cortex/data/logs/cortex.log (server log at ~/.cortex/data/server.log).

Settings persistence: PUT /api/config applies logging changes live without server restart.

CLI

cortex log show                          # Last 100 entries
cortex log show --lines=200 --level=warn # Filter by level
cortex log tail                          # Live tail (Ctrl+C to stop)
cortex log tail --level=debug            # Tail with filters
cortex log clear                         # Truncate log file
cortex log path                          # Print log file path
cortex log set-level info                # Update level in config
cortex log status                        # Show current config

Logger Registry

The logger system (src/utils/logger.ts) provides:

• Pluggable transports — console, file, OTLP
• Per-namespace log levels — granular control per subsystem
• Structured JSON output — machine-parseable log entries
• File transport — warning-level and above by default; all levels in verbose mode

Namespaces

Namespace	Subsystem
agent:loop	Main agent loop with debug tracing
server	HTTP server and router
server:ws	WebSocket handler
tools:executor	Tool call execution
trace	Distributed tracing spans
plugin:<name>	Per-plugin logger
langfuse	Langfuse integration
security:supervisor	LLM supervisor decisions
memory:heuristics	Heuristic learning cycles

Prometheus / Grafana

Metrics exposed at GET /metrics with 29+ metric families:

Metric Family	Type	Labels
cortex_agent_turns_total	Counter	provider, model
cortex_agent_tokens_total	Counter	provider, model, direction
cortex_agent_errors_total	Counter	provider, error_type
cortex_validator_intents_total	Counter	approved/rejected
cortex_executor_actions_total	Counter	tool
cortex_executor_duration_seconds	Histogram	tool
cortex_scheduler_jobs_total	Counter	status
cortex_memory_consolidations_total	Counter	tier
cortex_system_cpu_percent	Gauge	—
cortex_system_memory_bytes	Gauge	—
cortex_node_directives_*	Counter/Gauge	node_id, tier
cortex_qm_predictions_*	Counter/Gauge	mode

Scrape config:

scrape_configs:
  - job_name: cortex
    static_configs:
      - targets: ['localhost:3000']

OpenTelemetry (OTLP)

Trace/span export compatible with Grafana Tempo, Jaeger, and other OTLP receivers.

{
  "logging": {
    "otlp": {
      "endpoint": "http://localhost:4318",
      "headers": {
        "Authorization": "Bearer <token>"
      }
    }
  }
}

Connection test: POST /api/observability/test-otlp pings the configured OTLP endpoint and reports status.

Langfuse (LLM Observability)

Each agent turn creates a Langfuse trace with:

• Trace per agent turn
• Generation span per LLM round with token usage and cost metrics
• Span per tool call with input/output capture

{
  "logging": {
    "langfuse": {
      "publicKey": "pk-lf-...",
      "secretKey": "sk-lf-...",
      "baseUrl": "https://cloud.langfuse.com"
    }
  }
}

Connection test: POST /api/observability/test-langfuse authenticates and pings the Langfuse API health endpoint.

Web UI

The Observability tab in Settings provides:

• Log level selector (debug/trace/info/warn/error)
• File logging toggle and rotation settings
• OTLP endpoint configuration with test button
• Grafana dashboard link
• Langfuse public/secret key configuration with test button

See Also

• Configuration — Full logging config reference
• Security — Cortex Lens audit log
• REST API — Observability API endpoints