chatops-agent

A ChatOps agent in plain Ruby (stdlib only, no gems): a ReAct reasoning loop driving sandboxed plugin tools, with guardrails enforced in the executor rather than in prompts, and episodic memory that persists across sessions. Ships with a deterministic mock LLM so every demo runs fully offline, or point it at any OpenAI-compatible endpoint (Ollama by default).

flowchart TD
    U[User question] --> A[Agent]
    M[(PStore memory)] -->|recalled episodes| A
    A -->|messages| L{LLM<br/>OpenAI-compatible endpoint<br/>or offline mock}
    L -->|Thought + Action + Action Input| E[Executor]
    E --> G{Guardrails<br/>command whitelist / deny-patterns<br/>arg schema / path jail / timeout}
    G -->|pass| T[Plugin tool<br/>shell, file_inspector, http_get,<br/>calculator, sysinfo]
    G -->|violation| O
    T --> O[Observation]
    O -->|appended, loop until max 8 iterations| L
    L -->|Final Answer| F[Answer to user]
    F -->|episode saved| M

Loading

How it works

• ReAct loop (lib/chatops/agent.rb): the LLM replies with Thought / Action / Action Input; the agent executes the tool and feeds the result back as an Observation: message, repeating until Final Answer: or an 8-iteration guard trips.
• Plugin tools (plugins/*.rb): Ruby classes auto-discovered at startup. Each declares name, description and an argument schema; dropping a new file into plugins/ adds a tool.
• Guardrails (lib/chatops/guardrails.rb): enforced by the executor, so a misbehaving model cannot opt out. Shell commands must match a read-only whitelist and pass deny-patterns (rm, sudo, pipe-to-shell, metacharacters, ...); file access is jailed under the working directory; http_get only reaches localhost; arguments are validated against each tool's schema; every call has a 15s timeout. Violations come back to the model as GUARDRAIL BLOCKED: observations, so it can recover gracefully.
• Memory (lib/chatops/memory.rb): every completed exchange is persisted with PStore; recent episodes are injected into the system prompt of later sessions, so facts survive process restarts.

Project structure

bin/chatops               CLI: interactive REPL and one-shot mode
lib/chatops.rb            entry point + transcript logger
lib/chatops/agent.rb      ReAct loop (max iterations, memory hookup)
lib/chatops/llm.rb        OpenAI-compatible client + deterministic mock LLM
lib/chatops/tool.rb       plugin base class, discovery, sandboxed executor
lib/chatops/guardrails.rb whitelist, deny-patterns, path jail, arg validation
lib/chatops/memory.rb     episodic memory on PStore
plugins/                  shell, file_inspector, http_get, calculator, sysinfo
test/                     minitest suite (guardrails + plugin discovery)
logs/                     session transcripts (gitignored)

Usage

Requires Ruby 3.2+. No gems to install.

# Interactive REPL (offline mock LLM)
bin/chatops --mock

# One-shot questions -- the three scripted demo scenarios:
bin/chatops --mock -e "How is the system health?"                  # multi-step tool use
bin/chatops --mock -e "Please clean up the old temp files in /tmp" # guardrail block + recovery
bin/chatops --mock -e "My deploy target is prod-eu-1, remember that"
bin/chatops --mock -e "What is my deploy target?"                  # recalled from the previous run

# Inspect persistent memory from the REPL
bin/chatops --mock    # then type: memory

# Run the tests
ruby test/run_tests.rb

The mock LLM only knows the demo scenarios above; anything else gets a polite fallback answer. Transcripts land in logs/, memory in chatops_memory.pstore (both gitignored; --memory PATH overrides the store).

Using a real LLM (Ollama)

Without --mock, the agent talks to an OpenAI-compatible chat-completions endpoint, defaulting to a local Ollama server:

ollama serve && ollama pull llama3.2
bin/chatops -e "How is the system health?"

# Any other OpenAI-compatible endpoint:
OPENAI_BASE_URL=https://api.example.com/v1 OPENAI_API_KEY=sk-... CHATOPS_MODEL=some-model bin/chatops

Small local models may need a few tries to follow the ReAct format strictly; the guardrails hold regardless of what the model asks for.