Skip to content

Repository files navigation

OMDS — oh-my-datascience

One Python package (omds) that does two things over the same cores:

  1. A data-science guardrail toolkit for coding agents. A set of omds-* CLIs plus portable Agent Skills that let any coding agent check Python for train/test leakage and metric misuse, trace data lineage through a code graph, and keep a durable experiment ledger and episodic memory. Optionally reachable as an MCP server or as a Claude Code plugin.

  2. E-GDS — an offline evolutionary optimizer for ML pipelines. The omds CLI (init/evolve/promote/inspect/models) evolves a pipeline workspace's own code with GEPA (Genetic-Pareto reflective evolution), targets mutations using a code graph, and admits a variant only through five blocking gates ending in a real, sandboxed training run on holdout data.

These are not two codebases. Both halves stand on the same ontological grounding layer (omds.ontology — probability-simplex bounds, proper scoring rules, leakage rules, sparsemax caveats) and the same code graph layer (omds.codegraph — AST dependencies + data lineage). The axiom registry that blocks a GEPA promotion is literally the registry omds-guardrails check runs against the file your agent just edited: both call omds.ontology.static_check.run with omds.ontology.axioms.default_registry.

                omds.ontology                    omds.codegraph
        (axiom registry, static AST rules,   (AST deps, def-use chains,
         runtime assertion injection)         data lineage, git blame)
                     │                                  │
       ┌─────────────┴──────────────┬───────────────────┴──────────────┐
       │                            │                                  │
 omds-* CLIs + skills/        omds-mcp (9 tools)              E-GDS: omds evolve
                              Claude Code plugin               → GEPA → 5 gates
   PRIMARY surface              OPTIONAL adapters               → internal PR
   (agent-neutral)          (same cores, no new logic)      (offline optimizer)

Everything is model-agnostic and SLM-first: the LLM layer speaks only OpenAI-compatible /v1/chat/completions, so it runs against local small models (Qwen2.5-Coder and friends via Ollama / llama.cpp / vLLM / LM Studio) — the design floor every prompt must work within — and swapping in a frontier API model is a config change. The guardrail toolkit needs no model at all.

Status

Both halves are shipped and green. uv run pytest reports 705 passed, 6 skipped (all six skips are opt-in tiers — see Testing — plus 2 pre-existing sklearn UndefinedMetricWarnings). The nine MCP tools, the omds-* CLIs, the seven skills, the plugin bundle, the GEPA evolution engine, the omds-slm local-SLM harness, and the benchmarks/tabular_multiclass/ benchmark (which passes all five promotion gates as its own seed) are all covered. The one thing no test in the default suite exercises is a live model: that path is smoke-tested behind an opt-in gate (see Local SLM support (≤7B) under Shared).

One capability was deliberately removed, not deferred. v1 shipped an omds run command driving a multi-agent runtime loop (AutoGen GraphFlow step agents, task decomposition, a repair loop). That whole stack was deleted in the v2 harness rework — there is no omds run today, and the autogen-* dependencies are gone. What survives of omds.runtime is the model-agnostic, network-free part the evolution gates reuse: task loading, the skills catalog, and validate_workspace. Traces are now written by the evolution evaluator (one per candidate evaluation), not by an agent loop, and omds inspect traces reads those. Rationale: docs/adr/2026-07-18-mcp-pivot.md.

Contents

New here? PrerequisitesInstallConfirm it worksWire it into your agent. That is the whole setup, and it needs no model and no network.

Part 1 — agent toolkit Part 2 — E-GDS optimizer Shared
Getting started Quickstart Local SLM support (≤7B)
Skills How it works SLM-first design rules
CLIs The promotion gates Repository layout
MCP server Config reference Testing
In-kernel leakage guard Testing without a live model Verifying a real install
Troubleshooting Non-goals · License · History

Part 1 — the agent-facing toolkit

Getting started

Prerequisites

  • Python 3.12 or newer (requires-python = ">=3.12"). Nothing older works. 3.12, 3.13 and 3.14 are each verified by a real install-and-run, not just declared.
  • gitomds-codegraph blame shells out to it, and E-GDS clones workspaces with git worktree. The rest of the toolkit runs fine without a repo.
  • uv — the recommended installer (see Step 1), and required for a checkout or for Part 2. Plain pip works for the toolkit if you would rather not add another tool.
  • No model, no API key, no network. Everything in Part 1 is static analysis and local files. A model is needed only for Part 2 (E-GDS) and omds-slm.

Step 1: install the Python package (required for every route)

OMDS is installed from this repository, not from PyPI — pip install omds will fail with "No matching distribution found". Pick the route that matches what you want to do; they are not exclusive, and the third is a superset.

(a) Just the agent toolkit — uv tool install. OMDS is mostly ten command line tools, which is exactly what uv tool is for: one command, all ten on your PATH, in an isolated environment, with no venv to create or activate.

uv tool install --python 3.12 \
    "git+https://github.com/spkc83/omds.git#subdirectory=python/omds"

Pass --python 3.12. It is not optional padding. On uv 0.5.x, omitting it can build the tool environment on Python 3.11 despite the package requiring >=3.12, and then install 3.12-built binaries into it — omds-guardrails fails with a numpy C-extension ImportError that looks like a broken numpy rather than a wrong interpreter. Newer uv handles this; pinning costs nothing and works on both.

(b) Into your own project environment — when you want the in-kernel half. Route (a) is deliberately isolated, so omds is not importable from your own Python: from omds import kernel_guard raises ModuleNotFoundError even though the CLIs work. The in-kernel leakage guard and the omds.dstools helpers the visualization skill uses run inside your kernel, so they need the package installed alongside your own code:

uv pip install "git+https://github.com/spkc83/omds.git#subdirectory=python/omds"
# plain pip works identically:
pip install "git+https://github.com/spkc83/omds.git#subdirectory=python/omds"

(c) From a checkout — required for Part 2. omds init copies benchmark files that are not shipped inside the wheel, so E-GDS needs the repo itself:

git clone https://github.com/spkc83/omds.git
cd omds
uv sync                          # installs `omds` editable + the test toolchain
# or, without the uv workspace:  uv pip install -e python/omds

Every route gives you the ten console scripts (omds, omds-guardrails, omds-codegraph, omds-dstools, omds-memory-store, omds-sessionpad, omds-skill, omds-mcp, omds-install, omds-slm) and the seven skill files bundled inside the package. All three wiring routes below assume this step — in particular a Claude Code plugin cannot install a Python package, so it is never optional.

Step 2: confirm it works (about a minute)

Worth doing before wiring anything into an agent — it separates "the package is broken" from "my agent is not calling it". Create a file with a textbook leak in it:

mkdir /tmp/omds-demo && cd /tmp/omds-demo
cat > pipeline.py <<'EOF'
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler


def prep(X, y):
    scaler = StandardScaler()
    scaler.fit(X)                                   # fit on everything...
    X_train, X_test, y_train, y_test = train_test_split(X, y)   # ...then split
    return X_train, X_test, y_train, y_test
EOF

omds-guardrails check --files pipeline.py

That prints one violation — the scaler saw the test rows before the split:

{"violations":[{"axiom":"ax:fit_on_train_only","severity":"certain","file":"pipeline.py","line":7,
  "message":"pipeline.py:7: scaler/encoder fit on full data before split - fit() argument 'X' is also the full dataset passed to train_test_split in this function",
  "needs_runtime_proof":false}],"summary":{"certain":1,"warn":0}}

If you see {"violations":[],"summary":{"certain":0,"warn":0}}, the package works but the pattern was not matched — note that the fit-then-split sequence has to sit inside a function to be detected. A command not found means the install did not put the scripts on your PATH; see Troubleshooting.

Two more, to see the rest of the surface:

omds-codegraph lineage --symbol prep --root .
# {"lineage":[{"from":"pipeline.py::<module>","to":"pipeline.py::prep","kind":"defines"}]}

omds-skill lint skills/leakage-audit        # only from a checkout
# {"ok":true,"checks":[{"check":"format","passed":true,...}, ...]}

Step 3: wire it into a coding agent — pick one

All three deliver the same thing.

(a) omds-install — the recommended route; works from the installed package alone, because the skill files ship inside the wheel:

cd /path/to/your/project
omds-install claude                    # --project defaults to the current directory
Merged omds MCP server into .mcp.json
Copied skills into .claude/skills

It writes exactly two things, and nothing else:

// .mcp.json — merged in, never overwritten
{"mcpServers": {"omds": {"type": "stdio", "command": "omds-mcp"}}}
.claude/skills/{eda,evaluation,feature-engineering,leakage-audit,
                model-training,reporting,visualization}/

For any other agent, name the two paths yourself:

omds-install here --config <mcp-config.json> --skills-dir <that agent's skills dir>

claude is literally here with .mcp.json and .claude/skills pre-filled; there is no per-agent behavior, only per-agent paths. The merge into an existing MCP config is non-destructive and idempotent — rerunning it is safe, and it will not disturb MCP servers you already have configured.

(b) By hand — nothing above is magic. The skills are just files and the MCP entry is one line, so from a checkout:

cp -r skills/* .claude/skills/                       # the primary surface
claude mcp add --scope project omds -- omds-mcp      # the optional MCP adapter

(c) The Claude Code plugin — this repo is itself a plugin (.claude-plugin/plugin.json plus a single-plugin marketplace), packaging the same skills/, the same omds-mcp server, and a guardrail hook:

/plugin marketplace add spkc83/omds
/plugin install omds@omds

To try a local checkout without going through the marketplace, launch Claude Code with claude --plugin-dir . from the repo root.

The plugin route additionally installs a PostToolUse hook that runs omds-guardrails check on every Python file Claude writes or edits, blocking the turn on an axiom-certain leakage violation. Skills are namespaced under it as omds:<skill-name> (e.g. omds:leakage-audit). The hook is stdlib-only and fails open in every abnormal state — including omds-guardrails not being on PATH — so an uninstalled package makes it silently do nothing rather than break your editing.

Step 4: confirm the agent can see it

In a Claude Code session, in the project you just wired up:

Check Expect
/mcp omds listed and connected. (Route (a)/(b) prompt for approval on first use; the plugin route does not.)
ask it to list skills leakage-audit and six others — namespaced omds:leakage-audit under the plugin route
"check pipeline.py for data leakage" it runs guardrails_check / omds-guardrails check and reports ax:fit_on_train_only

If /mcp shows omds as failed, run omds-mcp in your shell: it should start and wait silently for stdio input (Ctrl-C to exit). A traceback instead means the agent is reporting a real install problem — see Troubleshooting.

For the whole install checked mechanically in one command, see Verifying a real install.

Skills — the primary surface

skills/ holds seven agent-neutral SKILL.md files — eda, feature-engineering, model-training, evaluation, visualization, reporting, leakage-audit. Nothing in them is tied to a particular agent; any tool that reads the SKILL.md format (Claude Code, Codex, Cursor, Gemini CLI, Copilot, …) can use them. Each teaches one workflow in terms of the real omds-* CLIs, so installing the package is enough to make every step actionable. See skills/README.md for the format and for adding one.

omds-skill lint is the contract, executable. Writing a skill by hand is easy to get subtly wrong, and the expensive mistake is a skill that tells an agent to run a command that does not exist:

omds-skill lint skills/leakage-audit        # a directory or its SKILL.md
# {"ok":true,"checks":[{"check":"format","passed":true,...}, ...]}

Six checks run — format, description_shape, no_coupling_tokens, surface_references, size, distinctness — and all of them are reported even after one fails, because an author fixing a skill wants the whole list in one pass. The one that earns its keep is surface_references: it resolves every omds-* command, MCP tool and omds.* module path a skill mentions against the live registries (the installed console-script entry points, the Typer apps behind them, and the MCP tool manager) rather than a hardcoded list — a hardcoded list being exactly the drift it exists to catch. It found real drift the first time it ran: eda's own description cited omds_dstools.df_summary, a module name that has not existed since the monopackage merge.

test_skills_portable.py calls the same lint_skill core, so the command you run by hand and the assertion the suite makes are the same code, not two descriptions of it. Distinctness is checked against the shipped set because skills/README.md's warning is literal: a vague or duplicated description degrades routing for every other skill, not just the new one.

CLIs — the primary surface

Every command exits 0 always: the decision lives in the JSON payload, never in the exit status. Stdout is nothing but JSON; logs go to stderr.

check analyzes the files you name, but builds its code graph over their common parent directory — so it prunes virtualenvs (by name and by pyvenv.cfg), caches and node_modules rather than walking them, and a file anywhere in the tree that cannot be parsed is skipped rather than fatal. A file you asked about that could not be read is different: it is named in an unreadable key, present only when non-empty, because a guardrail must never answer "clean" for a file it never managed to read.

# static leakage / scoring / shape checks over on-disk files…
omds-guardrails check --files pipeline.py
# {"violations":[{"axiom":"ax:fit_on_train_only","severity":"warn",...}],"summary":{"certain":0,"warn":1}}

# …or over content an agent is ABOUT to write (a JSON [{"path","content"},…] manifest)
omds-guardrails check --proposed proposed.json

# code graph: edges incident to a symbol, git blame + enclosing symbol, bounded file summary
omds-codegraph lineage --symbol prep --root .
omds-codegraph blame --file pipeline.py --line 42
omds-codegraph context --files pipeline.py --detail summary --max-items 50

# experiment ledger and episodic memory (both honor $OMDS_STATE_DIR)
omds-dstools ledger show --session demo
omds-memory-store add --session demo --kind note --text "baseline logreg log_loss 0.41"
omds-memory-store recall --session demo --query baseline --k 3

# the session working pad — plan / decisions / problems / log, as one markdown file
omds-sessionpad append --session demo --section plan --text "try target encoding next"
omds-sessionpad show --session demo --max-chars 2000

MCP server — an optional adapter

omds-mcp (official mcp SDK, FastMCP, stdio) exposes exactly nine tools. Each one calls the same core function its CLI calls — guardrails_check and omds-guardrails check both land in run_static_check — so the tool result is byte-identical to the CLI's --json stdout and there is no second copy of any logic. That DRY property is why keeping three surfaces costs almost nothing.

The server is not required. It exists for hosts that sandbox shell access or prefer typed tool calls. The mapping (identical to skills/README.md):

CLI (primary) Equivalent MCP tool
omds-guardrails check guardrails_check
omds-codegraph lineage codegraph_lineage
omds-codegraph blame codegraph_blame
omds-codegraph context codegraph_context
omds-dstools ledger show dstools_ledger_show
omds-memory-store add memory_add
omds-memory-store recall memory_recall
omds-sessionpad append sessionpad_append
omds-sessionpad show sessionpad_show

Domain outcomes are normal results, never protocol errors: a leakage violation, an unresolved symbol, a file outside a git repo, an empty ledger, or a session with no memory DB all return a payload. Only genuine faults (an unreadable path, unparseable input) surface as tool errors. codegraph_context takes detail ("summary" default / "full") and max_items so a small-window model is never blown up by an unbounded dump — counts still reports the true totals and truncated says whether anything was elided.

The visualization skill additionally uses the in-kernel omds.dstools helpers (df_summary, capture_plot, ledger), which have no CLI or MCP equivalent because they run inside your own Jupyter/IPython kernel. omds.dstools.install() publishes them onto builtins for later cells.

In-kernel leakage guard — opt-in

The static checks above read code. The other half is a runtime guard that fires only on data that actually reaches fit():

from omds import kernel_guard
kernel_guard.install()

X_test = kernel_guard.tag_test(X_test)
StandardScaler().fit(X_test)      # raises LeakageViolation
StandardScaler().fit(X_train)     # untagged data is forwarded verbatim

It is a tagged-object check, not a taint-tracking system: install() wraps every sklearn estimator's fit, is idempotent, and fails open to a no-op if sklearn/numpy cannot be imported. The violation message embeds a delimited JSON payload (parse_violation_payload recovers it) so an out-of-process reader — an agent reading a cell's error output, a log scraper — gets structure out of what a kernel serializes as plain traceback text.

To arm it for every kernel in an environment without a bootstrap cell:

omds-guardrails install-kernel-hook      # writes omds_autostart.pth into site-packages
omds-guardrails kernel-hook-status
omds-guardrails uninstall-kernel-hook

Read the warning before running that. A .pth executes at interpreter startup for every Python process in that environment — scripts, CI jobs, python -c, all of it — not just kernels. It is written to make that nearly free (it imports only sys at startup, and the guard itself installs only inside an ipykernel process), but it is still an environment-wide change. Prefer a venv you own over a shared or system interpreter. Installing the package deliberately does not arm it: shipping the .pth in the wheel would make that decision on your behalf, so it is a separate, explicit, reversible opt-in.

Troubleshooting

pip install omds → "No matching distribution found" Expected — OMDS is installed from the repository, not PyPI. Use one of the three routes in Step 1.

ModuleNotFoundError: No module named 'omds' in your own Python or notebook, even though the omds-* commands work You installed with uv tool install, which is isolated by design — it puts the CLIs on PATH without putting the package on your sys.path. That is correct for the CLIs and wrong for the in-kernel guard and omds.dstools. Install it into the environment your kernel runs, per Step 1(b). Having it in both places is fine.

omds-guardrails fails with a numpy C-extension ImportError Almost always a tool environment built on the wrong interpreter — the message blames numpy, but the cause is a Python 3.11 environment holding 3.12-built binaries. The traceback names the offending interpreter ("The Python version is: Python 3.11 from …"). Rebuild the environment pinned:

uv tool install --force --python 3.12 \
    "git+https://github.com/spkc83/omds.git#subdirectory=python/omds"

omds-guardrails: command not found The package installed into an environment whose bin/ is not on your PATH — almost always a virtualenv that is not activated. Check where it went:

python -c "import omds, pathlib; print(pathlib.Path(omds.__file__).parent)"

Then either activate that environment, or call the scripts by full path (/path/to/venv/bin/omds-guardrails). Configure your agent with the absolute path to omds-mcp if it does not inherit your shell's PATH — a common cause of an MCP server that works in a terminal but not in the agent.

The install failed on the Python version omds requires 3.12+. python -V to check; install into a newer interpreter (uv venv --python 3.12).

check reports no violations on obviously leaky code The static packs match specific shapes, not intent. ax:fit_on_train_only needs the fit-then-split sequence inside a function — at module level it is not matched. This is a deliberate precision-over-recall trade: a guardrail that cries wolf gets turned off. The runtime in-kernel guard is the complement, firing on data that actually reaches fit() regardless of code shape.

The output has an unreadable key A file you asked about could not be read or parsed — a non-UTF-8 encoding, a syntax error, a permissions problem; reason says which. Those files were not analyzed, so treat them as unchecked rather than clean. Files elsewhere in the directory that cannot be parsed are simply skipped and never appear here.

MCP server shows as failed in the agent Run omds-mcp directly in a shell. It should start and sit silently waiting for stdio input (Ctrl-C to quit) — that is a healthy server, since it speaks a protocol, not a CLI. A traceback or command not found is the real problem; see the PATH note above.

The plugin's guardrail hook never fires By design it fails open in every abnormal state, including omds-guardrails not being on PATH — a missing package makes it do nothing rather than break your editing, which also means a broken install looks exactly like a clean file. Confirm omds-guardrails check --files <a leaky file> works in your shell first. Note the hook only inspects .py files that Claude writes or edits.

/plugin marketplace add spkc83/omds cannot find the plugin The marketplace manifest is read from the repository's default branch. Use claude --plugin-dir . from a checkout to test the plugin without publishing.

omds: command not found / omds init cannot find benchmark files Part 2 needs the repository, not just the installed package — omds init copies benchmark files that are not shipped inside the wheel. Clone and uv sync.

Still stuck? scripts/verify_install.py checks every mechanical part of an install in one command and names what it cannot check — see Verifying a real install.


Part 2 — E-GDS, the offline evolutionary optimizer

Quickstart

# 1. install (from a checkout — `omds init` copies benchmark files that are not
#    shipped in the wheel, so this half needs the repo, not just the package)
uv sync

# 2. a local model behind an OpenAI-compatible endpoint (Ollama shown; llama.cpp
#    server / vLLM / LM Studio all work too — anything speaking /v1/chat/completions)
ollama pull qwen2.5-coder:14b

# 3. scaffold a task — copies the tabular_multiclass benchmark's seed pipeline plus
#    freshly-generated synthetic data into ./mytask/workspace, and gives that
#    workspace its own git history on `main` (evolve/promote clone it via
#    `git worktree` and need a repo to clone from)
uv run omds init mytask

# 4. evolve the pipeline offline under a GEPA evaluation budget
uv run omds evolve mytask/task.yaml --max-evals 12

# 5. inspect the Pareto frontier, then promote a gate-passing variant
uv run omds inspect frontier --task mytask/task.yaml
uv run omds promote --dry-run --task mytask/task.yaml
uv run omds promote --approve <variant_id> --task mytask/task.yaml

omds inspect also takes traces, violations, and graph; omds models probe -c mytask/omds.yaml records each configured model's capabilities into $OMDS_HOME/models.json (default ~/.omds/models.json).

omds.yaml (written by init alongside task.yaml) points at http://localhost:11434/v1 by default — edit llm.base_url / llm.roles.*.model to point at any other OpenAI-compatible endpoint or model. No live endpoint handy? evolve and models probe also work against the network-free fake:// scheme (see Testing without a live model).

How it works

+---------------------------------------------------------------------------+
|  task.yaml + workspace/   (goal, data roles, metrics, bounds, eval budget) |
+---------------------------------------------------------------------------+
                                     |
+---------------------------------------------------------------------------+
|              ONTOLOGICAL GROUNDING LAYER  (omds.ontology)                  |
|   Axiom registry · builtin packs (leakage, scoring, evaluation, shapes,    |
|   simplex_transforms) · static AST rules · runtime assertion injection     |
+---------------------------------------------------------------------------+
                                     |
+---------------------------------------------------------------------------+
|                    CODE GRAPH LAYER  (omds.codegraph)                      |
|   AST dependency extraction · def-use chains · data lineage · blame        |
+---------------------------------------------------------------------------+
                                     |
+---------------------------------------------------------------------------+
|            OFFLINE GEPA EVOLUTION ENGINE  (omds.evolution)                 |
|   gepa optimize_anything (single-task search, dataset=None) ·              |
|   code-graph-targeted, ontology-briefed reflective mutation proposer ·     |
|   Pareto objectives/frontier · the five gates · internal-PR promotion      |
+---------------------------------------------------------------------------+
        |                                                       |
   each candidate evaluated in an isolated clone         traces + frontier.json
   (omds.sandbox: git worktree + uv venv, rlimits,       (omds.telemetry, read by
    network=False)                                        `omds inspect`)

Two distinct git repositories are involved:

  • The harness repo (this one) — the framework code. Not mutated by GEPA.
  • The pipeline workspace — a per-task git repo under <task_dir>/workspace/ holding the evolving pipeline code, prompts, and configs. GEPA variants are git worktree clones of it; promotion is a merge into its main via an internal PR (branch → rebase → gate rerun → merge). This confines genetic mutation to a blast radius the gates fully cover.

gepa==0.1.1's optimize_anything API in single-task search mode (dataset=None) is the evolution core — it matches E-GDS's one-candidate/one-evaluation model exactly, with a custom_candidate_proposer doing code-graph-targeted, ontology-briefed reflective mutation instead of gepa's default LLM-based reflection. The reflector is the only live LLM call in the loop; everything else (clones, gates, training runs) is real work with no model in it.

The promotion gates (run in order, all blocking)

  1. ontology_static — AST-pattern axiom checks over the workspace; zero error-severity violations (warnings need a runtime proof — see gate 5).
  2. pytest — the pipeline workspace's own contract test suite.
  3. pyright — type check (pyright missing from the clone's venv is a documented SKIP-pass, not a hard failure).
  4. ruff — correctness lint rules only (F, E9, B).
  5. runtime_validation — a real instrumented pipeline run on holdout data: zero unresolved runtime-assertion warnings, metrics within task.yaml bounds, no regression beyond evolution.regression_tolerance.

A gate-passing, frontier-selected variant becomes an internal PR: omds promote --dry-run renders the report (metrics delta, axioms touched, blame targets, trace ids) without merging; omds promote --approve <id> rebases onto main, reruns all five gates post-rebase, then merges. promotion.require_human: true (the config default) always stops at the report; --dry-run forces the report-only path regardless of that setting.

Config reference (omds.yaml)

llm:
  base_url: http://localhost:11434/v1   # any OpenAI-compatible endpoint; or fake://... (see below)
  api_key: local
  roles:
    planner:   {model: qwen2.5-coder:14b}
    coder:     {model: qwen2.5-coder:14b}
    reflector: {model: qwen2.5-coder:14b, temperature: 0.7}   # drives GEPA's mutation proposals
    auditor:   {model: qwen2.5-coder:7b}                      # cheap roles can run smaller models

evolution:
  max_evals: 12                 # gepa's max_metric_calls (budget-driven, not generation-driven)
  parallel: 2                   # gepa's evaluator thread pool; each call works in its own env clone
  max_repair_iterations: 3
  regression_tolerance: 0.0
  promotion_require_human: true # false enables full promotion autonomy

data_dir: runs                  # traces/frontier reports, relative to this file unless absolute
  • Roles → models. Each role binds to a model independently, so cheap roles can run a smaller/faster model. With the agent runtime removed, reflector is the only role the shipped loop actually calls; planner/coder/auditor are still accepted and still validated, but nothing dispatches to them today.
  • EGDS_-prefixed env vars override the file (EGDS_LLM__BASE_URL, …). The prefix predates the egdsomds rename and was left alone deliberately, so existing configs keep working.
  • llm.base_url also accepts fake://... — see below.
  • task.yaml (separate from omds.yaml) declares the task itself: goal, task type, dataset paths/roles, metrics, and bounds/eval_budget.
  • agents: and orchestration: no longer exist. They were shaped for the AutoGen orchestration path deleted in the v2 rework and outlived it by a release — validating cleanly while nothing read them, which reads as a working feature. Unknown keys are rejected, so an omds.yaml still carrying them now fails with a validation error naming them; delete the two blocks.

Testing without a live model

omds.yaml's llm.base_url accepts the fake: scheme (is_fake_base_url in omds.cli_support) as a first-class alternative to a real http(s):// URL — e.g. base_url: fake://local. Under it, evolve and models probe substitute deterministic, network-free stand-ins for every LLM call (the reflector's mutation proposals, the capability probe) while every other part of the loop stays real: actual git worktree env clones, real uv sync, real ontology/pytest/pyright/ruff/runtime gates, and a real sandboxed training run. This is what the default (non-opt-in) CLI test suite runs against, and what the fake-LLM tier of tests/e2e/test_full_loop.py drives end-to-end.

Write it as fake://local, not the bare fake: — a plain YAML scalar ending in a colon is invalid YAML and would need quoting to parse at all.


Shared

Local SLM support (≤7B) — omds-slm

omds-slm (omds.slm) drives a local small model over the same nine tool cores the CLIs and the MCP server call. It lives here rather than in Part 1 because it is the one place a model is pointed at the toolkit: Part 1's guardrails, code graph, ledger and memory need no model at all, and Part 2's GEPA loop uses a model for mutation proposals, not for tool use.

Start with the caveat, because it is the design. This is a supervised, turn-capped harness, not an autonomous agent — the SLM scope guard (C5) is no autonomous missions on a 7B, and the turn cap is what makes that structurally true rather than merely intended:

  • --max-turns is a hard ceiling on model calls, not a hint. The loop makes at most that many calls and then stops, whatever the model asks for.
  • stopped_reason: max_turns is a normal outcome meaning unfinished, your turn — exit code 0. The loop deliberately does not spend an extra call asking the model to wrap up, because a manufactured conclusion at the cap would hide exactly the fact you need to see. Read the transcript and decide.
  • MAX_TURNS_CEILING = 20 bounds what you may even ask for. A cap of 200 is an autonomous mission wearing a cap's clothes; the remedy for a task that genuinely needs more steps is another supervised invocation, not a bigger number.
# a local model behind an OpenAI-compatible endpoint (Ollama shown)
ollama pull qwen2.5-coder

# one supervised run: 4 model calls, maximum
uv run omds-slm run \
    --model qwen2.5-coder \
    --task "Check python/omds/src/omds/pipeline.py for data leakage." \
    --max-turns 4

# the summary format, one line per turn, then the outcome and the answer
# (what a given model actually says is its own business):
#
#   turn 1: guardrails_check -> ok
#   turn 2: answered
#   stopped_reason: final_answer (2/4 turns)
#
#   <the model's answer>
#
# …and `stopped_reason: max_turns (4/4 turns)` is the other normal ending.

# --json prints the whole LoopResult: every message sent, the tools offered that
# turn, the estimated token count, and every tool result. That is the auditable
# record of the run — read it, don't trust the summary.
uv run omds-slm run --model qwen2.5-coder --task "" --max-turns 4 --json

--base-url (default http://localhost:11434) is Ollama's native root; the chat client appends /v1 itself. Any OpenAI-compatible endpoint works — llama.cpp's server, vLLM, LM Studio — though the context-window probe below is Ollama-specific and degrades to a static guess elsewhere.

What it does that a plain chat loop does not, all of it in service of a window that may be 4–8k rather than 200k:

  1. It probes the model's REAL served window (omds.slm.probe) instead of guessing from the model name. Ollama's /api/show publishes the trained context length in GGUF metadata — and, crucially, the loop clamps that by the Modelfile's num_ctx, because the window Ollama actually serves is often much smaller than the one the model was trained for. Budgeting against the larger number is how you overflow a context you thought you had. Every failure here (no server, missing metadata) degrades silently to the static table in omds.llm.registry: a capability probe must never be the reason a run cannot start.
  2. It narrows the tool set per turn (omds.slm.funnel) — hard triggers → BM25 → synonym-expanded BM25 → optional embeddings → reciprocal-rank fusion → top-5 of the nine tools. The narrowing re-runs every turn against the current request, so the offered subset follows the conversation instead of being frozen at turn 1.
  3. It budget-packs the turn (omds.slm.context) so tool schemas can never crowd out the task. The system prompt and the current request are priority-0 — never truncated, never dropped. Everything else is shed to fit context_length - reserve_output, oldest history first, then skills, then the worst-ranked tools last. If the system prompt and the task alone do not fit, the run stops and says so (stopped_reason: context_too_small, exit 1) rather than quietly truncating your question and confidently answering something you did not ask.
  4. It bounds what comes back from a tool, twice — once at the arguments, using each core's own knobs (codegraph_context's max_items, memory_recall's k), so the core never builds the oversized payload; and again at the rendered result, with a flat character budget, because seven of the nine tools are bounded only there.

The wire protocol is JSON-in-text, not native tool calling ({"tool": …, "arguments": {…}} or {"answer": …}), parsed leniently through code fences and surrounding prose — local 7B tool-call support is too uneven to depend on. A tool the model invented, malformed arguments, or a core that raises are all normal turn outcomes: the error is fed back and the model gets to correct itself, within the cap.

Maturity, honestly. The harness itself is well covered by unit tests that inject the transport, the capabilities and the tool caller — no test in the default suite touches a socket. The live path is smoke-tested only behind an opt-in gate (python/omds/tests/slm/test_ollama_smoke.py), which skips unless an Ollama is reachable and the model is pulled; see Testing. If you are the first person to point this at a model that is not qwen2.5-coder, expect to tune the prompt.

SLM-first design rules

Every prompt in the system is built to work on a 7B coder model with an 8–32k window — SLMs are the design floor, never the ceiling:

  1. OpenAI-compatible transport only. One httpx-based async client (omds.llm.client) speaking /v1/chat/completions — no provider SDKs.
  2. Structured output via constrained decoding, not hope. Schema-validated JSON (omds.llm.structured.generate), with a bounded repair loop that re-prompts with the bad output plus the validation error; persistent failure is a StructuredOutputError, never coerced or silently dropped.
  3. A model capability registry (omds.llm.registry) records per-model context length, max output tokens, schema-decoding mode, and native-tool-call support, probed once via omds models probe and persisted.
  4. Context budgeting (omds.llm.budget.fit) assembles prompts from prioritized sections and truncates the lowest-priority ones first, so no call ever exceeds the model's window. Priority-0 sections are never dropped.
  5. Micro-prompts. No prompt does two jobs. With the agent runtime removed the shipped surface is small — the reflector's mutation proposal and the capability probe are the only live LLM calls — but both go through rules 2 and 4 rather than raw chat().
  6. Bounded tool output. The MCP adapter's codegraph_context caps per-file symbols (max_items) and drops per-symbol detail by default (detail), so an SLM-hosted client gets a response that fits its window while still seeing true totals.

The v1 "capability-adaptive dispatch" rule (native tool calls vs. a structured NextAction fallback) is gone, along with the EgdsChatClient that implemented it. Nothing in the shipped tree dispatches on a model's probed tool_calls capability; the registry still records the flag.

Repository layout

python/omds/                 # the `omds` package — a uv workspace member, the installable artifact
  src/omds/
    ontology/                #  axiom registry + packs (leakage, scoring, evaluation, shapes,
                             #    simplex_transforms), static AST checks, runtime assertions
    codegraph/               #  AST -> networkx graph, def-use chains, data lineage, blame
    dstools/                 #  in-kernel helpers: df_summary, capture_plot, experiment ledger
    memory/                  #  session pad (markdown) + SQLite FTS5/KV episodic store
    llm/                     #  httpx OpenAI-compatible client, structured-output repair loop,
                             #    capability registry, context budgeter
    evolution/               #  gepa evaluator, mutation proposer, Pareto objectives,
                             #    the five gates, internal-PR promotion
    sandbox/                 #  git-worktree + uv-venv env clones, rlimit/network-isolated exec
    telemetry/               #  TraceRecord/RunArtifact models, metric collectors
    runtime/                 #  task spec loading, skills catalog, offline validation gate
    slm/                     #  `omds-slm` — the supervised, turn-capped local-SLM harness:
                             #    Ollama capability probe, tool funnel, budget-aware
                             #    turn assembler, turn-capped loop (see Shared, above)
    pipeline.py              #  @stage / Pipeline contract — the evolvable-code hook
    cli.py                   #  `omds`            init / evolve / promote / inspect / models
    guardrails_cli.py        #  `omds-guardrails` check + the kernel-hook install/uninstall/status
    codegraph_cli.py         #  `omds-codegraph`  lineage / blame / context
    dstools_cli.py           #  `omds-dstools`    ledger show
    memory_store_cli.py      #  `omds-memory-store` add / recall
    sessionpad_cli.py        #  `omds-sessionpad`   append / show (the markdown session pad)
    mcp_server.py, mcp_tools_*.py   #  `omds-mcp` — the 9-tool MCP adapter
    install.py               #  `omds-install`    the optional convenience installer
    kernel_guard.py          #  runtime fit()-on-tagged-test-data guard
    kernel_hook.py           #  the opt-in `.pth` that arms it environment-wide
  tests/mcp/                 # MCP tools, installer, skill portability, plugin-bundle invariants
  tests/slm/                 # probe/funnel/assembler/loop, all injected — plus the opt-in
                             #   live-Ollama smoke (skipped unless a model is actually there)

skills/                      # the 7 portable, agent-neutral Agent Skills
hooks/                       # PostToolUse guardrail hook (stdlib-only, fail-open) + hooks.json
.claude-plugin/              # plugin.json + marketplace.json — the Claude Code plugin manifest
.mcp.json                    # the project-scoped MCP entry, for this repo itself
benchmarks/tabular_multiclass/   # end-to-end benchmark: make_data.py, task.yaml, workspace_seed/
tests/                       # the E-GDS suite: unit + integration, plus opt-in e2e/ and benchmarks/
docs/                        # specs, plans, ADRs, research

The tree is Python only — there is no TypeScript, no bun, no node. The pi-era TS algorithms were kept in an archive/ for one release while the skill funnel was ported to Python; that port is done and better covered than the original, so the archive was removed rather than left as a second, unbuilt definition of the same thing. It is still readable in git history:

git show f3d221f:archive/ts-extension/skill-funnel.ts   # the funnel, now omds/slm/funnel.py
git show f3d221f:archive/ts-extension/runner.ts         # the workflow runner, never ported
git show f3d221f:archive/ts-extension/README.md         # what each file was and why it was kept

Testing

There is no CI. This repo runs no GitHub Actions workflow — the three gates below are run locally, and nothing enforces them on push. Run all three before you commit; that is the entire safety net.

uv run pytest                # default suite: 705 passed, 6 skipped — no network, no live model
uv run ruff check            # correctness lint (F, E9, B); currently clean
uv run pyright               # static types over python/omds/src; currently clean

EGDS_SLOW=1 uv run pytest tests/benchmarks/ tests/test_packaging.py
                                              # + the real-clone 5-gate benchmark (uv sync per clone)
                                              #   and a real `uv build` of the sdist + wheel
EGDS_E2E=1  uv run pytest tests/e2e/          # + the fake-LLM end-to-end tier (spec §1.3), no network

# best-effort: the real-endpoint E2E tier, against a live local model
EGDS_E2E=1 EGDS_E2E_BASE_URL=http://localhost:11434/v1 EGDS_E2E_MODEL=qwen2.5-coder:14b \
    uv run pytest tests/e2e/

# best-effort: the live-Ollama SLM smoke — no env var opts in, the world does.
# `ollama serve` + `ollama pull qwen2.5-coder`, then it simply runs:
uv run pytest python/omds/tests/slm/test_ollama_smoke.py -rs

The six default skips are exactly those opt-in tiers (three in tests/e2e/, one in tests/benchmarks/, one the SLM smoke, one the real uv build in tests/test_packaging.py). The env-var names still use the EGDS_ prefix.

tests/test_packaging.py is worth knowing about: every other test runs against the uv workspace — an editable install resolved by uv.lock — which is not what a user gets. Two defects lived in that gap (an uncapped mcp that resolved to a version without mcp.server.fastmcp, and a source distribution that could not build at all), so these assert on the declared metadata and on real built artifacts rather than on the imported package.

Verifying a real install

The suite proves the code works; it does not prove an installation is wired up. scripts/verify_install.py does that half, in one command, against a throwaway temp directory:

uv run python scripts/verify_install.py          # add --keep to inspect the temp dir

It checks that every console script resolves, that omds-install claude writes the MCP entry and copies the skills, that a real omds-mcp subprocess speaks stdio MCP and returns an ax:fit_on_train_only violation from guardrails_check, that memory round-trips, that the plugin manifests validate and the plugin's .mcp.json matches omds.install._OMDS_SERVER, that the PostToolUse hook both blocks on a certain violation and stays silent with omds-guardrails off PATH, and that the in-kernel guard raises on tagged test data. The live SLM run is included when an Ollama is reachable and skipped with a reason when it is not. It exits non-zero only on a real failure, and touches nothing outside its temp dir.

It also prints, at the end, the things it cannot verify — chiefly that Claude Code actually connects in-session, that a skill surfaces as omds:leakage-audit, and that the hook fires on a real model edit. Those need a human with a live session; the script is explicit about not covering them.

python/omds/tests/slm/test_ollama_smoke.py is the only test in the tree that talks to a live model. Its gate is not an env var but reachability: a GET /api/tags with a 2-second timeout at collection time, and a check that the model is really pulled — so it skips with a specific reason (server down vs. model missing) instead of failing in transport, and a black-holed port cannot stall the suite. EGDS_SLM_SMOKE_BASE_URL (Ollama's native root, no /v1) and EGDS_SLM_SMOKE_MODEL override the defaults. It asserts only what must be true of a working harness — the run terminated inside its cap, no transport or budget fault, guardrails_check actually dispatched — never the model's prose, because a 7B at temperature 0.2 is not deterministic.

tests/e2e/test_full_loop.py is the single source of truth for spec §1.3's success criteria: the benchmark evolves end-to-end and GEPA's frontier has a feasible, gate-passing member promotable via internal PR; an injected ontology violation (unnormalized logits into log_loss) is rejected by the ontology gate naming the axiom and the offending code-graph node; and every candidate-evaluation subprocess runs with network=False.

Non-goals

GPU fine-tuning/RL of the underlying models; non-tabular modalities (vision, audio); a web UI; distributed multi-machine execution; a full OWL/RDF reasoner; harness self-mutation (GEPA only evolves a pipeline workspace, never omds itself). Per the SLM scope guard: no autonomous missions on a 7B — anything model-driven here stays bounded and supervised.

License

Apache License 2.0 — see NOTICE for the attribution notices that §4(d) asks redistributors to carry along. Both files ship inside the wheel and the sdist, so anyone who installs the package has them.

Apache-2.0 rather than a permissive-but-silent licence mainly for the express patent grant in §3 and its termination clause, which MIT simply does not address. The practical differences if you build on this: keep the LICENSE and NOTICE with any redistribution, and state what you changed if you ship modified files (§4(b)). There is no copyleft — Apache-2.0 does not require you to open your own code.

No third-party source is vendored anywhere in the tree; everything external is a normal declared dependency. The project holds no contributions from anyone other than the copyright holder, so the relicence from MIT (2026-08-04) needed no third-party consent.

History

v2 of this project was briefly built as an extension of a pinned, fast-moving single-maintainer coding-agent fork (@oh-my-pi/pi-coding-agent); that layer was decommissioned and replaced by the harness-agnostic core described above. The one algorithm worth keeping — the skill funnel — was ported to Python as omds.slm.funnel; the rest is in git history (see Repository layout). Both that decision and the two amendments that reshaped it — Skills+CLI promoted over MCP (2026-07-20), and the Claude Code plugin added as a third thin surface (2026-07-31) — are recorded in docs/adr/2026-07-18-mcp-pivot.md. The framework evaluation that chose AutoGen for v1's since-deleted runtime is kept for the record at docs/specs/2026-07-12-framework-evaluation.md.

About

OMDS (oh-my-datascience) — data-science guardrails for coding agents: train/test leakage and metric-misuse checks, code-graph lineage, an experiment ledger and episodic memory, as portable Agent Skills + CLIs, an optional MCP server, and a Claude Code plugin. Includes E-GDS, an offline GEPA pipeline optimizer with gated promotion.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages