PRAXIS

Self-improving knowledge loop for Claude Code agents.

Claude Code's auto-memory saves a few notes between sessions — but it's an unverified black box: no human approval, no deduplication, no measurement. PRAXIS mines the full JSONL session logs the agent already produces, distills durable lessons, runs them through a confidence score and human-approval gate, and injects only promoted knowledge into future sessions — so the agent provably stops relearning the same things and gets better over time.

Memory vs. knowledge. Auto-memory captures scattered, episodic notes. PRAXIS produces generalized, deduplicated, confidence-scored, human-approved, measured knowledge with full provenance.

Remote: GitLab — monicapeters/praxis
Architecture source of truth: docs/plans/PRAXIS_Project_Plan.html

The loop

raw logs → extract candidate lessons → consolidate/dedup/generalize → confidence score
         → [human approval gate] → Knowledge Graph → get-context tool → future sessions
         → measure improvement → repeat

flowchart LR
  JSONL[Claude Code JSONL logs] --> Ingest[Ingest and segment]
  Ingest --> Detect[Learning moment detection]
  Detect --> Distill[LLM distillation]
  Distill --> Consolidate[Cluster dedup score]
  Consolidate --> Gate[Human gate dashboard]
  Gate -->|proposed to suggested to active| KG[Knowledge Graph]
  KG --> GetContext[get-context tool]
  GetContext --> Agent[Future sessions]
  Agent --> JSONL
  KG --> Substrate["CLAUDE.md / skills substrate"]
  GetContext --> Eval[Eval harness]
  Eval --> Gate

Problem

Coding agents are less amnesiac than they used to be, but durable knowledge still lives in the gaps:

No quality gate — save decisions are opaque; wrong patterns can be memorized from one-off mistakes.
No dedup, decay, or conflict resolution — stale and contradictory notes coexist indefinitely.
No measurement — nothing verifies a saved memory actually helped a later session.
Per-repository only — nothing carries across projects, models, or domains.
Underused raw material — full JSONL transcripts (~/.claude/projects/<project>/<session>.jsonl) record every mistake, correction, and success; auto-memory skims in-flight and discards the rest.

PRAXIS treats that exhaust as a compounding asset.

Implementation status

Point-in-time snapshot as of 2026-06-18 (Sprint Day 2). See AUDIT.md for the full repo health review.

Area	Path	Owner	Status
Human-gate dashboard (Streamlit)	`frontend/`	Monica Peters	Demo-ready — mock fixtures, contract v1 API client, Render deploy blueprint
Knowledge Graph dashboard (React)	`frontend-react/`	Monica Peters (client) / Matthew Daw (server)	Demo-ready (mock) — Vite + TypeScript UI targeting same candidate-api-v1; Matthew validates his REST server without Streamlit
Knowledge substrate	`knowledge/`	Matthew Daw	Foundation — in-memory graph, prompt ingestor, whole-file reader, wiring factory
Eval harness	`knowledge/evals/`	Dominic Antonelli	Partial — 5 YAML cases, deterministic checks, real Claude Code runner + offline FakeRunner
Session capture	`session-capture/`	Dominic Antonelli	Working — Go `claude+` PTY daemon, JSONL tailer, DynamoDB writer
Cloud infra	`infra/`	Dominic Antonelli	Scaffolded — AWS CDK stack for sessions DynamoDB table
Candidate REST API	—	Matthew Daw	Planned — contract v1 documented; server not yet in-repo
Eval metrics endpoint	—	Dominic Antonelli	Planned — contract v1 documented; dashboard embed ready
CI pipeline	—	Team	Not yet — manual test runs only

Integration posture: The dashboard runs fully offline when PRAXIS_API_BASE_URL is unset. Set env vars per docs/integration/wire-up.md to wire live backend and eval metrics without code changes.

MVP scope

In scope	Out of scope
Ingest + segment real Claude Code JSONL logs	Training models from scratch
Learning-moment detection (heuristics + LLM)	Hosted SaaS
LLM distillation with provenance	Non–Claude-Code agents
Cluster/dedup + confidence scoring	Real-time mid-session learning
Knowledge Graph as primary knowledge store
get-context tool (session + codebase + graph → injected context)
Streamlit + React human-gate dashboards in `frontend/` and `frontend-react/` (`proposed → suggested → active`)
Complementary injection via generated `CLAUDE.md` / skills
Eval harness measuring correction rate before/after (VCS-agnostic PR/ticket replay)

Implemented beyond MVP shell: contradiction-resolution UI (dashboard); React client for Matthew API validation.

Stretch goals: trained classifier for learning moments; substrate bake-off (markdown/skills vs. vector RAG vs. knowledge graph); confidence decay and re-verification; pipeline-side contradiction detection; cross-project knowledge.

Success criteria

Primary metric: ≥50% fewer user corrections on benchmark tasks vs. cold runs, with no regression in task success rate.
Compounding proof: visible correction-rate curve falling across sessions.
Demo outcome: point PRAXIS at a repo's logs → ranked candidate lessons with evidence in minutes → human promotes the good ones → re-run shows quantified improvement (corrections, failures, tokens, time).

Team & pillars

Three Gauntlet AI Fellows, each owning one end-to-end pillar for a 9–10 day focused sprint:

Lead	Pillar	Focus
Matthew Daw	ML & Knowledge Pipeline	Ingestion, learning-moment detection, LLM distillation, consolidation/dedup/scoring, knowledge graph, provenance
Monica Peters	Dashboard & Human Gate	Streamlit + React human-gate dashboards, approval workflow, contradiction resolution UI, credibility metrics
Dominic Antonelli	Architecture, Eval & Integration	System design, eval harness, VCS-agnostic replay automation, session capture wrapper, deployment, compounding-curve proof

Daily 15-minute syncs; all code reviewed by at least one other member before merge.

Sprint timeline

Sprint Day 1 = Wednesday, June 16, 2026 (Thursday June 18 skipped). See the confidential project plan for the detailed day-by-day schedule.

Phase	Days	Milestones
Foundation & design	1–2	Architecture, data contracts, dashboard shell, eval skeleton, cold-run baseline
Parallel core build	3–5	Full pipeline, human-gate UI, scoring/decay, eval replay automation
Integration	6–7	Dashboard ↔ backend API, injection, eval harness, promotion triggers replay
Measurement	8	Compounding curve, threshold tuning, edge-case polish
Demo & handoff	9–10	Live demo script, documentation, presentation practice (internal Jun 26–27)
Gauntlet showcase	—	Mon Jun 29 — 10-minute live presentation

Team freeze gates and three practice runs: docs/monica/PLAN_ALIGNMENT_GAP_CHECKLIST.md.

Live demo (3 acts)

Dumb agent — fresh repo with deliberate quirks; agent stumbles, gets corrected; log captured.
Distillation — PRAXIS surfaces scored candidates linked to transcript lines; human promotes suggested → active.
Smart agent — sibling task nails quirks first try; side-by-side scoreboard plus compounding curve across a pre-run batch.

Demo script: docs/monica/DEMO_SCRIPT.md

Repository layout

praxis/
├── docs/                      # Plans, proposals, integration contracts, fixtures
│   ├── integration/           # candidate-api-v1, eval-metrics-v1, wire-up, JSON fixtures
│   ├── monica/                # Dashboard pillar — architecture, wireframes, deploy, demo
│   ├── matt/future-work/      # Post-MVP knowledge-graph eval design (parked)
│   └── plans/                 # MVP plan (mvp-plan.html)
├── .cursor/rules/             # Team Cursor rules (shared, dashboard, pipeline-eval, git-sync)
├── frontend/                  # Streamlit human-gate UI (Monica)
│   ├── app.py                 # Entry — provider wiring only
│   ├── components/            # List, detail, badges, contradiction panel, eval embed
│   ├── models/                # Candidate types (API contract surface)
│   ├── services/              # DataProvider, mock + API clients, contract_v1
│   ├── tests/                 # Contract fixture + mock workflow tests
│   ├── mock_data.py           # Local fixtures — no backend required
│   └── render.yaml            # Render.com deploy blueprint
├── frontend-react/            # React Knowledge Graph dashboard (Monica — Matthew API client)
│   ├── src/                   # Vite + TypeScript — same contract v1 as Streamlit
│   ├── public/mock-candidates.json
│   └── README.md              # Matthew self-serve wire-up (VITE_* env vars)
├── knowledge/                 # Knowledge substrate + eval harness (Matthew & Dominic)
│   ├── knowledge_graph/       # KnowledgeGraph ABC + InMemoryGraph
│   ├── injestion/             # Ingestor ABC + PromptIngestor
│   ├── graph_reader/          # WholeFileReader → Claude tool adapter
│   ├── evals/                 # YAML cases, deterministic checks, Claude Code runner
│   ├── wiring.py              # build_trio() factory
│   └── run.py                 # Debugger entry — ingest smoke + eval runner
├── session-capture/           # Go claude+ wrapper — PTY daemon + DynamoDB capture
│   └── wrapper/               # cmd/claude-plus, internal/{pty,daemon,capture,store}
├── infra/                     # AWS CDK — praxis-sessions DynamoDB table
├── run.py                     # Repo-root shim → knowledge/run.py
├── pyproject.toml             # Python 3.12+ deps (uv/pip)
├── uv.lock                    # Locked Python dependencies
└── README.md

Note: Early plans referenced top-level pipeline/ and eval/ directories. Current implementations live under knowledge/ (including knowledge/evals/) and session-capture/. API contracts are path-agnostic.

Prerequisites

Tool	Version	Used for
Python	≥ 3.12	Dashboard, knowledge package, eval harness
uv	latest (recommended)	Dependency install and script runner
Go	≥ 1.22	Building `session-capture/wrapper`
Node.js	≥ 20	React dashboard (`frontend-react/`), AWS CDK deploy (`infra/`)
AWS CLI	configured	DynamoDB session capture (optional — wrapper runs without it)

Quick start

1. Install Python dependencies

From the repo root:

uv sync

Or with pip:

python -m venv .venv
.\.venv\Scripts\pip install -e .

2. Run the human-gate dashboard

cd frontend
Remove-Item Env:PRAXIS_API_BASE_URL -ErrorAction SilentlyContinue
uv run streamlit run app.py

Mock mode loads fixtures from mock_data.py — no backend required. See docs/integration/wire-up.md for live API and eval-metrics wiring.

Render deploy (portfolio demo): docs/monica/RENDER_DEPLOY.md

2b. Run the React dashboard (Matthew API client)

cd frontend-react
npm install
npm run dev

Mock mode loads public/mock-candidates.json — no backend required. Set VITE_PRAXIS_API_BASE_URL in .env.local for Matthew's live server. See frontend-react/README.md and docs/integration/wire-up.md.

3. Run the eval harness

Offline (no Claude Code subscription):

$env:PRAXIS_EVAL_REAL = "0"
uv run python run.py

Real Claude Code (uses subscription credits):

uv run python run.py

Registered cases live in knowledge/evals/cases/. Results append to knowledge/evals/results/.

4. Build session capture (optional)

# Deploy DynamoDB table
cd infra
npm install
npm run deploy

# Build claude+ wrapper
cd ..\session-capture\wrapper
go build -o claude+ ./cmd/claude-plus

# Host a session (streams to DynamoDB when AWS creds present)
$env:SESSION_TABLE = "praxis-sessions"
$env:AWS_REGION = "us-east-1"
.\claude+

Full wrapper docs: session-capture/README.md

Configuration

Variable	Required	Component	Purpose
`PRAXIS_API_BASE_URL`	No	Dashboard	Candidate REST API base URL; unset → mock fixtures
`PRAXIS_API_TOKEN`	No	Dashboard	Bearer token for API auth
`PRAXIS_CONTRACT_VERSION`	No	Dashboard	API contract version header (default `1`)
`PRAXIS_EVAL_METRICS_URL`	No	Streamlit dashboard	GET endpoint returning eval metrics JSON for compounding-curve embed
`VITE_PRAXIS_API_BASE_URL`	No	React dashboard	Same as `PRAXIS_API_BASE_URL`; unset → mock fixtures
`VITE_PRAXIS_API_TOKEN`	No	React dashboard	Bearer token for API auth
`VITE_PRAXIS_EVAL_METRICS_URL`	No	React dashboard	Eval metrics JSON URL for compounding-curve embed
`VITE_PRAXIS_CONTRACT_VERSION`	No	React dashboard	API contract version header (default `1`)
`PRAXIS_EVAL_REAL`	No	Eval harness	Set to `0` for offline FakeRunner; default runs real Claude Code
`SESSION_TABLE`	No	Session capture	DynamoDB table name for transcript streaming
`AWS_REGION`	No	Session capture	AWS region for DynamoDB writer

Secrets are environment-only — never commit tokens or credentials.

Testing

Knowledge package (39 tests — run from repo root):

uv run pytest knowledge/ -q

Dashboard contract tests (11 tests — set PYTHONPATH from repo root):

$env:PYTHONPATH = "frontend"
uv run pytest frontend/tests/ -q

Contract fixtures are canonical in docs/integration/fixtures/.

Documentation

Document	Description
docs/plans/PRAXIS_Project_Plan.html	Source of truth — team plan, architecture overview, 9-day schedule
docs/plans/mvp-plan.html	MVP core contracts and eval schema
docs/plans/proposal-praxis.md	Capstone proposal — problem, direction, risks (historical)
docs/integration/candidate-api-v1.md	Matthew ↔ Monica candidate REST contract + fixtures
docs/integration/eval-metrics-v1.md	Dominic ↔ Monica eval metrics JSON contract
docs/integration/wire-up.md	Self-serve Streamlit + React wire-up (no pairing)
frontend-react/README.md	React Knowledge Graph dashboard — Matthew API validation
docs/monica/ARCHITECTURE_MONICA.md	Dashboard pillar architecture — Streamlit stack, API boundaries
docs/monica/monica-wireframes.md	Dashboard as-built spec and UX notes
docs/monica/DEMO_SCRIPT.md	Three-act live demo script
docs/monica/PLAN_ALIGNMENT_GAP_CHECKLIST.md	Team gap checklist, Scrum Master duties, demo freeze gates
docs/monica/STANDUP_TEMPLATE.md	Daily 15-min standup template
docs/Matthew-Daw-ML-Pipeline-PlanDRAFT.md	ML pipeline pillar plan
docs/Dominic-Antonelli-Architecture-Eval-PlanDRAFT.md	Architecture, eval & integration pillar plan
session-capture/README.md	Go wrapper — claude+ CLI, DynamoDB capture
AUDIT.md	Full-repo health audit (2026-06-18)
CHANGELOG.md	Version history and release notes

Agent and editor guidance for contributors lives in .cursor/rules/:

praxis-shared.mdc — commits, reviews, TypeScript style, provenance standards
praxis-dashboard.mdc — human-gate UI patterns (Monica's pillar)
praxis-pipeline-eval.mdc — pipeline data contracts, eval harness, integration (Matthew & Dominic)
praxis-git-sync.mdc — GitLab main sync workflow

Contributing

Use conventional commits (feat, fix, chore, docs, refactor, test) with #<issue> references.
Open small, focused GitLab merge requests with clear descriptions; at least one peer review required before merge.
Sync your dev branch with origin/main before starting work or opening an MR (git fetch origin main; git merge origin/main).
Preserve provenance on every candidate/lesson object (source log path + line offset) in code and UI.
Promotion actions should trigger VCS-agnostic eval replay (scripted PR/ticket scenarios) for before/after measurement.
All code must pass lint and type checks before review.

License

TBD — Gauntlet AI capstone project (2026).

Changelog

See CHANGELOG.md for the full version history. Current release: 0.1.0 (2026-06-18).

Navigation Menu

Repository files navigation

PRAXIS

Table of contents

The loop

Problem

Implementation status

MVP scope

Success criteria

Team & pillars

Sprint timeline

Live demo (3 acts)

Repository layout

Prerequisites

Quick start

1. Install Python dependencies

2. Run the human-gate dashboard

2b. Run the React dashboard (Matthew API client)

3. Run the eval harness

4. Build session capture (optional)

Configuration

Testing

Documentation

Contributing

License

Changelog

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 67 Commits
.cursor		.cursor
docs		docs
frontend-react		frontend-react
frontend		frontend
infra		infra
knowledge		knowledge
session-capture		session-capture
.env.example		.env.example
.gitignore		.gitignore
AUDIT.md		AUDIT.md
CHANGELOG.md		CHANGELOG.md
README.md		README.md
pyproject.toml		pyproject.toml
run.py		run.py
uv.lock		uv.lock

Folders and files

Latest commit

History

Repository files navigation

PRAXIS

Table of contents

The loop

Problem

Implementation status

MVP scope

Success criteria

Team & pillars

Sprint timeline

Live demo (3 acts)

Repository layout

Prerequisites

Quick start

1. Install Python dependencies

2. Run the human-gate dashboard

2b. Run the React dashboard (Matthew API client)

3. Run the eval harness

4. Build session capture (optional)

Configuration

Testing

Documentation

Contributing

License

Changelog

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages