recursive-self-improvement

A Recursive Self-Improving AI that rewrites its own agent code from failures and fine-tunes its own weights — meta agent, feedback agent, target agent, LoRA, repeat.

Built for the AIEWF Hackathon 2026.

What it does

flowchart TD
    A([Start Generation 0]) --> B

    B["① META AGENT — gen 0 only\nKimi K2.6\nSingle API call → system prompt"]
    B --> C

    B2["① FEEDBACK AGENT — gen 1+\nKimi K2.6\nSingle API call using:\n• prev system prompt\n• prev sol.py\n• eval result\n• context.md history"]
    B2 --> C

    C["② TARGET AGENT\nQwen3-Coder-30B\nRuns target_agent.py → writes sol.py"]
    C --> D

    D["③ TENSARA EVAL\nH100 GPU\ncorrectness check + benchmark"]
    D --> B2
    D --> E{Result?}

    E -- "FAILED" --> F1["④ TRAINING DATA GENERATOR\nKimi K2.6\ncorrectness pair → train.jsonl"]
    E -- "ACCEPTED\nGFLOPS < perf threshold" --> F1P["④ TRAINING DATA GENERATOR\nKimi K2.6\nperf pair → train.jsonl"]

    F1 --> G
    F1P --> G

    G["⑤ LORA TRAINER\nNebius post-training API\nFine-tunes target agent model"]
    G --> G2["Updated target model\nweights"]
    G2 -.->|"replaces base model\nnext gen"| C

    style B fill:#dbeafe,stroke:#3b82f6
    style B2 fill:#bfdbfe,stroke:#2563eb
    style C fill:#fef3c7,stroke:#f59e0b
    style D fill:#dcfce7,stroke:#22c55e
    style F1 fill:#fee2e2,stroke:#ef4444
    style F1P fill:#fde68a,stroke:#d97706
    style G fill:#f3e8ff,stroke:#a855f7
    style G2 fill:#e9d5ff,stroke:#7c3aed

Each generation runs five steps. The key innovation is the feedback agent (gen 1+): instead of writing target_agent.py from scratch with no memory, it reads the previous script, the kernel it produced, the specific error, and the full generation history — then surgically rewrites it to fix the exact failure pattern seen.

Evaluation can run on Tensara (remote H100, leaderboard-official) or locally / on Modal (no rate limits, one container cold-start covers the entire run).

Meta agent (Kimi K2.6, gen 0 only) writes the initial target_agent.py — a Python script that instructs the target LLM how to write a Triton kernel
Feedback agent (Kimi K2.6, gen 1+) receives prev system prompt + prev sol.py + eval result + context.md in a single API call and returns an improved system prompt. loop.py injects it into the fixed scaffold to produce the next target_agent.py. The feedback agent has no connection to the curator or LoRA loop.
Target agent (Nemotron-3-Ultra-550b) executes target_agent.py, calls the LLM, and writes a Triton kernel to sol.py
Tensara evaluator submits sol.py to a remote H100 — checks correctness, then benchmarks for GFLOPS and latency
Curator (Kimi K2.6) generates a structured (prompt, completion) training pair → appended to train.jsonl. Two modes:
- Correctness — on any failed submission: "here is broken code + error, produce fixed version"
- Performance — on accepted-but-slow submissions (below --curate-perf-threshold): "here is working-but-slow code + benchmark, produce faster version"
LoRA trainer (llama-finetune) fine-tunes Gemma4-1B on train.jsonl on CPU — improving the curator model itself so it generates better training pairs in future gens. Note: this does not feed back into the feedback agent or target agent — the feedback agent reads raw eval results directly and is independent of this loop.

Gen 0:  Meta writes strategy  → Nemotron writes kernel → COMPILE_ERROR → curate → train Gemma4-1B
Gen 1:  Feedback sees error   → fixes system prompt    → WRONG_ANSWER  → curate → train
Gen 2:  Feedback sees pattern → adds post-processing   → ACCEPTED      → 558 GFLOPS (below threshold) → curate perf
Gen 3:  ...                                                             → 621 GFLOPS

Execution flow in detail

Step 1 — Meta agent (gen 0) and Feedback agent (gen 1+)

Gen 0 — run_meta_agent(): Calls Kimi K2.6 with the task description and a system prompt containing Triton kernel rules. Produces the first target_agent.py.

Gen 1+ — run_feedback_agent(): Calls Kimi K2.6 with:

The previous gen's target_agent.py source
The sol.py it produced
The eval result (COMPILE_ERROR, STATIC_CHECK_FAILED, etc.)
context.md — one line per generation summarising what was tried and what failed

The feedback agent writes improvement.md-style reasoning into its rewritten script, ensuring each gen addresses a specific failure rather than starting blind.

Step 2 — Target agent (`rsi/loop.py` → subprocess)

run_target_agent() executes the generated target_agent.py as a subprocess with a 900-second timeout. Environment variables passed in:

Env var	Value
`OPENAI_BASE_URL`	`https://api.tokenfactory.us-central1.nebius.com/v1/`
`OPENAI_API_KEY`	`NEBIUS_API_KEY`
`MODEL_NAME`	`nvidia/Nemotron-3-Ultra-550b-a55b`
`TASK_MD`	path to `tasks/gpu_kernel_task/task.md`
`OUTPUT_DIR`	path to `runs/run-NNN/gen-N/`

The target agent calls Nemotron and writes sol.py to OUTPUT_DIR. A _clean_sol() post-processor strips any reasoning prose prepended before the code.

Step 3 — Evaluation

Two evaluators are available, selected by CLI flag:

Tensara (tasks/gpu_kernel_task/evaluate.py) — official leaderboard, rate-limited (~25 submissions / 18 h):

evaluate.py is invoked as a subprocess. It:

Locates sol.py in the gen directory
Static analysis — AST-checks the Triton kernel for common errors before hitting the API:
- Unused tl.constexpr params (cause COMPILE_ERROR)
- .ravel() calls (invalid in Triton)
- tl.cdiv() with non-constexpr first arg
- Missing import torch
Correctness check — calls TensaraClient.run_checker() against the reference implementation on H100
Benchmark — if correctness passes, calls TensaraClient.run_benchmark() and collects per-shape GFLOPS and latency via SSE streaming
Leaderboard comparison — fetches the current leaderboard best and reports whether the solution beats it
Saves results.json with status, accuracy, average latency (ms), average GFLOPS, and per-shape breakdown

Possible status values: ACCEPTED, WRONG_ANSWER, COMPILE_ERROR, RUNTIME_ERROR, STATIC_CHECK_FAILED, NO_SOLUTION_FILE.

Local / Modal (rsi/evaluate_local.py) — no Tensara API, no rate limits. Activated with --local-eval or --modal-eval:

Runs the same correctness check and torch.cuda.Event benchmarking entirely on a local GPU (or a Modal H100 via --modal)
Maintains a separate ownbest archive under tasks/local/{slug}/ownbest/ (local) or tasks/modal/{slug}/ownbest/ (Modal) — distinct from the Tensara ownbest under tasks/tensara/
RSI_OWNBEST_ROOT env var overrides the ownbest root (used by modal_run.py to redirect writes to the Modal Volume)
Seeded from the best available ownbest: local/modal dir first, then falls back to the Tensara ownbest

Step 4 — Curation (`rsi/curate.py`)

Curation fires in two cases:

Correctness failure (any non-ACCEPTED status): calls the curator with the failed kernel + error message. System prompt asks for a fix-the-bug training pair.
Performance shortfall (ACCEPTED but gflops < --curate-perf-threshold × leaderboard_target): calls the curator with the working-but-slow kernel + benchmark numbers. System prompt asks for an optimization-focused training pair, emphasising memory access patterns, tiling, vectorization, and occupancy tuning.

Both modes produce a JSON { "prompt": ..., "completion": ... } line appended to train.jsonl.

Step 5 — LoRA training (`rsi/train.py`)

Only runs if --base-model is provided and train.jsonl is non-empty.

run_lora_training() invokes llama-finetune with:

--model-base — Gemma4-1B GGUF (or the previous gen's merged GGUF)
--train-data — train.jsonl
--checkpoint-in — previous gen's LoRA adapter (Gen 0 starts fresh)
LoRA hyperparameters from LoraConfig

After training, merge_lora() bakes the adapter into a new standalone GGUF (gemma4-genN.gguf).

Hyperparameter self-adjustment (HyperparamTracker):

ΔPerformance > 0 (improving): keep current config
ΔPerformance < 0 (regression): halve LR, add 2 epochs
ΔPerformance = 0 (stalled): double rank and alpha, increase LR by 1.5×

Architecture

providers/nebius.json           ← Nebius Token Factory endpoint + auth env var

profiles/kimi26-nebius.json     ← meta/feedback agent  (Kimi K2.6 on Nebius)
profiles/nemotron-nebius.json   ← target agent         (Nemotron-3-Ultra-550b on Nebius)
profiles/curator-nebius.json    ← curator              (Kimi K2.6 on Nebius)

rsi/loop.py                     ← main orchestrator
                                     run_meta_agent()     — gen 0 bootstrap
                                     run_feedback_agent() — gen 1+ improvement loop
                                     update_context()     — context.md history tracker
                                     run_evaluation()     — dispatches to Tensara or local eval
rsi/multi.py                    ← batch runner (multiple problems, progress checkpoint)
rsi/agent.py                    ← OpenAI-compat LLM caller
rsi/curate.py                   ← failure + perf-shortfall logs → train.jsonl
rsi/train.py                    ← llama-finetune wrapper + hyperparameter tracker
rsi/evaluate_local.py           ← local / Modal H100 evaluator (no Tensara API)
rsi/modal_run.py                ← Modal entrypoint: full loop in one H100 container

tasks/gpu_kernel_task/          ← scaffold template (shared files, not a real problem dir)
  evaluate.py                   ← Tensara submission + SSE result parser
  tensara_client.py             ← lightweight Tensara API client
  meta_system.md / feedback_system.md / reference_target_agent.py

tasks/tensara/                  ← one dir per problem, auto-created by scaffold_task()
  swish/ gelu/ rms-norm/ l2-norm/ log-softmax/ cosine-similarity/ matrix-vector/
    task.md / reference_kernel.py / task_config.json
    ownbest/tensara/*.py        ← personal best kernels submitted to Tensara

tasks/local/                    ← local-GPU ownbest (keyed by GPU model, e.g. GB10)
  swish/ownbest/swish-triton-GB10.py
  ...

tasks/modal/                    ← Modal H100 ownbest (populated by rsi/modal_run.py)
  swish/ownbest/swish-triton-H100.py
  ...

runs/run-NNN/
  context.md                    ← one-line-per-gen history for feedback agent
  run.log                       ← full stdout+stderr log of the run
  train.jsonl                   ← curator training pairs (correctness + performance)
  gen-N/
    target_agent.py             ← script written by meta/feedback agent
    sol.py                      ← Triton kernel written by target agent
    results.json                ← eval output (Tensara or local)

Quickstart

1. Clone and set environment variables

git clone https://github.com/whatdhack/recursive-self-improvement
cd recursive-self-improvement
export NEBIUS_API_KEY="..."
export TENSARA_API_KEY="..."
export TENSARA_USERID="..."

2. Install (via miniforge3)

mamba create -n rsi python=3.13 -y
mamba activate rsi
pip install -r requirements.txt

3. Run a single problem (API-only, no local training)

python -m rsi run \
  --problem matrix-vector \
  --meta-agent-profile profiles/kimi26-nebius.json \
  --target-agent-profile profiles/nemotron-nebius.json \
  --curator-profile profiles/curator-nebius.json \
  --provider providers/nebius.json \
  --max-gen 10 \
  --run-id 001 \
  --run-dir ~/ai26/rsi_runs/runs

Each run appends to runs/run-001/context.md so the feedback agent builds on prior generations. Logs are saved to runs/run-001/run.log.

If the task directory for a problem doesn't exist, it is auto-scaffolded on first run: loop.py calls the Tensara API to fetch the problem description, starter code, and leaderboard best, then copies the generic template files from tasks/gpu_kernel_task/.

Interrupted runs resume automatically — gens with an existing results.json are skipped and state is restored for the feedback agent.

4. Run on Modal (recommended — no rate limits, one H100 cold start)

Modal runs the entire loop inside a single H100 container: meta agent, target agent, and all evaluations. No Tensara API calls during optimization; the best result is submitted to Tensara at the end.

pip install modal
modal setup   # browser auth

# Create the secret with your API keys (one-time)
modal secret create rsi-api-keys \
    NEBIUS_API_KEY=<your-key> \
    TENSARA_API_KEY=<your-key> \
    TENSARA_USERID=<your-username>

# Single problem, 3 gens
modal run rsi/modal_run.py --problem swish --max-gen 3 --submit

# Multiple problems, 5 gens × 5 repeats (each repeat seeds from previous ownbest)
modal run rsi/modal_run.py \
  --command batch \
  --problems swish,gelu,rms-norm,l2-norm,log-softmax,cosine-similarity \
  --batch-id batch003 --max-gen 5 --repeat 5 --submit

Results persist in a Modal Volume (rsi-runs). Download after a run:

modal volume get rsi-runs runs/          # run directories
modal volume get rsi-runs tasks/local/  # best kernels found

5. Run a batch of problems

python -m rsi batch \
  --problems rms-norm,gelu,swish,l2-norm,log-softmax,cosine-similarity \
  --max-gen 3 \
  --batch-id batch001 \
  --provider providers/nebius.json \
  --meta-agent-profile profiles/kimi26-nebius.json \
  --target-agent-profile profiles/nemotron-nebius.json \
  --submit \
  --run-dir ~/ai26/rsi_runs/runs

Progress is saved to runs/batch-batch001/progress.json after each problem. If the run is interrupted by a rate limit (exit 2), re-run the same command — it picks up from where it left off.

By default, the curator also fires on ACCEPTED solutions below 85% of the leaderboard target GFLOPS. Tune with --curate-perf-threshold (0.0 to disable, 1.0 to always curate accepted):

  --curate-perf-threshold 0.95   # curate anything below 95% of leaderboard target

6. Run with LoRA training on CPU

# First provision a CPU droplet:
bash setup/droplet_setup.sh

# Then run with the local base model:
python -m rsi run \
  --problem matrix-vector \
  --meta-agent-profile profiles/kimi26-nebius.json \
  --target-agent-profile profiles/nemotron-nebius.json \
  --curator-profile profiles/curator-nebius.json \
  --provider providers/nebius.json \
  --base-model /opt/models/google_gemma-4-1b-it-Q4_K_M.gguf \
  --llama-bin-dir /opt/llama.cpp/build/bin \
  --threads 8 \
  --max-gen 10 \
  --run-id 001 \
  --run-dir ~/ai26/rsi_runs/runs

Environment variables

Variable	Used by	Description
`NEBIUS_API_KEY`	`providers/nebius.json`	Nebius Token Factory API key (`v1....`)
`TENSARA_API_KEY`	`evaluate.py`	Tensara platform key
`TENSARA_USERID`	`tensara_client.py`	Username for leaderboard submission

CLI reference

Flag	Default	Description
`--problem`	`matrix-vector`	Tensara problem slug
`--task`	(= `tensara/<problem>`)	Task directory under `tasks/` (auto-scaffolded under `tasks/tensara/` if missing)
`--provider`	`providers/do.json`	Provider config (endpoint + auth env var)
`--meta-agent-profile`	`profiles/kimi26-do.json`	Meta/feedback agent profile
`--target-agent-profile`	`profiles/nemotron-do.json`	Target agent profile
`--curator-profile`	`profiles/curator-do.json`	Curator profile
`--max-gen`	`10`	Number of generations to run
`--run-id`	`001`	Run identifier (creates `runs/run-<id>/`)
`--run-dir`	`./runs`	Root directory for run output
`--gpu-type`	`H100`	GPU type for Tensara benchmark
`--curate-perf-threshold`	`0.85`	Curate ACCEPTED solutions below this fraction of leaderboard target GFLOPS
`--submit`	off	Submit to Tensara leaderboard when ACCEPTED and beats personal best
`--local-eval`	off	Use `evaluate_local.py` instead of Tensara API (requires local GPU)
`--modal-eval`	off	Use `evaluate_local.py --modal` (each eval spawns a Modal H100 — use `modal_run.py` instead for efficiency)
`--base-model`	—	Path to Gemma4-1B GGUF for LoRA training
`--llama-bin-dir`	`./llama.cpp/build/bin`	Directory with `llama-finetune`, `llama-export-lora`, `llama-server`
`--local-curator`	off	Use local llama-server for curation instead of cloud API
`--threads`	`8`	CPU threads for llama.cpp
`--lora-rank`	`16`	Initial LoRA rank

`batch` command flags

Flag	Default	Description
`--problems`	(required)	Comma-separated Tensara problem slugs
`--max-gen`	`3`	Generations per problem
`--batch-id`	`batch001`	Identifies the batch; progress stored under `runs/batch-<id>/`
`--run-dir`	`./runs`	Root directory for run output
`--provider`	`providers/nebius.json`	Provider config
`--meta-agent-profile`	`profiles/kimi26-nebius.json`	Meta/feedback agent
`--target-agent-profile`	`profiles/nemotron-nebius.json`	Target agent
`--curator-profile`	`profiles/curator-nebius.json`	Curator
`--gpu-type`	`H100`	GPU type for Tensara
`--submit`	off	Submit accepted solutions to leaderboard
`--local-eval`	off	Use local GPU evaluator (no Tensara API)
`--modal-eval`	off	Use Modal H100 evaluator (no Tensara API)
`--curate-perf-threshold`	`0.85`	Performance curation threshold

`modal run rsi/modal_run.py` flags

Flag	Default	Description
`--command`	`run`	`run` (single problem) or `batch` (multiple)
`--problem`	`swish`	Problem slug (for `run`)
`--problems`	—	Comma-separated slugs (for `batch`)
`--run-id`	`001`	Run identifier (for `run`)
`--batch-id`	`batch001`	Batch identifier (for `batch`)
`--max-gen`	`3`	Generations per problem per repeat
`--repeat`	`1`	How many times to run the batch, each seeding from the previous ownbest
`--submit`	off	Submit best local ownbest to Tensara after the final repeat
`--gpu-type`	`H100`	GPU type label (informational)
`--provider`	`providers/nebius.json`	Provider config
`--meta-agent-profile`	`profiles/kimi26-nebius.json`	Meta/feedback agent
`--target-agent-profile`	`profiles/nemotron-nebius.json`	Target agent

Requirements

Python 3.13+
openai Python package
Nebius API key (or another OpenAI-compatible provider)
Tensara API key
(for LoRA training) CPU machine with llama.cpp built (llama-finetune, llama-export-lora)

What makes it recursive?

Two self-improvement loops run simultaneously:

Harness loop (every gen): The feedback agent reads its own execution history and rewrites the agent script that drives code generation. The strategy improves.

Weight loop (every gen, when enabled): The curator converts failures and performance shortfalls to training pairs; LoRA fine-tunes Gemma4-1B on those pairs — improving the curator's ability to generate better training data in future gens. This loop is self-contained: it does not feed back into the feedback agent or target agent in the current design.

The harness loop is the primary driver of per-run improvement. The weight loop is a longer-horizon investment: a better curator produces higher-quality training pairs, but those pairs would only improve kernel quality if the target agent model itself were also fine-tunable — which requires a smaller local model in place of the current API-hosted Nemotron-550b.

This is an early-stage implementation of Recursive Self-Improvement — systems that bootstrap their own capabilities from their own failures, with no human in the loop after generation 0.

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 41 Commits
profiles		profiles
providers		providers
rsi		rsi
setup		setup
tasks		tasks
.env.example		.env.example
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
make_video.py		make_video.py
requirements.txt		requirements.txt
rsi_explainer.mp4		rsi_explainer.mp4

Navigation Menu

Repository files navigation

recursive-self-improvement

What it does

Execution flow in detail

Step 1 — Meta agent (gen 0) and Feedback agent (gen 1+)

Step 2 — Target agent (`rsi/loop.py` → subprocess)

Step 3 — Evaluation

Step 4 — Curation (`rsi/curate.py`)

Step 5 — LoRA training (`rsi/train.py`)

Architecture

Quickstart

1. Clone and set environment variables

2. Install (via miniforge3)

3. Run a single problem (API-only, no local training)

4. Run on Modal (recommended — no rate limits, one H100 cold start)

5. Run a batch of problems

6. Run with LoRA training on CPU

Environment variables

CLI reference

`batch` command flags

`modal run rsi/modal_run.py` flags

Requirements

What makes it recursive?

License

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

recursive-self-improvement

What it does

Execution flow in detail

Step 1 — Meta agent (gen 0) and Feedback agent (gen 1+)

Step 2 — Target agent (rsi/loop.py → subprocess)

Step 3 — Evaluation

Step 4 — Curation (rsi/curate.py)

Step 5 — LoRA training (rsi/train.py)

Architecture

Quickstart

1. Clone and set environment variables

2. Install (via miniforge3)

3. Run a single problem (API-only, no local training)

4. Run on Modal (recommended — no rate limits, one H100 cold start)

5. Run a batch of problems

6. Run with LoRA training on CPU

Environment variables

CLI reference

batch command flags

modal run rsi/modal_run.py flags

Requirements

What makes it recursive?

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Step 2 — Target agent (`rsi/loop.py` → subprocess)

Step 4 — Curation (`rsi/curate.py`)

Step 5 — LoRA training (`rsi/train.py`)

`batch` command flags

`modal run rsi/modal_run.py` flags