CodeK

CodeK is a research repo for testing whether Karpathy-style code clarity improves software repair performance on SWE-bench.

The working project plan currently lives in codek_v1_plan.md. The original SWE-bench rewrite plan has been archived to karpathy_swebench_plan_archived.md.

The working dataset-size target is about 10,000 rows across all final splits. See docs/dataset_strategy.md.

Status

See status.md for the full phase breakdown and checkpoint.md for the milestone log.

• Phase 1 (Extract & Codify): complete
• Phase 2.1 (Synthetic generation): complete — 201/201 rows in data/seeds/synthetic_karpathy.jsonl
• Phase 2.2+ (old SWE-bench rewrites): REPLACED by new augmentation plan (see codek_v1_plan.md)
• Phase 2 new (Augmentation passes 1–3): not started

Repository Layout

CodeK/
├── README.md
├── status.md                           # current phase status
├── checkpoint.md                       # milestone log
├── codek_v1_plan.md                    # active project plan
├── karpathy_swebench_plan_archived.md  # original SWE-bench rewrite plan (archived)
├── karpathy_style_codex.md             # generated style guide (Phase 1 output)
├── CODEX.md                            # agent operating rules
├── TASKS.md                            # atomic task checklist
├── docs/
├── config/                             # task catalogs, path config, augmentation config
├── scripts/                            # pipeline entrypoints
├── data/
│   ├── seeds/                          # tracked seed data
│   │   └── synthetic_karpathy.jsonl   # 201-row seed batch (tracked)
│   ├── augmented/                      # augmentation pass outputs (untracked)
│   └── splits/                         # train/val/test splits (untracked)
├── training/                           # training scripts and configs
├── eval/                               # evaluation scripts and outputs
├── logs/                               # run logs (untracked)
├── dataset/                            # packaged HuggingFace dataset (untracked)
├── models/                             # LoRA checkpoints (untracked)
├── predictions/                        # SWE-bench prediction JSONL (untracked)
└── results/                            # evaluation outputs and analysis

Notes

• Large generated assets and checkpoints stay out of git; final datasets publish to Hugging Face.
• See CODEX.md for agent operating rules before making changes.