Skip to content

Man1Lab v1.0.0 (Research Prototype)

Pre-release
Pre-release

Choose a tag to compare

View all tags
@maniac1um maniac1um released this 29 Jun 02:43
v1.0.0
56c4638

Man1Lab v1.0.0 (Research Prototype)

Release date: 2026-06-29
Repository: https://github.com/maniac1um/Man1Lab

Project Overview

Man1Lab is a research prototype that automates the engineering workflow for reproducing a machine learning paper from PDF. The system reads a paper, plans implementation tasks, generates a Python reproduction repository, installs dependencies, runs the training script, verifies the outcome, reviews failures with an LLM, and writes a final report.

This is the first public release of the MVP. It demonstrates a complete autonomous pipeline with real LLM integration. It does not guarantee successful end-to-end training on arbitrary papers.

Highlights

  • Full pipeline — Reader through Reporter runs without manual intervention
  • Real LLM integration — OpenAI-compatible API (tested with DeepSeek)
  • Deterministic routing — TaskRouter maps engineering tasks to repository files
  • Repository Acceptance Gate (RAG) — Rejects structurally defective repositories before execution
  • 126 unit tests — Agents, services, coder quality, and acceptance gate covered
  • Documented architecture — ADRs, capability reference, milestone review archive

Implemented Capabilities

Stage Component Output
Ingestion Reader PaperModel
Planning Planner TaskModel
Code generation Coder Workspace
Execution Runner ExecutionResult
Verification VerificationService VerificationResult
Review Reviewer ReviewReport
Patch planning PatchPlanner PatchPlan
Reporting Reporter ReportModel

The orchestrator schedules all stages. Agents do not communicate directly; they pass typed Pydantic artifacts.

Repository Acceptance Gate

The RAG is the final step inside Coder.run(). Before Runner executes, the generated repository must pass:

Check What it catches
Import closure Third-party imports missing from requirements.txt
Framework binding Mixed or forbidden framework imports
Internal imports Scripts importing symbols not in the interface registry
Training entrypoint Missing or empty scripts/train.py

When rejected, Coder raises RepositoryAcceptanceError and Runner is not invoked. Warnings (config drift, README issues) do not block acceptance.

On the DeiT benchmark re-run, RAG accepted a repository with complete requirements.txt (torch, timm, torchvision, PyYAML, tqdm). Execution then failed at a timm runtime API issue — outside RAG scope.

Details: docs/reviews/repository_acceptance_gate/implementation_review.md

Benchmark Summary

Runs used scripts/run_integration_m7_1.py with DeepSeek API.

Benchmark Paper Result
M8.1 ResNet (1512.03385v1.pdf) Pipeline complete; execution failed (No module named 'torch') — pre-GQ-1
M8.2 DeiT (2012.12877v2.pdf) Pipeline complete; stub requirements.txt — pre-RAG
RAG DeiT (re-run) RAG ACCEPTED; execution failed (timm _pil_interp runtime error)

Takeaway: The pipeline reliably completes on real papers. Delivery defects (missing deps, broken imports) are caught at Coder. Successful training reproduction on benchmark papers remains an open goal.

Reports: M8.1 · M8.2

Known Limitations

  1. No guaranteed reproduction — Training may fail due to runtime, paper-specific, or LLM generation issues
  2. Review loop not closed — PatchPlan is produced but Coder/Runner are not re-invoked
  3. Framework coverage — Binding profiles for PyTorch, TensorFlow, JAX, Caffe only
  4. External API dependency — Reviewer can fail on LLM timeouts unrelated to code quality
  5. Research prototype — Single maintainer; pull requests not accepted (issues welcome)

Full list: docs/CURRENT_STATUS.md

Future Work

Area Direction
v1.1 Close review loop — re-run Coder/Runner on PatchPlan
v1.2 GitHub repository initialization for generated workspaces
v1.3 Multi-model LLM provider support
v2.0 Memory, human-in-the-loop, multi-agent collaboration

Roadmap: docs/roadmap/ROADMAP.md

Quick Start

git clone https://github.com/maniac1um/Man1Lab.git
cd Man1Lab
pip install -r requirements.txt
PYTHONPATH=. python -m pytest tests/ -v
PYTHONPATH=. python app.py

Set PAPER_PATH to a PDF. Copy .env.example to .env for real LLM calls.

Guide: docs/GETTING_STARTED.md

Documentation

Resource Link
Current status docs/CURRENT_STATUS.md
Architecture docs/architecture/ARCHITECTURE.md
Changelog CHANGELOG.md
Contributing CONTRIBUTING.md

Citation

@software{man1lab_2026,
  author       = {maniac1um},
  title        = {Man1Lab: An Autonomous Research Paper Reproduction Pipeline},
  year         = {2026},
  version      = {1.0.0},
  url          = {https://github.com/maniac1um/Man1Lab}
}
Assets 2
Loading