Agent-Native Research Artifacts (ARA) — Benchmark Collection

This repository collects 32 Agent-Native Research Artifacts (ARAs): a machine-readable, reproducibility-first alternative to the PDF paper. Each ARA decomposes a research paper into four grounded layers so that both humans and agents can read the claims, run the code, inspect the evidence, and replay how the research was actually explored (dead ends included).

Anatomy of an ARA

Every artifact lives at artifacts/<benchmark>/<name>/ and contains:

Layer	Path	What it holds
Summary	PAPER.md	Human-readable overview of the paper.
Cognitive	logic/	claims.md, concepts.md, experiments.md, problem.md, related_work.md, and solution/ (algorithm.md, architecture.md, constraints.md, heuristics.md). Claims carry falsification criteria and proof pointers.
Artifact	src/	Runnable code, configs, and environment.md.
Evidence	evidence/	figures/ and tables/, each tied back to the claims it supports.
Trajectory	trajectory.html	Self-contained interactive viewer: a clickable process map (left) plus a per-step drill-down (right) linking each step to its claim, grounded result, and code. Open in a browser.
Trace	trace/	exploration_tree.yaml (the research DAG source) and exploration_tree.html (a tree-only view).

How to browse

• Read any artifacts/<benchmark>/<name>/PAPER.md for the overview, then drill into logic/ and evidence/.
• Open an artifact's trajectory.html in a browser to replay the research process step by step: a process map on the left (dead-end branches included) and a per-step drill-down on the right (what the step did, its linked claim, the grounded result, and the code pointer). (GitHub shows the HTML source inline; clone the repo or download the file to view it rendered.)

Artifacts

Paperbench (23)

Artifact	Title	Trajectory
adaptive-pruning	APT: Adaptive Pruning and Tuning of Pretrained Language Models for Efficient Training and Inference	view
all-in-one	All-in-one Simulation-Based Inference (Simformer)	view
bam	Batch and Match: Black-Box Variational Inference with a Score-Based Divergence	view
bbox	BBox-Adapter: Lightweight Adapting for Black-Box Large Language Models	view
bridging-data-gaps	Efficient Transfer Learning in Diffusion Models via Adversarial Noise	view
fre	Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodings	view
ftrl	Fine-Tuning RL Models is Secretly a Forgetting Mitigation Problem	view
lbcs	Refined Coreset Selection: Minimal Coreset Size under Model Performance Constraints	view
lca-on-the-line	LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies	view
mechanistic-understanding	A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO	view
pinn	Challenges in Training PINNs: A Loss Landscape Perspective	view
rice	RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation	view
robust-clip	Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Vision-Language Models	view
sample-specific-masks	Sample-specific Masks for Visual Reprogramming-based Prompting	view
sapg	SAPG: Split and Aggregate Policy Gradients	view
self-composing-policies	Self-Composing Policies for Scalable Continual Reinforcement Learning	view
self-expansion	Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning	view
semantic-self-consistency	Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting	view
sequential-neural-score-estimation	Sequential Neural Posterior Score Estimation (NPSE)	view
stay-on-topic-with-classifier-free-guidance	Stay on Topic with Classifier-Free Guidance	view
stochastic-interpolants	Stochastic Interpolants with Data-Dependent Couplings	view
test-time-model-adaptation	Test-Time Model Adaptation with Only Forward Passes (FOA)	view
what-will-my-model-forget	What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement	view

ReBench (5)

Artifact	Title	Trajectory
rebench-fix_embedding	Fix Embedding (RE-Bench task)	view
rebench-nanogpt_chat_rl	nanoGPT Chat RL (RE-Bench task)	view
rebench-restricted_mlm	Restricted-Architecture MLM (RE-Bench task)	view
rebench-rust_codecontests	Rust CodeContests Inference (RE-Bench task)	view
rebench-triton_cumsum	Triton Cumsum Kernel (RE-Bench task)	view

Speedrun (1)

Artifact	Title	Trajectory
nanogpt-speedrun	NanoGPT Speedrun	view

Extra (3)

Artifact	Title	Trajectory
andes	Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Services	view
venn	Venn: Resource Management for Collaborative Learning Jobs	view
expbench	EXP-Bench: Can AI Conduct AI Research Experiments?	view

Provenance & quality

Each ARA was compiled from a source paper and its accompanying code; the content is grounded to the source paper (claims, figures, and tables trace back to the original). A subset additionally passed SEAL structural and cross-layer validation. These artifacts are reconstructions for research and evaluation purposes; consult the original papers and their licenses for authoritative results.

Benchmarks

The paperbench, rebench, and speedrun artifacts correspond to the ARA evaluation benchmark. The extra artifacts are additional complete ARAs beyond the benchmark set.

License

Content is released under Creative Commons Attribution 4.0 International (CC BY 4.0).