Home

Next-Gen Architecture Lab

Architecture search for language models: controlled component experiments, distributed pretraining, frontier-model adaptation, and reinforcement learning.

Start here

Your goal	Read
Understand the research	Research Program
Find the code to change	Code Map
Design a comparison	Experiment Design
Prepare or resume a run	Runbook
Compare Limite normal and 2-simplicial	Limite SFT comparison
Understand the evidence	Results
Join the project	Contributing

Research tracks

Track	Examples
Components	Attention, recurrent mixers, memory, residual routing, positional encoding
Models and scale	Dense and MoE Qwen families; DeepSeek scratch models
Pretrained adaptation	Ordinary and simplicial branches; frozen-backbone and full-weight fine-tuning
RL	Verifiable math rewards, policy replay, generation efficiency, capability retention

Where information belongs

• Wiki: navigation, methods, decisions, and concise experiment summaries.
• Repository: executable code, recipes, tests, and versioned evidence.
• Run artifacts and MLflow: checkpoint identities, detailed metrics, and live progress.

Start with the repository. A dated report is a snapshot; check run receipts for current state.