The primary purpose of the Forge ecosystem is to delineate infra concerns from model concerns thereby making RL experimentation easier. Forge delivers this by providing clear RL abstractions and one scalable implementation of these abstractions. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm.
Key features:
- Usability for rapid research (isolating the RL loop from infrastructure)
- Hackability for power users (all parts of the RL loop can be easily modified without interacting with infrastructure)
- Scalability (ability to shift between async and synchronous training and across thousands of GPUs)
⚠️ Early Development Warning Forge is currently in an experimental stage. You should expect bugs, incomplete features, and APIs that may change in future versions. The project welcomes bugfixes, but to make sure things are well coordinated you should discuss any significant change before starting the work. It's recommended that you signal your intention to contribute in the issue tracker, either by filing a new issue or by claiming an existing one.
View Forge's hosted documentation (coming soon)
You can also find our notebook tutorials (coming soon)
Forge requires the latest PyTorch nightly with Monarch, vLLM, and torchtitan. For convenience, we have pre-packaged these dependencies as wheels in assets/wheels. (Note that the basic install script uses DNF, but could be easily extended to other Linux OS.)
Forge requires the Github CLI (gh) to download a compatible vLLM package. See here for gh install instructions before continuting. Please login to gh with your Github account before continuing with gh auth login. You may use either https or ssh as the protocol for authentication.
conda create -n forge python=3.10
conda activate forge
./scripts/install.shOptional: By default, the packages installation uses conda. If user wants to install system packages on the target machine instead of conda, they can pass the --use-sudo to the installation script: ./script/install.sh --use-sudo.
After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 3 GPU devices):
python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml
If you need to re-build the wheels for whatever reason, you can do so with:
./scripts/build_wheels.shFor your information, since the vLLM wheel is too large for GitHub, we uploaded it as a release in the install.sh script:
$ gh release create v0.0.0 assets/wheels/vllm-*.whl --title "Forge Wheels v0.0.0"
To run SFT for Llama3 8B, run
uv run forge download meta-llama/Meta-Llama-3.1-8B-Instruct --output-dir /tmp/Meta-Llama-3.1-8B-Instruct --ignore-patterns "original/consolidated.00.pth"
uv run forge run --nproc_per_node 2 apps/sft/main.py --config apps/sft/llama3_8b.yamlSource code is made available under a BSD 3 license, however you may have other legal obligations that govern your use of other content linked in this repository, such as the license or terms of service for third-party data and models.
