Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -199,3 +199,4 @@ assets/wheels/vllm*.whl
# DCP artifacts
model_state_dict/
forge_dcp_tmp/
demo_top_down.md
58 changes: 10 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# <img width="35" height="35" alt="image" src="https://github.com/user-attachments/assets/2700a971-e5d6-4036-b03f-2f89c9791609" /> Forge


#### A PyTorch native agentic library for RL post-training and agentic development
#### A PyTorch native agentic library for RL post-training and agentic development that lets you focus on algorithms instead of writing infra code.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure you want to make this too long

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to add nod to infra in headline but will leave it to your call, we can revert if you feel old was better

Copy link

@HamidShojanazeri HamidShojanazeri Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe "Forge — a PyTorch library that abstracts infrastructure to simplify RL Post-training" or "a PyTorch library for streamlined agentic RL post-training, with clear abstractions and built-in infra." cc @dcci

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Hamid. Personally I want to choose first one, Davide any opinions please?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the abstract infra language personally - our audience is RL researchers with a goal of enabling them to quickly explore the space. Not necessarily make people infra experts.

Copy link
Contributor

@jspisak jspisak Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also note that we will have an agentic API so I don't want to tie it too much to just post training which is why i worded the original language the way I did.


## Overview
Forge was built with one core principle in mind: researchers should write algorithms, not infrastructure. Forge introduces a “service”-centric architecture that provides the right abstractions for distributed complexity. When you need fine-grained control over placement, fault handling or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm.
The primary purpose of the Forge ecosystem is to delineate infra concerns from model concerns thereby making RL experimentation easier. Forge delivers this by providing clear RL abstractions and one scalable implementation of these abstractions. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm.

Key features:
- Usability for rapid research (isolating the RL loop from infrastructure)
Expand All @@ -18,15 +18,19 @@ Key features:
> work. It's recommended that you signal your intention to contribute in the
> issue tracker, either by filing a new issue or by claiming an existing one.

## 📖 Documentation
## 📖 Documentation (Coming Soon)

View Forge's hosted documentation [at this link](https://meta-pytorch.org/forge/).
View Forge's hosted documentation (coming soon)

## Tutorials

You can also find our notebook tutorials (coming soon)

## Installation

### Basic

Forge requires the latest PyTorch nightly with Monarch, vLLM, and torchtitan. For convenience,
Forge requires the latest PyTorch nightly with [Monarch](https://github.com/meta-pytorch/monarch), [vLLM](https://github.com/vllm-project/vllm), and [torchtitan](https://github.com/pytorch/torchtitan). For convenience,
we have pre-packaged these dependencies as wheels in assets/wheels. (Note that the basic install script
uses [DNF](https://docs.fedoraproject.org/en-US/quick-docs/dnf/), but could be easily extended to other Linux OS.)

Expand All @@ -40,7 +44,7 @@ conda activate forge

Optional: By default, the packages installation uses conda. If user wants to install system packages on the target machine instead of conda, they can pass the `--use-sudo` to the installation script: `./script/install.sh --use-sudo`.

After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 3 GPU devices).
After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 3 GPU devices):

```
python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml
Expand All @@ -56,48 +60,6 @@ For your information, since the vLLM wheel is too large for GitHub, we uploaded
$ gh release create v0.0.0 assets/wheels/vllm-*.whl --title "Forge Wheels v0.0.0"
```

### Meta Internal Build (Alternative Route)

1. Build uv package

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/pytorch-labs/forge
cd forge
uv sync --all-extras
source .venv/bin/activate
```

2. Setup CUDA on local machine

```bash
# feature install if you don't have /user/local/cuda-12.8
feature install --persist cuda_12_9

# add env variables
export CUDA_VERSION=12.9
export NVCC=/usr/local/cuda-$CUDA_VERSION/bin/nvcc
export CUDA_NVCC_EXECUTABLE=/usr/local/cuda-$CUDA_VERSION/bin/nvcc
export CUDA_HOME=/usr/local/cuda-$CUDA_VERSION
export PATH="$CUDA_HOME/bin:$PATH"
export CUDA_INCLUDE_DIRS=$CUDA_HOME/include
export CUDA_CUDART_LIBRARY=$CUDA_HOME/lib64/libcudart.so
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH
```

3. Build vllm from source

```bash
git clone https://github.com/vllm-project/vllm.git --branch v0.10.0
cd vllm
python use_existing_torch.py
uv pip install -r requirements/build.txt
uv pip install --no-build-isolation -e .
```

> [!WARNING]
> If you add packages to the pyproject.toml, use `uv sync --inexact` so it doesn't remove Monarch and vLLM

## Quick Start

To run SFT for Llama3 8B, run
Expand Down
Loading