From 8c6befa4684cb3e0990fb4486c636394783a26a5 Mon Sep 17 00:00:00 2001 From: Sanyam Bhutani Date: Tue, 30 Sep 2025 13:02:31 -0700 Subject: [PATCH 1/8] Update README.md --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 036403b51..83b998185 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ # image Forge -#### A PyTorch native agentic library for RL post-training and agentic development +#### A PyTorch native agentic library for RL post-training and agentic development that lets you focus on algorithms instead of writing infra code. ## Overview -Forge was built with one core principle in mind: researchers should write algorithms, not infrastructure. Forge introduces a “service”-centric architecture that provides the right abstractions for distributed complexity. When you need fine-grained control over placement, fault handling or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm. +Forge was built with one core principle in mind: researchers should write algorithms, not infrastructure. Forge introduces a “service”-centric architecture that provides the right abstractions for distributed complexity and workloads. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm. Key features: - Usability for rapid research (isolating the RL loop from infrastructure) @@ -18,15 +18,15 @@ Key features: > work. It's recommended that you signal your intention to contribute in the > issue tracker, either by filing a new issue or by claiming an existing one. -## 📖 Documentation +## 📖 Documentation (Coming Soon) -View Forge's hosted documentation [at this link](https://meta-pytorch.org/forge/). +View Forge's hosted documentation [at this link (coming soon)](https://meta-pytorch.org/forge/). ## Installation ### Basic -Forge requires the latest PyTorch nightly with Monarch, vLLM, and torchtitan. For convenience, +Forge requires the latest PyTorch nightly with [Monarch](https://github.com/meta-pytorch/monarch), [vLLM](https://pytorch.org/blog/pytorch-vllm-♥%EF%B8%8F/), and [torchtitan](https://github.com/pytorch/torchtitan). For convenience, we have pre-packaged these dependencies as wheels in assets/wheels. (Note that the basic install script uses [DNF](https://docs.fedoraproject.org/en-US/quick-docs/dnf/), but could be easily extended to other Linux OS.) @@ -40,7 +40,7 @@ conda activate forge Optional: By default, the packages installation uses conda. If user wants to install system packages on the target machine instead of conda, they can pass the `--use-sudo` to the installation script: `./script/install.sh --use-sudo`. -After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 3 GPU devices). +After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 3 GPU devices), right now this will run Full Fine-Tuning: ``` python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml From 64b9ab5c5428215025f1c51d5895d55fc436ed4f Mon Sep 17 00:00:00 2001 From: Sanyam Bhutani Date: Tue, 30 Sep 2025 13:04:45 -0700 Subject: [PATCH 2/8] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 83b998185..b52d7530c 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,8 @@ #### A PyTorch native agentic library for RL post-training and agentic development that lets you focus on algorithms instead of writing infra code. +> Note: We also support Supervised Fine-Tuning and what you expect from [TorchTune](https://github.com/meta-pytorch/torchtune/issues/2883) + ## Overview Forge was built with one core principle in mind: researchers should write algorithms, not infrastructure. Forge introduces a “service”-centric architecture that provides the right abstractions for distributed complexity and workloads. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm. From d366ab4cf78ad8a0fddb8c8b85a9b038cb53d820 Mon Sep 17 00:00:00 2001 From: Sanyam Bhutani Date: Tue, 30 Sep 2025 13:06:25 -0700 Subject: [PATCH 3/8] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b52d7530c..519b90c83 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ #### A PyTorch native agentic library for RL post-training and agentic development that lets you focus on algorithms instead of writing infra code. -> Note: We also support Supervised Fine-Tuning and what you expect from [TorchTune](https://github.com/meta-pytorch/torchtune/issues/2883) +> Note: We also support Supervised Fine-Tuning and what you might expect from [TorchTune](https://github.com/meta-pytorch/torchtune/issues/2883) ## Overview Forge was built with one core principle in mind: researchers should write algorithms, not infrastructure. Forge introduces a “service”-centric architecture that provides the right abstractions for distributed complexity and workloads. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm. From 39ffb8b5879b1e815bb29c3c725d81a76f479b1c Mon Sep 17 00:00:00 2001 From: Sanyam Bhutani Date: Tue, 30 Sep 2025 13:07:07 -0700 Subject: [PATCH 4/8] Update README.md --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 519b90c83..95c1bb452 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,10 @@ Key features: View Forge's hosted documentation [at this link (coming soon)](https://meta-pytorch.org/forge/). +## WIP + +You can also find our notebook tutorials below (coming soon) + ## Installation ### Basic From a26c39f0179a76c858b9ccb6874b88506c7337f1 Mon Sep 17 00:00:00 2001 From: Sanyam Bhutani Date: Tue, 30 Sep 2025 13:22:04 -0700 Subject: [PATCH 5/8] Update README.md --- README.md | 42 ------------------------------------------ 1 file changed, 42 deletions(-) diff --git a/README.md b/README.md index 95c1bb452..8ff7aefb7 100644 --- a/README.md +++ b/README.md @@ -62,48 +62,6 @@ For your information, since the vLLM wheel is too large for GitHub, we uploaded $ gh release create v0.0.0 assets/wheels/vllm-*.whl --title "Forge Wheels v0.0.0" ``` -### Meta Internal Build (Alternative Route) - -1. Build uv package - -```bash -curl -LsSf https://astral.sh/uv/install.sh | sh -git clone https://github.com/pytorch-labs/forge -cd forge -uv sync --all-extras -source .venv/bin/activate -``` - -2. Setup CUDA on local machine - -```bash -# feature install if you don't have /user/local/cuda-12.8 -feature install --persist cuda_12_9 - -# add env variables -export CUDA_VERSION=12.9 -export NVCC=/usr/local/cuda-$CUDA_VERSION/bin/nvcc -export CUDA_NVCC_EXECUTABLE=/usr/local/cuda-$CUDA_VERSION/bin/nvcc -export CUDA_HOME=/usr/local/cuda-$CUDA_VERSION -export PATH="$CUDA_HOME/bin:$PATH" -export CUDA_INCLUDE_DIRS=$CUDA_HOME/include -export CUDA_CUDART_LIBRARY=$CUDA_HOME/lib64/libcudart.so -export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH -``` - -3. Build vllm from source - -```bash -git clone https://github.com/vllm-project/vllm.git --branch v0.10.0 -cd vllm -python use_existing_torch.py -uv pip install -r requirements/build.txt -uv pip install --no-build-isolation -e . -``` - -> [!WARNING] -> If you add packages to the pyproject.toml, use `uv sync --inexact` so it doesn't remove Monarch and vLLM - ## Quick Start To run SFT for Llama3 8B, run From 4f36c9966fb316d0ffaf94d61b55faf8d2742d48 Mon Sep 17 00:00:00 2001 From: Sanyam Bhutani Date: Tue, 30 Sep 2025 13:28:13 -0700 Subject: [PATCH 6/8] remove doc links --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8ff7aefb7..e9467caf0 100644 --- a/README.md +++ b/README.md @@ -22,17 +22,17 @@ Key features: ## 📖 Documentation (Coming Soon) -View Forge's hosted documentation [at this link (coming soon)](https://meta-pytorch.org/forge/). +View Forge's hosted documentation (coming soon) -## WIP +## Tutorials -You can also find our notebook tutorials below (coming soon) +You can also find our notebook tutorials (coming soon) ## Installation ### Basic -Forge requires the latest PyTorch nightly with [Monarch](https://github.com/meta-pytorch/monarch), [vLLM](https://pytorch.org/blog/pytorch-vllm-♥%EF%B8%8F/), and [torchtitan](https://github.com/pytorch/torchtitan). For convenience, +Forge requires the latest PyTorch nightly with [Monarch](https://github.com/meta-pytorch/monarch), [vLLM](https://github.com/vllm-project/vllm), and [torchtitan](https://github.com/pytorch/torchtitan). For convenience, we have pre-packaged these dependencies as wheels in assets/wheels. (Note that the basic install script uses [DNF](https://docs.fedoraproject.org/en-US/quick-docs/dnf/), but could be easily extended to other Linux OS.) From 527732b97899a5a8cf485735926d469dfcae8846 Mon Sep 17 00:00:00 2001 From: Sanyam Bhutani Date: Tue, 30 Sep 2025 14:34:52 -0700 Subject: [PATCH 7/8] Update README.md --- README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/README.md b/README.md index e9467caf0..8283719f5 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,6 @@ #### A PyTorch native agentic library for RL post-training and agentic development that lets you focus on algorithms instead of writing infra code. -> Note: We also support Supervised Fine-Tuning and what you might expect from [TorchTune](https://github.com/meta-pytorch/torchtune/issues/2883) - ## Overview Forge was built with one core principle in mind: researchers should write algorithms, not infrastructure. Forge introduces a “service”-centric architecture that provides the right abstractions for distributed complexity and workloads. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm. @@ -46,7 +44,7 @@ conda activate forge Optional: By default, the packages installation uses conda. If user wants to install system packages on the target machine instead of conda, they can pass the `--use-sudo` to the installation script: `./script/install.sh --use-sudo`. -After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 3 GPU devices), right now this will run Full Fine-Tuning: +After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 3 GPU devices): ``` python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml From 7fddc39532dbd04bdf2fba550f5fc78d9d8cd718 Mon Sep 17 00:00:00 2001 From: Sanyam Bhutani Date: Tue, 30 Sep 2025 15:02:26 -0700 Subject: [PATCH 8/8] add --- .gitignore | 1 + README.md | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index e7835f51e..760a4f4cf 100644 --- a/.gitignore +++ b/.gitignore @@ -199,3 +199,4 @@ assets/wheels/vllm*.whl # DCP artifacts model_state_dict/ forge_dcp_tmp/ +demo_top_down.md diff --git a/README.md b/README.md index 8283719f5..093c70c77 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ #### A PyTorch native agentic library for RL post-training and agentic development that lets you focus on algorithms instead of writing infra code. ## Overview -Forge was built with one core principle in mind: researchers should write algorithms, not infrastructure. Forge introduces a “service”-centric architecture that provides the right abstractions for distributed complexity and workloads. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm. +The primary purpose of the Forge ecosystem is to delineate infra concerns from model concerns thereby making RL experimentation easier. Forge delivers this by providing clear RL abstractions and one scalable implementation of these abstractions. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm. Key features: - Usability for rapid research (isolating the RL loop from infrastructure)