Auto build new torchtitan wheels from Github UI

[joecummings]

TorchTitan Wheel Building Workflow

This PR includes a GitHub Actions workflow that allows you to build wheels from specific commits in the pytorch/torchtitan repository.

How to Use

1. Trigger the Workflow

1. Go to the Actions tab in your GitHub repository
2. Click on "Build TorchTitan Wheel" workflow
3. Click "Run workflow"
4. Fill in the required parameter:
   • TorchTitan commit SHA, branch/tag, or "latest": The specific commit, branch, tag, or "latest" for the most recent commit from pytorch/torchtitan main branch

2. What the Workflow Does

The workflow will:

1. Setup Environment: Installs dependencies including CUDA toolkit and conda
2. Build TorchTitan Wheel: Uses the modified scripts/build_wheels.sh to build the torchtitan wheel from the chosen commit
3. Extract Commit Info: Gets detailed information about the commit used for building
4. Update Repository: Updates the TORCHTITAN_COMMIT in scripts/build_wheels.sh to match the built commit
5. Create Pull Request: Automatically creates a PR with:
   • The newly built torchtitan wheel in assets/wheels/
   • Updated commit reference in the build script
   • Detailed information about the build

3. Review and Merge

After the workflow completes:

1. A pull request will be automatically created
2. Review the changes and test the new wheel if needed
3. Merge the PR to update your repository with the new wheel

Manual Usage

You can also use the modified build script locally:

# Build only torchtitan wheel
./scripts/build_wheels.sh --wheel=torchtitan

# Build only vLLM wheel
./scripts/build_wheels.sh --wheel=vllm

# Build all wheels (default behavior)
./scripts/build_wheels.sh --wheel=all

# Use custom torchtitan commit
CUSTOM_TORCHTITAN_COMMIT=your_commit_sha ./scripts/build_wheels.sh --wheel=torchtitan

# Use latest torchtitan commit from main
CUSTOM_TORCHTITAN_COMMIT=latest ./scripts/build_wheels.sh --wheel=torchtitan

Available Options

Wheel Types

• torchtitan: PyTorch-based training framework
• vllm: High-throughput LLM serving engine
• monarch: Structured matrix library
• torchstore: Distributed storage for PyTorch tensors
• all: Build all available wheels

Environment Variables

• CUSTOM_TORCHTITAN_COMMIT: Override the default torchtitan commit used for building
  • Can be a specific commit SHA, branch name, tag, or "latest" for the latest commit from main

Requirements

The workflow runs on Ubuntu and includes all necessary dependencies. For local builds, ensure you have:

• Conda/Miniconda
• CUDA toolkit 12.9
• Git
• Build tools (gcc, make, etc.)
• Sufficient disk space (~10GB recommended)

Artifacts

Each workflow run creates artifacts containing:

• Built wheel files
• Build logs
• Commit information

Artifacts are retained for 30 days and can be downloaded from the Actions tab.

[codecov-commenter]

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️