Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion flash/apps/build-app.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ If you haven't already, we recommend starting with the [Quickstart](/flash/quick

- You've [created a Runpod account](/get-started/manage-accounts).
- You've [created a Runpod API key](/get-started/api-keys).
- You've installed [Python 3.12](https://www.python.org/downloads/).
- You've installed [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/).

## Step 1: Initialize a new project

Expand Down
29 changes: 26 additions & 3 deletions flash/cli/build.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,13 @@ Custom name for the output archive file.
Comma-separated list of packages to exclude from the build (e.g., `torch,torchvision`). Use this to skip packages already in the base image.
</ResponseField>

<ResponseField name="--python-version" type="string">
Target Python version for worker images (3.10, 3.11, 3.12, or 3.13). Overrides per-resource `python_version` declarations and local interpreter detection.
</ResponseField>

## What happens during build

1. **Python version validation**: Verifies your local Python version is supported (3.12).
1. **Python version resolution**: Resolves the target Python version from CLI flag, resource configs, or your local interpreter.
2. **Function discovery**: Finds all `@Endpoint` decorated functions.
3. **Grouping**: Groups functions by their endpoint configuration.
4. **Manifest generation**: Creates `.flash/flash_manifest.json` with endpoint definitions.
Expand Down Expand Up @@ -90,9 +94,28 @@ Flash automatically handles cross-platform builds:

### Python version in deployed workers
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added Python version selection documentation based on PR #322 which introduces per-app python_version config and --python-version CLI flag for flash build and flash deploy. The PR's constants.py defines supported versions (3.10, 3.11, 3.12) and manifest.py implements version reconciliation logic.

Source: runpod/flash#322


All Flash workers (GPU and CPU) run Python 3.12. `flash build` downloads wheels targeting Python 3.12 automatically.
Flash workers support Python 3.10, 3.11, 3.12, and 3.13. The target version is determined by:

1. **CLI flag:** The `--python-version` flag takes precedence.
2. **Resource config:** The `python_version` parameter on your endpoint configs.
3. **Local interpreter:** Your local Python version (from `sys.version_info`) when neither is specified.

All resources in a Flash app must use the same Python version because Flash ships a single tarball for the entire app. If resources declare conflicting versions, the build fails.

<Warning>
**Breaking change in Flash 0.x:** Flash now matches your local Python version by default instead of always defaulting to Python 3.12. If you upgrade Flash and your local Python differs from 3.12, your first deploy will trigger a rolling release. To maintain consistent behavior across team members, declare `python_version` explicitly on your resource configs or use `--python-version` on the CLI.
</Warning>

| Version | Cold start | Notes |
|---------|------------|-------|
| 3.12 | No overhead | PyTorch pre-installed in base image |
| 3.13 | +~7 GB on GPU | Alternative Python install required |
| 3.11 | +~7 GB on GPU | Alternative Python install required |
| 3.10 | +~7 GB on GPU | EOL 2026-10-31; consider migrating to 3.11+ |

If your local Python version is not supported (for example, 3.9 or 3.14), the build fails with an actionable error message listing the supported versions.

Image tags follow the pattern `py3.12-{tag}` (for example, `runpod/flash:py3.12-latest`).
Image tags follow the pattern `py{version}-{tag}` (for example, `runpod/flash:py3.12-latest`).

## Managing deployment size

Expand Down
4 changes: 4 additions & 0 deletions flash/cli/deploy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,10 @@ Custom archive name for the build artifact.
Build and launch a local Docker-based preview environment instead of deploying to Runpod.
</ResponseField>

<ResponseField name="--python-version" type="string">
Target Python version for worker images (3.10, 3.11, 3.12, or 3.13). Overrides per-resource `python_version` declarations and local interpreter detection.
</ResponseField>

## What happens during deployment

1. **Build phase**: Creates the deployment artifact (same as `flash build`).
Expand Down
49 changes: 49 additions & 0 deletions flash/configuration/parameters.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ This page provides a complete reference for all parameters available on the `End
| `scaler_value` | `int` | Scaling threshold | `4` |
| `template` | `PodTemplate` | Pod template overrides | `None` |
| `min_cuda_version` | `str` or `CudaVersion` | Minimum CUDA version for GPU host selection | `"12.8"` (GPU) / `None` (CPU) |
| `python_version` | `str` | Python version for the worker image | Local Python |

## Parameter details

Expand Down Expand Up @@ -575,6 +576,53 @@ This parameter has no effect on CPU endpoints.
Valid CUDA versions: `CudaVersion.V11_1`, `V11_4`, `V11_7`, `V11_8`, `V12_0`, `V12_1`, `V12_2`, `V12_3`, `V12_4`, `V12_6`, `V12_8` (or equivalent strings like `"12.4"`). Invalid values raise a `ValueError`.
</Note>

### python_version
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added python_version parameter documentation from PR #322. The PR's manifest.py:221-225 extracts python_version from resource configs, and cold-start overhead (~7 GB for 3.10/3.11) is noted in docs/Flash_Deploy_Guide.md within the PR.

Source: runpod/flash#322


**Type**: `str`
**Default**: Local Python version

Sets the Python version for the worker image. Supported values: `"3.10"`, `"3.11"`, `"3.12"`, and `"3.13"`.

When you don't specify a Python version, Flash matches your local interpreter (the Python version you run Flash from). The resolution order is:

1. `--python-version` CLI flag (highest priority)
2. `python_version` declared on resource configs
3. Your local Python version (`sys.version_info`)

```python
from runpod_flash import Endpoint, GpuGroup

# Explicitly set Python 3.11 for this endpoint
@Endpoint(
name="legacy-model",
gpu=GpuGroup.ANY,
python_version="3.11"
)
async def process(data): ...

# Uses your local Python version (e.g., 3.12 if that's what you're running)
@Endpoint(name="modern-model", gpu=GpuGroup.ANY)
async def infer(data): ...
```

All resources in a Flash app must use the same Python version because Flash ships a single tarball for the entire app. If resources declare conflicting versions, the build fails.

<Warning>
**Breaking change:** Flash now matches your local Python version by default instead of always defaulting to Python 3.12. If your local Python differs from 3.12, your first deploy after upgrading Flash will trigger a rolling release. For consistent behavior across team members, declare `python_version` explicitly or use the `--python-version` CLI flag.
</Warning>

<Warning>
Python 3.10, 3.11, and 3.13 workers incur approximately 7 GB of additional cold-start overhead on GPU endpoints because the alternative Python interpreter must be installed alongside the base image's PyTorch environment.
</Warning>

<Note>
Python 3.10 reaches end-of-life on 2026-10-31. Consider migrating to Python 3.11 or later.
</Note>

If your local Python version is not supported (for example, 3.9 or 3.14), the build fails with an actionable error message listing the supported versions.

The `--python-version` CLI flag on `flash build` and `flash deploy` overrides both per-resource declarations and local interpreter detection.

## EndpointJob

When using `Endpoint(id=...)` or `Endpoint(image=...)`, the `.run()` method returns an `EndpointJob` object for async operations:
Expand Down Expand Up @@ -615,6 +663,7 @@ These changes restart all workers:
- Datacenter (`datacenter`)
- Flashboot setting (`flashboot`)
- CUDA version requirement (`min_cuda_version`)
- Python version (`python_version`)

Workers are temporarily unavailable during recreation (typically 30-90 seconds).

Expand Down
2 changes: 1 addition & 1 deletion flash/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Flash requires a Runpod account with a verified email address.

<Note>

Flash requires [Python 3.12](https://www.python.org/downloads/) and is currently available for macOS and Linux.
Flash requires [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/) and is currently available for macOS and Linux.
</Note>

Install Flash using `pip` or `uv`:
Expand Down
2 changes: 1 addition & 1 deletion flash/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ This quickstart gets you running GPU workloads on Runpod in minutes. You'll exec

- [Runpod account](/get-started/manage-accounts) with a verified email address.
- [An API key](/get-started/api-keys) with **All** access permissions to your Runpod account.
- [Python 3.12](https://www.python.org/downloads/) installed.
- [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/) installed.
- [uv](https://docs.astral.sh/uv/) installed.

## Step 1: Install Flash
Expand Down
40 changes: 27 additions & 13 deletions flash/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -208,27 +208,41 @@ Duplicate route 'POST /process' in endpoint 'my-api'

**Error:**
```
Python 3.13 is not supported for Flash deployment.
Supported versions: 3.12
Local Python 3.9 is not supported by Flash workers (supported: 3.10, 3.11, 3.12, 3.13).
Pass --python-version, declare python_version on a resource config, or run flash from a supported interpreter.
```

**Cause:** Flash requires Python 3.12.
**Cause:** Flash supports Python 3.10, 3.11, 3.12, and 3.13. Your local Python version is outside this range.

**Solution:**

Switch to Python 3.12 using a virtual environment:
You have three options:

```bash
# Using pyenv
pyenv install 3.12
pyenv local 3.12
1. **Use the `--python-version` CLI flag** to override local detection:
```bash
flash build --python-version 3.12
flash deploy --python-version 3.12
```

# Or using uv
uv venv --python 3.12
source .venv/bin/activate
```
2. **Declare `python_version` on your resource configs:**
```python
@Endpoint(name="my-endpoint", gpu=GpuGroup.ANY, python_version="3.12")
```

3. **Switch to a supported Python version** using a virtual environment:
```bash
# Using pyenv
pyenv install 3.12
pyenv local 3.12

# Or using uv
uv venv --python 3.12
source .venv/bin/activate
```

Alternatively, use a Docker container with Python 3.12 for your build environment.
<Tip>
Python 3.12 is recommended for best performance with no cold-start overhead. Python 3.10, 3.11, and 3.13 incur additional cold-start overhead on GPU workers because an alternative Python interpreter must be installed.
</Tip>

## Deployment errors

Expand Down
2 changes: 1 addition & 1 deletion tutorials/flash/build-rest-api-with-load-balancer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This tutorial shows you how to build a REST API using Flash load-balanced endpoi

- You've [created a Runpod account](/get-started/manage-accounts)
- You've [created a Runpod API key](/get-started/api-keys)
- You've installed [Python 3.12](https://www.python.org/downloads/).
- You've installed [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/).
- You've completed the [Flash quickstart](/flash/quickstart) or are familiar with Flash basics

## What you'll build
Expand Down
2 changes: 1 addition & 1 deletion tutorials/flash/image-generation-with-sdxl.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This tutorial shows you how to build an image generation script using Flash and

- You've [created a Runpod account](/get-started/manage-accounts).
- You've [created a Runpod API key](/get-started/api-keys).
- You've installed [Python 3.12](https://www.python.org/downloads/).
- You've installed [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/).
- You've completed the [Flash quickstart](/flash/quickstart) or are familiar with Flash basics.

## What you'll build
Expand Down
2 changes: 1 addition & 1 deletion tutorials/flash/text-generation-with-transformers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This tutorial shows you how to build a text generation script using Flash and Hu

- You've [created a Runpod account](/get-started/manage-accounts).
- You've [created a Runpod API key](/get-started/api-keys).
- You've installed [Python 3.12](https://www.python.org/downloads/).
- You've installed [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/).
- You've completed the [Flash quickstart](/flash/quickstart) or are familiar with Flash basics.

## What you'll build
Expand Down
Loading