diff --git a/flash/apps/build-app.mdx b/flash/apps/build-app.mdx index eb54ca15..4fe51903 100644 --- a/flash/apps/build-app.mdx +++ b/flash/apps/build-app.mdx @@ -14,7 +14,7 @@ If you haven't already, we recommend starting with the [Quickstart](/flash/quick - You've [created a Runpod account](/get-started/manage-accounts). - You've [created a Runpod API key](/get-started/api-keys). -- You've installed [Python 3.12](https://www.python.org/downloads/). +- You've installed [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/). ## Step 1: Initialize a new project diff --git a/flash/cli/build.mdx b/flash/cli/build.mdx index f9fa4330..f4a6c998 100644 --- a/flash/cli/build.mdx +++ b/flash/cli/build.mdx @@ -43,9 +43,13 @@ Custom name for the output archive file. Comma-separated list of packages to exclude from the build (e.g., `torch,torchvision`). Use this to skip packages already in the base image. + +Target Python version for worker images (3.10, 3.11, 3.12, or 3.13). Overrides per-resource `python_version` declarations and local interpreter detection. + + ## What happens during build -1. **Python version validation**: Verifies your local Python version is supported (3.12). +1. **Python version resolution**: Resolves the target Python version from CLI flag, resource configs, or your local interpreter. 2. **Function discovery**: Finds all `@Endpoint` decorated functions. 3. **Grouping**: Groups functions by their endpoint configuration. 4. **Manifest generation**: Creates `.flash/flash_manifest.json` with endpoint definitions. @@ -90,9 +94,28 @@ Flash automatically handles cross-platform builds: ### Python version in deployed workers -All Flash workers (GPU and CPU) run Python 3.12. `flash build` downloads wheels targeting Python 3.12 automatically. +Flash workers support Python 3.10, 3.11, 3.12, and 3.13. The target version is determined by: + +1. **CLI flag:** The `--python-version` flag takes precedence. +2. **Resource config:** The `python_version` parameter on your endpoint configs. +3. **Local interpreter:** Your local Python version (from `sys.version_info`) when neither is specified. + +All resources in a Flash app must use the same Python version because Flash ships a single tarball for the entire app. If resources declare conflicting versions, the build fails. + + +**Breaking change in Flash 0.x:** Flash now matches your local Python version by default instead of always defaulting to Python 3.12. If you upgrade Flash and your local Python differs from 3.12, your first deploy will trigger a rolling release. To maintain consistent behavior across team members, declare `python_version` explicitly on your resource configs or use `--python-version` on the CLI. + + +| Version | Cold start | Notes | +|---------|------------|-------| +| 3.12 | No overhead | PyTorch pre-installed in base image | +| 3.13 | +~7 GB on GPU | Alternative Python install required | +| 3.11 | +~7 GB on GPU | Alternative Python install required | +| 3.10 | +~7 GB on GPU | EOL 2026-10-31; consider migrating to 3.11+ | + +If your local Python version is not supported (for example, 3.9 or 3.14), the build fails with an actionable error message listing the supported versions. -Image tags follow the pattern `py3.12-{tag}` (for example, `runpod/flash:py3.12-latest`). +Image tags follow the pattern `py{version}-{tag}` (for example, `runpod/flash:py3.12-latest`). ## Managing deployment size diff --git a/flash/cli/deploy.mdx b/flash/cli/deploy.mdx index 41c87d86..4ff566d1 100644 --- a/flash/cli/deploy.mdx +++ b/flash/cli/deploy.mdx @@ -61,6 +61,10 @@ Custom archive name for the build artifact. Build and launch a local Docker-based preview environment instead of deploying to Runpod. + +Target Python version for worker images (3.10, 3.11, 3.12, or 3.13). Overrides per-resource `python_version` declarations and local interpreter detection. + + ## What happens during deployment 1. **Build phase**: Creates the deployment artifact (same as `flash build`). diff --git a/flash/configuration/parameters.mdx b/flash/configuration/parameters.mdx index 9e15767f..10c78980 100644 --- a/flash/configuration/parameters.mdx +++ b/flash/configuration/parameters.mdx @@ -30,6 +30,7 @@ This page provides a complete reference for all parameters available on the `End | `scaler_value` | `int` | Scaling threshold | `4` | | `template` | `PodTemplate` | Pod template overrides | `None` | | `min_cuda_version` | `str` or `CudaVersion` | Minimum CUDA version for GPU host selection | `"12.8"` (GPU) / `None` (CPU) | +| `python_version` | `str` | Python version for the worker image | Local Python | ## Parameter details @@ -575,6 +576,53 @@ This parameter has no effect on CPU endpoints. Valid CUDA versions: `CudaVersion.V11_1`, `V11_4`, `V11_7`, `V11_8`, `V12_0`, `V12_1`, `V12_2`, `V12_3`, `V12_4`, `V12_6`, `V12_8` (or equivalent strings like `"12.4"`). Invalid values raise a `ValueError`. +### python_version + +**Type**: `str` +**Default**: Local Python version + +Sets the Python version for the worker image. Supported values: `"3.10"`, `"3.11"`, `"3.12"`, and `"3.13"`. + +When you don't specify a Python version, Flash matches your local interpreter (the Python version you run Flash from). The resolution order is: + +1. `--python-version` CLI flag (highest priority) +2. `python_version` declared on resource configs +3. Your local Python version (`sys.version_info`) + +```python +from runpod_flash import Endpoint, GpuGroup + +# Explicitly set Python 3.11 for this endpoint +@Endpoint( + name="legacy-model", + gpu=GpuGroup.ANY, + python_version="3.11" +) +async def process(data): ... + +# Uses your local Python version (e.g., 3.12 if that's what you're running) +@Endpoint(name="modern-model", gpu=GpuGroup.ANY) +async def infer(data): ... +``` + +All resources in a Flash app must use the same Python version because Flash ships a single tarball for the entire app. If resources declare conflicting versions, the build fails. + + +**Breaking change:** Flash now matches your local Python version by default instead of always defaulting to Python 3.12. If your local Python differs from 3.12, your first deploy after upgrading Flash will trigger a rolling release. For consistent behavior across team members, declare `python_version` explicitly or use the `--python-version` CLI flag. + + + +Python 3.10, 3.11, and 3.13 workers incur approximately 7 GB of additional cold-start overhead on GPU endpoints because the alternative Python interpreter must be installed alongside the base image's PyTorch environment. + + + +Python 3.10 reaches end-of-life on 2026-10-31. Consider migrating to Python 3.11 or later. + + +If your local Python version is not supported (for example, 3.9 or 3.14), the build fails with an actionable error message listing the supported versions. + +The `--python-version` CLI flag on `flash build` and `flash deploy` overrides both per-resource declarations and local interpreter detection. + ## EndpointJob When using `Endpoint(id=...)` or `Endpoint(image=...)`, the `.run()` method returns an `EndpointJob` object for async operations: @@ -615,6 +663,7 @@ These changes restart all workers: - Datacenter (`datacenter`) - Flashboot setting (`flashboot`) - CUDA version requirement (`min_cuda_version`) +- Python version (`python_version`) Workers are temporarily unavailable during recreation (typically 30-90 seconds). diff --git a/flash/overview.mdx b/flash/overview.mdx index 169e3148..4977ae1d 100644 --- a/flash/overview.mdx +++ b/flash/overview.mdx @@ -56,7 +56,7 @@ Flash requires a Runpod account with a verified email address. -Flash requires [Python 3.12](https://www.python.org/downloads/) and is currently available for macOS and Linux. +Flash requires [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/) and is currently available for macOS and Linux. Install Flash using `pip` or `uv`: diff --git a/flash/quickstart.mdx b/flash/quickstart.mdx index 6c625914..475e8704 100644 --- a/flash/quickstart.mdx +++ b/flash/quickstart.mdx @@ -14,7 +14,7 @@ This quickstart gets you running GPU workloads on Runpod in minutes. You'll exec - [Runpod account](/get-started/manage-accounts) with a verified email address. - [An API key](/get-started/api-keys) with **All** access permissions to your Runpod account. -- [Python 3.12](https://www.python.org/downloads/) installed. +- [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/) installed. - [uv](https://docs.astral.sh/uv/) installed. ## Step 1: Install Flash diff --git a/flash/troubleshooting.mdx b/flash/troubleshooting.mdx index 6e4bbd9d..b8a9f5a9 100644 --- a/flash/troubleshooting.mdx +++ b/flash/troubleshooting.mdx @@ -208,27 +208,41 @@ Duplicate route 'POST /process' in endpoint 'my-api' **Error:** ``` -Python 3.13 is not supported for Flash deployment. -Supported versions: 3.12 +Local Python 3.9 is not supported by Flash workers (supported: 3.10, 3.11, 3.12, 3.13). +Pass --python-version, declare python_version on a resource config, or run flash from a supported interpreter. ``` -**Cause:** Flash requires Python 3.12. +**Cause:** Flash supports Python 3.10, 3.11, 3.12, and 3.13. Your local Python version is outside this range. **Solution:** -Switch to Python 3.12 using a virtual environment: +You have three options: -```bash -# Using pyenv -pyenv install 3.12 -pyenv local 3.12 +1. **Use the `--python-version` CLI flag** to override local detection: + ```bash + flash build --python-version 3.12 + flash deploy --python-version 3.12 + ``` -# Or using uv -uv venv --python 3.12 -source .venv/bin/activate -``` +2. **Declare `python_version` on your resource configs:** + ```python + @Endpoint(name="my-endpoint", gpu=GpuGroup.ANY, python_version="3.12") + ``` + +3. **Switch to a supported Python version** using a virtual environment: + ```bash + # Using pyenv + pyenv install 3.12 + pyenv local 3.12 + + # Or using uv + uv venv --python 3.12 + source .venv/bin/activate + ``` -Alternatively, use a Docker container with Python 3.12 for your build environment. + +Python 3.12 is recommended for best performance with no cold-start overhead. Python 3.10, 3.11, and 3.13 incur additional cold-start overhead on GPU workers because an alternative Python interpreter must be installed. + ## Deployment errors diff --git a/tutorials/flash/build-rest-api-with-load-balancer.mdx b/tutorials/flash/build-rest-api-with-load-balancer.mdx index 5abc959c..28fa2b18 100644 --- a/tutorials/flash/build-rest-api-with-load-balancer.mdx +++ b/tutorials/flash/build-rest-api-with-load-balancer.mdx @@ -11,7 +11,7 @@ This tutorial shows you how to build a REST API using Flash load-balanced endpoi - You've [created a Runpod account](/get-started/manage-accounts) - You've [created a Runpod API key](/get-started/api-keys) -- You've installed [Python 3.12](https://www.python.org/downloads/). +- You've installed [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/). - You've completed the [Flash quickstart](/flash/quickstart) or are familiar with Flash basics ## What you'll build diff --git a/tutorials/flash/image-generation-with-sdxl.mdx b/tutorials/flash/image-generation-with-sdxl.mdx index b0413eaa..2c281f97 100644 --- a/tutorials/flash/image-generation-with-sdxl.mdx +++ b/tutorials/flash/image-generation-with-sdxl.mdx @@ -15,7 +15,7 @@ This tutorial shows you how to build an image generation script using Flash and - You've [created a Runpod account](/get-started/manage-accounts). - You've [created a Runpod API key](/get-started/api-keys). -- You've installed [Python 3.12](https://www.python.org/downloads/). +- You've installed [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/). - You've completed the [Flash quickstart](/flash/quickstart) or are familiar with Flash basics. ## What you'll build diff --git a/tutorials/flash/text-generation-with-transformers.mdx b/tutorials/flash/text-generation-with-transformers.mdx index 64695b20..47b9e552 100644 --- a/tutorials/flash/text-generation-with-transformers.mdx +++ b/tutorials/flash/text-generation-with-transformers.mdx @@ -11,7 +11,7 @@ This tutorial shows you how to build a text generation script using Flash and Hu - You've [created a Runpod account](/get-started/manage-accounts). - You've [created a Runpod API key](/get-started/api-keys). -- You've installed [Python 3.12](https://www.python.org/downloads/). +- You've installed [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/). - You've completed the [Flash quickstart](/flash/quickstart) or are familiar with Flash basics. ## What you'll build