diff --git a/README.md b/README.md index a6e5b8d..b6978ae 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ It runs as a **single process on one port**: the Inline Core engine (Python) ser ## Get Started -The built web UI ships as a Python package, so all you need is [Python 3.11+](https://python.org), no Node. **`--install --extra all` is the single command that installs everything** - the engine, the local model runtime, the LoRA trainer, and the UI. On an NVIDIA machine it detects the GPU and pulls the CUDA build of PyTorch for you. +The built web UI ships as a Python package, so all you need is [Python 3.11+](https://python.org), no Node. **`--install --extra all` is the single command that installs everything** - the engine, the local model runtime, the LoRA trainer, and the UI. On an NVIDIA machine it reads the GPU's compute capability and pulls the CUDA build of PyTorch that has kernels for it, RTX 50-series included. **macOS / Linux:** @@ -70,13 +70,29 @@ Prefer pip over the launcher? `pip install -r requirements.txt` (from the repo r Honest status - what's actually been run, versus what has a code path but no one has verified: -| Hardware | Status | Extra steps | -| ----------------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **NVIDIA, Linux** | **Tested** - Z-Image Turbo 1024² on a T4 (16 GB); Krea 2 1024² and LoRA training on an L40S (48 GB) | None. `webui.sh --install` picks the CUDA build automatically. | -| **NVIDIA, Windows** | Supported, needs one step | Run `.\webui.bat --install` (the Windows launcher; it detects the GPU). PyPI's default `torch` is **CPU-only on Windows**, so `--install` pulls the CUDA build for you, or install torch from `https://download.pytorch.org/whl/cu124`. Core warns at startup if it finds an NVIDIA GPU behind a CPU-only torch. | -| **Apple Silicon (MPS)** | Code path exists, **untested** | None. int8 quantisation doesn't apply on MPS, so a model too big for unified memory won't fit. | -| **AMD (ROCm), Linux** | **Untested** - reports welcome | Needs a ROCm build of PyTorch - see [AMD (ROCm) setup](#amd-rocm-setup) below. | -| **CPU only** | Works, very slow | `./webui.sh --cpu` (Windows: `.\webui.bat --cpu`) | +| Hardware | Status | Extra steps | +| ----------------------- | --------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **NVIDIA, Linux** | **Tested** - Z-Image Turbo 1024² on a T4 (16 GB); Krea 2 1024² and LoRA training on an L40S (48 GB) | None. `webui.sh --install` picks the CUDA build automatically. | +| **NVIDIA, Windows** | Supported, needs one step | Run `.\webui.bat --install` (the Windows launcher). PyPI's default `torch` is **CPU-only on Windows**, so `--install` reads your GPU's compute capability and pulls the matching CUDA build: `cu130` for RTX 50-series (Blackwell), `cu126` for everything older. Override it with `--torch-index` - see [RTX 50-series](#rtx-50-series-blackwell) below. | +| **Apple Silicon (MPS)** | Code path exists, **untested** | None. int8 quantisation doesn't apply on MPS, so a model too big for unified memory won't fit. | +| **AMD (ROCm), Linux** | **Untested** - reports welcome | Needs a ROCm build of PyTorch - see [AMD (ROCm) setup](#amd-rocm-setup) below. | +| **CPU only** | Works, very slow | `./webui.sh --cpu` (Windows: `.\webui.bat --cpu`) | + +#### RTX 50-series (Blackwell) + +RTX 50-series cards (5060/5070/5080/5090 and the RTX PRO Blackwell line) are compute capability **sm_120**, and no PyTorch wheel built for CUDA 12.4 or 12.6 has kernels for them. `--install` handles this: it reads the compute capability off the driver and picks `cu130`, so a plain `.\webui.bat --install --extra all` is all you need. + +Two cases where you may want to say it yourself: + +```powershell +rem Blackwell card, but a driver older than CUDA 13 (R580) - cu128 has sm_120 and a lower floor +.\webui.bat --install --extra all --torch-index cu128 + +rem Or set it once for the shell, same effect +set INLINE_TORCH_INDEX=cu128 +``` + +`--torch-index` takes a short name (`cu130`, `cu128`, `cu126`), a full index URL, or `cpu` to force the CPU-only build. `webui.sh` takes the same flag. If the installed build turns out to have no kernels for your card, Core says so by name at startup rather than leaving you with PyTorch's own `sm_120 is not compatible` warning. #### AMD (ROCm) setup @@ -100,7 +116,7 @@ Then run `./webui.sh` as usual. Three gotchas: -- **Don't run `uv sync` afterwards** - it re-resolves the environment against the lockfile and will pull the PyPI torch back over your ROCm build. Use `uv pip install --python .venv/bin/python` for follow-up installs. +- **Don't run `uv sync` afterwards** - it re-resolves the environment against the lockfile and will pull the PyPI torch back over your ROCm build. Use `uv pip install --python .venv/bin/python` for follow-up installs. The same applies to a hand-picked CUDA index. - **Don't pass `--recreate`** - it rebuilds `.venv` from scratch and your ROCm torch goes with it. A plain `--install` re-run reuses the venv and is safe. - ROCm presents itself through `torch.cuda`, so the engine will treat it as a CUDA device and may largely work. But the dtype heuristics key off **NVIDIA** compute capability (`< 8.0` → fp16), which is meaningless on RDNA/CDNA, and the int8 (torchao) path is unverified on ROCm. If it works - or doesn't - [open an issue](https://github.com/inlineresearch/Inline-Studio/issues); that's the fastest way to get AMD properly supported. @@ -164,7 +180,7 @@ The friendly launcher (in `core/`) maps flags onto the engine's `INLINE_*` envir -`webui.sh` also has `--install` / `--extra NAME` to set up the venv, plus `--recreate` (rebuild `.venv` from scratch) and `--use-active-env` (install into / run from the environment activated in your shell instead of `.venv`). New to Inline Studio? The [Getting Started guide](https://inlinestudio.art/getting-started) walks you through your first render. +`webui.sh` also has `--install` / `--extra NAME` to set up the venv, plus `--torch-index WHICH` (`INLINE_TORCH_INDEX`) to override the PyTorch wheel index picked from your GPU's compute capability, `--recreate` (rebuild `.venv` from scratch) and `--use-active-env` (install into / run from the environment activated in your shell instead of `.venv`). New to Inline Studio? The [Getting Started guide](https://inlinestudio.art/getting-started) walks you through your first render. ## Features diff --git a/core/pyproject.toml b/core/pyproject.toml index 6e0543e..09739dd 100644 --- a/core/pyproject.toml +++ b/core/pyproject.toml @@ -101,13 +101,18 @@ all = [ ] # PyPI ships CPU-only torch on Windows; pin the CUDA index there (pip: see requirements.txt). +# cu126 rather than a newer index because it is the last one still built for sm_50..sm_70 (Maxwell +# through Volta), so it is the safe default for a card we know nothing about. Blackwell (sm_120) has +# no wheel here at all - webui.sh/.bat detect the compute capability and pass the matching index. [[tool.uv.index]] -name = "pytorch-cu124" -url = "https://download.pytorch.org/whl/cu124" +name = "pytorch-cu126" +url = "https://download.pytorch.org/whl/cu126" explicit = true +# Launcher installs pass `--no-sources-package torch` so their detected index wins over this pin; +# anything else added to this table would still apply to them. [tool.uv.sources] -torch = [{ index = "pytorch-cu124", marker = "sys_platform == 'win32'" }] +torch = [{ index = "pytorch-cu126", marker = "sys_platform == 'win32'" }] [project.scripts] inline-studio = "inline_core.server.__main__:main" diff --git a/core/src/inline_core/device/detect.py b/core/src/inline_core/device/detect.py index ff761e0..ae495e7 100644 --- a/core/src/inline_core/device/detect.py +++ b/core/src/inline_core/device/detect.py @@ -76,10 +76,58 @@ def cpu_only_torch_warning() -> str | None: "This install has a CPU-ONLY build of PyTorch, but an NVIDIA GPU was detected. " "Generation will run on the CPU and be roughly 100x slower. Reinstall torch from the CUDA " "index, e.g. `pip install --force-reinstall --extra-index-url " - "https://download.pytorch.org/whl/cu124 torch` (or re-run `webui.sh --install`)." + "https://download.pytorch.org/whl/cu126 torch` (or re-run `webui.sh --install`)." ) +def unsupported_arch_warning() -> str | None: + """A warning when the installed torch has no kernels for the GPU it is about to run on. + + An RTX 50-series card (sm_120) under a wheel built for sm_50..sm_90 is the sharpest case: torch + reports a CUDA build and a visible device, so every check in ``cpu_only_torch_warning`` passes, + and the only clue the user gets is PyTorch's own late UserWarning followed by kernels that + cannot launch. Returns None on any uncertainty, same as the rest of this module. + """ + try: + import torch + + if getattr(torch.version, "hip", None): + return None # the sm_ vocabulary is NVIDIA's; HIP reports gfx arches through it + get_arch_list = getattr(torch.cuda, "get_arch_list", None) + get_capability = getattr(torch.cuda, "get_device_capability", None) + device_count = getattr(torch.cuda, "device_count", None) + if get_arch_list is None or get_capability is None or device_count is None: + return None + if device_count() < 1: + return None + arches = [str(arch) for arch in get_arch_list() if str(arch).startswith("sm_")] + if not arches: + return None # a CPU-only build; cpu_only_torch_warning owns that case + major, minor = get_capability(0) + target = f"sm_{major}{minor}" + # startswith, because a wheel lists tuned variants like sm_90a for the same architecture. + if any(arch.startswith(target) for arch in arches): + return None + name = _device_name(torch) or "The detected NVIDIA GPU" + return ( + f"{name} is compute capability {target}, but this install's PyTorch only has kernels " + f"for {' '.join(arches)}. Generation will fail or fall back to the CPU. Re-run " + "`webui.sh --install` (Windows: `.\\webui.bat --install`) to pick the wheel index that " + "matches the card, or force one with `--torch-index cu130` - `cu128` if the driver " + "predates CUDA 13." + ) + except Exception: # noqa: BLE001 - a diagnostic must never break startup + return None + + +def _device_name(torch: object) -> str | None: + try: + get_name = getattr(getattr(torch, "cuda", None), "get_device_name", None) + return str(get_name(0)) if get_name is not None else None + except Exception: # noqa: BLE001 + return None + + def _nvidia_gpu_present() -> bool: """Best-effort check for NVIDIA hardware that does NOT rely on torch's CUDA support - that is the whole point, since we are called precisely when torch cannot see the GPU. diff --git a/core/src/inline_core/server/__main__.py b/core/src/inline_core/server/__main__.py index a625d28..f7bc227 100644 --- a/core/src/inline_core/server/__main__.py +++ b/core/src/inline_core/server/__main__.py @@ -19,7 +19,7 @@ import uvicorn from ..config import data_dir, server_host, server_port -from ..device.detect import cpu_only_torch_warning +from ..device.detect import cpu_only_torch_warning, unsupported_arch_warning from ..device.memory import MemoryPolicy from ..extensions.loader import LoadedExtension from ..graph.cache import InMemoryCache @@ -55,7 +55,7 @@ def main() -> None: print(f"Extensions: {_extension_summary(extensions)}") # A CPU-only torch wheel on a CUDA machine is a silent ~100x slowdown, so say it loudly here # rather than letting the user conclude the engine is just slow. - torch_warning = cpu_only_torch_warning() + torch_warning = cpu_only_torch_warning() or unsupported_arch_warning() if torch_warning: print(f"WARNING: {torch_warning}") frontend_root = resolve_frontend_root() diff --git a/core/tests/test_device_detect.py b/core/tests/test_device_detect.py index 551a0c5..1f45b6e 100644 --- a/core/tests/test_device_detect.py +++ b/core/tests/test_device_detect.py @@ -78,7 +78,8 @@ def test_bf16_gate_uses_the_capability_rule_on_nvidia(monkeypatch) -> None: from inline_core.device.types import Device, DeviceKind dev = Device(DeviceKind.CUDA, 0) - for capability, expected in (((8, 6), True), ((9, 0), True), ((7, 5), False), ((7, 0), False)): + caps = (((12, 0), True), ((8, 6), True), ((9, 0), True), ((7, 5), False), ((7, 0), False)) + for capability, expected in caps: monkeypatch.setitem( __import__("sys").modules, "torch", @@ -110,6 +111,73 @@ def test_bf16_gate_asks_torch_directly_on_rocm(monkeypatch) -> None: assert detect.cuda_supports_bf16(dev) is True +# What a cu124 wheel reports - the build every Windows install used to be pinned to. +_CU124_ARCHES = ["sm_50", "sm_60", "sm_61", "sm_70", "sm_75", "sm_80", "sm_86", "sm_90"] + + +def _fake_arch_torch( + *, capability: tuple[int, int], arches: list[str], name: str = "NVIDIA GeForce RTX 5070 Ti" +) -> types.SimpleNamespace: + return types.SimpleNamespace( + version=types.SimpleNamespace(cuda="12.4", hip=None), + cuda=types.SimpleNamespace( + device_count=lambda: 1, + get_arch_list=lambda: arches, + get_device_capability=lambda i=0: capability, + get_device_name=lambda i=0: name, + ), + ) + + +def test_warns_when_the_wheel_has_no_kernels_for_the_card(monkeypatch) -> None: + """The RTX 50-series failure: torch is a CUDA build and the device is visible, so every check in + cpu_only_torch_warning passes and the user is left with PyTorch's own cryptic UserWarning.""" + monkeypatch.setitem( + __import__("sys").modules, + "torch", + _fake_arch_torch(capability=(12, 0), arches=_CU124_ARCHES), + ) + warning = detect.unsupported_arch_warning() + assert warning is not None + assert "sm_120" in warning + assert "RTX 5070 Ti" in warning + assert "--torch-index" in warning # tells them how to fix it + + +def test_silent_when_the_wheel_covers_the_card(monkeypatch) -> None: + monkeypatch.setitem( + __import__("sys").modules, + "torch", + _fake_arch_torch(capability=(8, 6), arches=_CU124_ARCHES), + ) + assert detect.unsupported_arch_warning() is None + + +def test_arch_warning_accepts_a_tuned_variant(monkeypatch) -> None: + """Wheels list per-architecture variants like sm_90a; that is still a match for sm_90.""" + monkeypatch.setitem( + __import__("sys").modules, + "torch", + _fake_arch_torch(capability=(9, 0), arches=["sm_80", "sm_90a"]), + ) + assert detect.unsupported_arch_warning() is None + + +def test_arch_warning_silent_when_torch_cannot_answer(monkeypatch) -> None: + """An older torch has no get_arch_list, and a CPU-only build reports no sm_ arches at all - + cpu_only_torch_warning owns that second case. Neither may produce a false alarm here.""" + monkeypatch.setitem( + __import__("sys").modules, "torch", _fake_torch(cuda_available=False, cuda_version="12.4") + ) + assert detect.unsupported_arch_warning() is None + monkeypatch.setitem( + __import__("sys").modules, + "torch", + _fake_arch_torch(capability=(12, 0), arches=[]), + ) + assert detect.unsupported_arch_warning() is None + + def test_silent_without_torch(monkeypatch) -> None: """A hosted-only (fal) install deliberately has no torch; that is not a misconfiguration.""" import builtins diff --git a/core/tests/test_webui_install.py b/core/tests/test_webui_install.py index 2c02d37..53e6512 100644 --- a/core/tests/test_webui_install.py +++ b/core/tests/test_webui_install.py @@ -31,9 +31,10 @@ class Sandbox: active_env: Path uv_log: Path python_log: Path + stubs: Path path: str - def run(self, *args: str) -> subprocess.CompletedProcess[str]: + def run(self, *args: str, **env: str) -> subprocess.CompletedProcess[str]: return subprocess.run( # noqa: S603 - fixed argv from our own code, no shell ["bash", str(self.script), *args], cwd=self.script.parent, @@ -46,6 +47,7 @@ def run(self, *args: str) -> subprocess.CompletedProcess[str]: "HOME": str(self.script.parent.parent), "VIRTUAL_ENV": str(self.active_env), "PYTHON_LOG": str(self.python_log), + **env, }, ) @@ -58,6 +60,25 @@ def make_venv_python(self) -> None: self.venv_python.parent.mkdir(parents=True, exist_ok=True) _stub(self.venv_python, 'echo "$*" >> "$PYTHON_LOG"\nexit 0\n') + def pretend_nvidia_gpu(self, *compute_caps: str) -> None: + """A driver that answers `-L` and the compute_cap query. Pass no caps for an older driver + that does not know the query, which is the case the fallback index exists for.""" + answer = "".join(f"printf '{cap}\\n'\n" for cap in compute_caps) or "exit 1\n" + _stub( + self.stubs / "nvidia-smi", + f'case "$*" in\n *compute_cap*) {answer.strip()} ;;\nesac\nexit 0\n', + ) + + def pretend_windows(self) -> None: + """Only Windows needs an explicit CUDA index; this script runs there under Git Bash.""" + _stub(self.stubs / "uname", "printf 'MINGW64_NT-10.0-22631\\n'\nexit 0\n") + + def project_install(self) -> str: + """The one uv call that resolves pyproject's dependencies.""" + calls = [call for call in self.uv_calls() if "-e ." in call] + assert len(calls) == 1, self.uv_calls() + return calls[0] + def _stub(path: Path, body: str) -> None: path.write_text(f"#!/bin/sh\n{body}", encoding="utf-8") @@ -86,6 +107,7 @@ def sandbox(tmp_path: Path) -> Sandbox: active_env=active, uv_log=uv_log, python_log=tmp_path / "python.log", + stubs=stubs, path=f"{stubs}:/usr/bin:/bin", ) @@ -141,6 +163,83 @@ def test_unknown_extra_fails_with_the_valid_list(sandbox: Sandbox) -> None: assert not sandbox.uv_calls() +_WHL = "--extra-index-url https://download.pytorch.org/whl" + + +def test_blackwell_gets_an_index_that_has_sm_120_wheels(sandbox: Sandbox) -> None: + """The RTX 50-series bug: cu124 is frozen at torch 2.6.0 and never gained sm_120, so a Blackwell + card has to be sent somewhere else entirely.""" + sandbox.make_venv_python() + sandbox.pretend_windows() + sandbox.pretend_nvidia_gpu("12.0") + + done = sandbox.run("--install", "--extra", "runtime") + + assert done.returncode == 0, done.stderr + install = sandbox.project_install() + assert f"{_WHL}/cu130" in install + # Without this the pyproject pin wins and the detected index is silently ignored. + assert "--no-sources-package torch" in install + + +def test_older_cards_get_the_index_that_still_covers_them(sandbox: Sandbox) -> None: + """cu126 is the last index built for Maxwell..Volta, so it is both the Ampere answer and the + fallback when the driver is too old to report a compute capability at all.""" + sandbox.make_venv_python() + sandbox.pretend_windows() + sandbox.pretend_nvidia_gpu("8.6") + + assert f"{_WHL}/cu126" in _install(sandbox) + + sandbox.pretend_nvidia_gpu() # driver that does not know the query + assert f"{_WHL}/cu126" in _install(sandbox) + + +def test_the_highest_capability_across_gpus_decides(sandbox: Sandbox) -> None: + sandbox.make_venv_python() + sandbox.pretend_windows() + sandbox.pretend_nvidia_gpu("8.6", "12.0") + + assert f"{_WHL}/cu130" in _install(sandbox) + + +def test_torch_index_override_beats_detection(sandbox: Sandbox) -> None: + """The reporter hand-edited webui.bat because there was no way to say this; there is now.""" + sandbox.make_venv_python() + sandbox.pretend_windows() + sandbox.pretend_nvidia_gpu("12.0") + + assert f"{_WHL}/cu128" in _install(sandbox, "--torch-index", "cu128") + assert f"{_WHL}/cu128" in _install(sandbox, INLINE_TORCH_INDEX="cu128") + assert "https://mirror.example/whl/cu128" in _install( + sandbox, "--torch-index", "https://mirror.example/whl/cu128" + ) + + forced_cpu = _install(sandbox, "--torch-index", "cpu") + assert "--extra-index-url" not in forced_cpu + + +def test_linux_keeps_the_default_pypi_wheels(sandbox: Sandbox) -> None: + """Linux torch on PyPI already bundles CUDA, so naming an index there would only pin us to an + older build than the default one.""" + sandbox.make_venv_python() + sandbox.pretend_nvidia_gpu("12.0") + + done = sandbox.run("--install", "--extra", "runtime") + + assert done.returncode == 0, done.stderr + assert "--extra-index-url" not in sandbox.project_install() + # Not because detection failed - the GPU was found, it just does not need an index here. + assert "NVIDIA GPU detected" in done.stdout + + +def _install(sandbox: Sandbox, *args: str, **env: str) -> str: + sandbox.uv_log.unlink(missing_ok=True) + done = sandbox.run("--install", "--extra", "runtime", *args, **env) + assert done.returncode == 0, done.stderr + return sandbox.project_install() + + def test_launch_prefers_our_venv_over_the_active_environment(sandbox: Sandbox) -> None: """The second leak: on-demand installs (torchao, the frontend) ran in the activated venv too.""" sandbox.make_venv_python() diff --git a/core/uv.lock b/core/uv.lock index e1b9a42..0099078 100644 --- a/core/uv.lock +++ b/core/uv.lock @@ -20,8 +20,8 @@ dependencies = [ { name = "psutil" }, { name = "pyyaml" }, { name = "safetensors" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/8d/75/94cd5d389649578aca399e5aa822637eec18319a1dadc400ffe2f9a7493f/accelerate-1.14.0.tar.gz", hash = "sha256:41b9c4377a54e0b460a959b0defa1b736e4ca0a2373252d9a539964c2afe3c8d", size = 412167, upload-time = "2026-06-11T13:45:52.326Z" } wheels = [ @@ -106,8 +106,8 @@ dependencies = [ { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, { name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, { name = "packaging" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/29/71/acff7af06c818664aa87ff73e17a52c7788ad746b72aea09d3cb8e424348/bitsandbytes-0.49.2-py3-none-manylinux_2_24_aarch64.whl", hash = "sha256:2fc0830c5f7169be36e60e11f2be067c8f812dfcb829801a8703735842450750", size = 31442815, upload-time = "2026-02-16T21:26:06.783Z" }, @@ -236,10 +236,9 @@ dependencies = [ { name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, { name = "scipy", version = "1.18.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, { name = "timm" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, - { name = "torchvision", version = "0.21.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform == 'win32'" }, - { name = "torchvision", version = "0.28.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, + { name = "torchvision" }, ] sdist = { url = "https://files.pythonhosted.org/packages/14/ad/2eb8cd9a8e17e35b9e5d39ad29afdde8fe810bda85e6e59117050519955d/controlnet_aux-0.0.10.tar.gz", hash = "sha256:31dc265a54448bdcee033a130b47423c80587fa35ccac752113af1b4d48f5183", size = 215016, upload-time = "2025-05-08T10:38:30.845Z" } wheels = [ @@ -350,8 +349,8 @@ version = "0.0.0b5" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "diffusers" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, { name = "transformers" }, ] sdist = { url = "https://files.pythonhosted.org/packages/5d/d4/dcd2901874d54745512b7e8a45abc9ea555057935deda5228e33a250c026/distvae-0.0.0b5.tar.gz", hash = "sha256:4376467ed2b7d6e9e7cab0bc174e49f1771535d07eaa8d2a86ef6f537e2977f6", size = 25068, upload-time = "2025-01-23T10:08:17.677Z" } @@ -602,7 +601,7 @@ wheels = [ [[package]] name = "inline-core" -version = "1.2.61" +version = "1.2.63" source = { editable = "." } dependencies = [ { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, @@ -629,8 +628,8 @@ all = [ { name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, { name = "scipy", version = "1.18.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, { name = "timm" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, { name = "torchao" }, { name = "transformers" }, { name = "uvicorn", extra = ["standard"] }, @@ -655,8 +654,8 @@ runtime = [ { name = "safetensors" }, { name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, { name = "scipy", version = "1.18.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, { name = "torchao" }, { name = "transformers" }, ] @@ -717,8 +716,8 @@ requires-dist = [ { name = "scipy", marker = "extra == 'runtime'", specifier = ">=1.11" }, { name = "timm", marker = "extra == 'all'", specifier = ">=1.0" }, { name = "timm", marker = "extra == 'training'", specifier = ">=1.0" }, - { name = "torch", marker = "sys_platform == 'win32' and extra == 'all'", specifier = ">=2.2", index = "https://download.pytorch.org/whl/cu124" }, - { name = "torch", marker = "sys_platform == 'win32' and extra == 'runtime'", specifier = ">=2.2", index = "https://download.pytorch.org/whl/cu124" }, + { name = "torch", marker = "sys_platform == 'win32' and extra == 'all'", specifier = ">=2.2", index = "https://download.pytorch.org/whl/cu126" }, + { name = "torch", marker = "sys_platform == 'win32' and extra == 'runtime'", specifier = ">=2.2", index = "https://download.pytorch.org/whl/cu126" }, { name = "torch", marker = "sys_platform != 'win32' and extra == 'all'", specifier = ">=2.2" }, { name = "torch", marker = "sys_platform != 'win32' and extra == 'runtime'", specifier = ">=2.2" }, { name = "torchao", marker = "extra == 'all'", specifier = ">=0.14" }, @@ -1256,8 +1255,8 @@ dependencies = [ { name = "psutil" }, { name = "pyyaml" }, { name = "safetensors" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, { name = "tqdm" }, { name = "transformers" }, ] @@ -2092,30 +2091,10 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ec/bb/2799cc2ede3ed41131f8975621e7213dfc7ef4acbbaadfa440f32500c370/starlette-1.3.1-py3-none-any.whl", hash = "sha256:c7372aae11c3c3f26a42df7bd626cec2f47d03483d261d369516a615a53714c6", size = 73632, upload-time = "2026-06-12T09:23:10.017Z" }, ] -[[package]] -name = "sympy" -version = "1.13.1" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.12' and sys_platform == 'win32'", - "python_full_version < '3.12' and sys_platform == 'win32'", -] -dependencies = [ - { name = "mpmath" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/ca/99/5a5b6f19ff9f083671ddf7b9632028436167cd3d33e11015754e41b249a4/sympy-1.13.1.tar.gz", hash = "sha256:9cebf7e04ff162015ce31c9c6c9144daa34a93bd082f54fd8f12deca4f47515f", size = 7533040, upload-time = "2024-07-19T09:26:51.238Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b2/fe/81695a1aa331a842b582453b605175f419fe8540355886031328089d840a/sympy-1.13.1-py3-none-any.whl", hash = "sha256:db36cdc64bf61b9b24578b6f7bab1ecdd2452cf008f34faa33776680c26d66f8", size = 6189177, upload-time = "2024-07-19T09:26:48.863Z" }, -] - [[package]] name = "sympy" version = "1.14.0" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.12' and sys_platform != 'win32'", - "python_full_version < '3.12' and sys_platform != 'win32'", -] dependencies = [ { name = "mpmath" }, ] @@ -2164,10 +2143,9 @@ dependencies = [ { name = "huggingface-hub" }, { name = "pyyaml" }, { name = "safetensors" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, - { name = "torchvision", version = "0.21.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform == 'win32'" }, - { name = "torchvision", version = "0.28.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, + { name = "torchvision" }, ] sdist = { url = "https://files.pythonhosted.org/packages/35/03/e41389ac641747bfec48d016fde8be1eade1901e6f2c1aedcb0c8cb4b5d9/timm-1.0.28.tar.gz", hash = "sha256:3789d313fdd5541a327b60180d70dbb4bdec73db8ff0655e413db3c3d134a9a4", size = 2451413, upload-time = "2026-07-11T17:24:32.615Z" } wheels = [ @@ -2200,29 +2178,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/72/f4/0de46cfa12cdcbcd464cc59fde36912af405696f687e53a091fb432f694c/tokenizers-0.22.2-cp39-abi3-win_arm64.whl", hash = "sha256:9ce725d22864a1e965217204946f830c37876eee3b2ba6fc6255e8e903d5fcbc", size = 2612133, upload-time = "2026-01-05T10:45:17.232Z" }, ] -[[package]] -name = "torch" -version = "2.6.0+cu124" -source = { registry = "https://download.pytorch.org/whl/cu124" } -resolution-markers = [ - "python_full_version >= '3.12' and sys_platform == 'win32'", - "python_full_version < '3.12' and sys_platform == 'win32'", -] -dependencies = [ - { name = "filelock" }, - { name = "fsspec" }, - { name = "jinja2" }, - { name = "networkx" }, - { name = "setuptools", marker = "python_full_version >= '3.12'" }, - { name = "sympy", version = "1.13.1", source = { registry = "https://pypi.org/simple" } }, - { name = "typing-extensions" }, -] -wheels = [ - { url = "https://download-r2.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp311-cp311-win_amd64.whl", hash = "sha256:6a1fb2714e9323f11edb6e8abf7aad5f79e45ad25c081cde87681a18d99c29eb", upload-time = "2025-01-30T00:55:31Z" }, - { url = "https://download-r2.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp312-cp312-win_amd64.whl", hash = "sha256:3313061c1fec4c7310cf47944e84513dcd27b6173b72a349bb7ca68d0ee6e9c0", upload-time = "2025-01-30T00:57:40Z" }, - { url = "https://download-r2.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp313-cp313-win_amd64.whl", hash = "sha256:519330eef09534acad8110b6f423d2fe58c1d8e9ada999ed077a637a0021f908", upload-time = "2025-01-30T00:59:45Z" }, -] - [[package]] name = "torch" version = "2.13.0" @@ -2243,7 +2198,7 @@ dependencies = [ { name = "nvidia-nccl-cu13", marker = "sys_platform == 'linux'" }, { name = "nvidia-nvshmem-cu13", marker = "sys_platform == 'linux'" }, { name = "setuptools" }, - { name = "sympy", version = "1.14.0", source = { registry = "https://pypi.org/simple" } }, + { name = "sympy" }, { name = "triton", marker = "python_full_version < '3.15' and sys_platform == 'linux'" }, { name = "typing-extensions" }, ] @@ -2266,64 +2221,71 @@ wheels = [ ] [[package]] -name = "torchao" -version = "0.17.0" -source = { registry = "https://pypi.org/simple" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/32/fe/a4036a8e80fa800c92dbcbf75f541cd4c106248b6b579db6dab1800f616a/torchao-0.17.0-cp310-abi3-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:87a418ce0ec064a821ceab83c921b501acef0ce9a6ccd1be358fcd16c3ae8c58", size = 3206172, upload-time = "2026-03-30T22:25:52.974Z" }, - { url = "https://files.pythonhosted.org/packages/c9/37/ef37ca885265e5f79a168616767dd416a3cea1cc3b28bb6b503ce4a5b652/torchao-0.17.0-py3-none-any.whl", hash = "sha256:02eba449036715b9ae784fbaa1a6f97994bb7b0421ce92d1d5d1c08e5bd6d349", size = 1200680, upload-time = "2026-03-30T22:25:54.457Z" }, -] - -[[package]] -name = "torchvision" -version = "0.21.0" -source = { registry = "https://pypi.org/simple" } +name = "torch" +version = "2.13.0+cu126" +source = { registry = "https://download.pytorch.org/whl/cu126" } resolution-markers = [ "python_full_version >= '3.12' and sys_platform == 'win32'", "python_full_version < '3.12' and sys_platform == 'win32'", ] dependencies = [ - { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, - { name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, - { name = "pillow" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" } }, + { name = "filelock" }, + { name = "fsspec" }, + { name = "jinja2" }, + { name = "networkx" }, + { name = "setuptools" }, + { name = "sympy" }, + { name = "typing-extensions" }, ] wheels = [ - { url = "https://files.pythonhosted.org/packages/88/53/4ad334b9b1d8dd99836869fec139cb74a27781298360b91b9506c53f1d10/torchvision-0.21.0-cp311-cp311-win_amd64.whl", hash = "sha256:49bcfad8cfe2c27dee116c45d4f866d7974bcf14a5a9fbef893635deae322f2f", size = 1560523, upload-time = "2025-01-29T16:28:48.751Z" }, - { url = "https://files.pythonhosted.org/packages/4c/6a/c7752603060d076dfed95135b78b047dc71792630cbcb022e3693d6f32ef/torchvision-0.21.0-cp312-cp312-win_amd64.whl", hash = "sha256:6eb75d41e3bbfc2f7642d0abba9383cc9ae6c5a4ca8d6b00628c225e1eaa63b3", size = 1560520, upload-time = "2025-01-29T16:28:42.122Z" }, - { url = "https://files.pythonhosted.org/packages/ed/b4/fc60e3bc003879d3de842baea258fffc3586f4b49cd435a5ba1e09c33315/torchvision-0.21.0-cp313-cp313-win_amd64.whl", hash = "sha256:9147f5e096a9270684e3befdee350f3cacafd48e0c54ab195f45790a9c146d67", size = 1560519, upload-time = "2025-01-29T16:28:22.527Z" }, + { url = "https://download-r2.pytorch.org/whl/cu126/torch-2.13.0%2Bcu126-cp311-cp311-win_amd64.whl", hash = "sha256:8095729db14e7fd5178a39676fdd679208eff4041407ea34e3d898336c90f5c5", upload-time = "2026-07-08T19:40:16Z" }, + { url = "https://download-r2.pytorch.org/whl/cu126/torch-2.13.0%2Bcu126-cp312-cp312-win_amd64.whl", hash = "sha256:380081ea098bf2b9e727aa85205d94790d884d17c62df3bb00a4f6a1047010a2", upload-time = "2026-07-08T19:44:46Z" }, + { url = "https://download-r2.pytorch.org/whl/cu126/torch-2.13.0%2Bcu126-cp313-cp313-win_amd64.whl", hash = "sha256:cb91b2f91d053bb91e413459a13ec8b036ff348d03e411e9c29988cc6edb7b32", upload-time = "2026-07-08T19:48:48Z" }, + { url = "https://download-r2.pytorch.org/whl/cu126/torch-2.13.0%2Bcu126-cp314-cp314-win_amd64.whl", hash = "sha256:136a8383b152dadc73d8944af9a796a76ff2038d59de666660d419a381b68078", upload-time = "2026-07-08T19:52:51Z" }, + { url = "https://download-r2.pytorch.org/whl/cu126/torch-2.13.0%2Bcu126-cp314-cp314t-win_amd64.whl", hash = "sha256:f50c0df3604e118003b5e598855be74a5725593b402fa855cc645f1ee18ffa9f", upload-time = "2026-07-08T19:56:53Z" }, +] + +[[package]] +name = "torchao" +version = "0.17.0" +source = { registry = "https://pypi.org/simple" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/32/fe/a4036a8e80fa800c92dbcbf75f541cd4c106248b6b579db6dab1800f616a/torchao-0.17.0-cp310-abi3-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:87a418ce0ec064a821ceab83c921b501acef0ce9a6ccd1be358fcd16c3ae8c58", size = 3206172, upload-time = "2026-03-30T22:25:52.974Z" }, + { url = "https://files.pythonhosted.org/packages/c9/37/ef37ca885265e5f79a168616767dd416a3cea1cc3b28bb6b503ce4a5b652/torchao-0.17.0-py3-none-any.whl", hash = "sha256:02eba449036715b9ae784fbaa1a6f97994bb7b0421ce92d1d5d1c08e5bd6d349", size = 1200680, upload-time = "2026-03-30T22:25:54.457Z" }, ] [[package]] name = "torchvision" version = "0.28.0" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.12' and sys_platform != 'win32'", - "python_full_version < '3.12' and sys_platform != 'win32'", -] dependencies = [ { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, { name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, { name = "pillow" }, - { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" } }, + { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/7a/b2/1e010052079e4c577007b789db336ea7075f1a426e84d17121fbc3745516/torchvision-0.28.0-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:83fe6c020866a85acd7d97deccc45ff11d66daf42916d04396a4309c66c0ccb8", size = 1856017, upload-time = "2026-07-08T16:07:55.533Z" }, { url = "https://files.pythonhosted.org/packages/27/be/1b9c5de9c655ca2df4a74100fa671a7b848532ff787e077ccde14a7dea2a/torchvision-0.28.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:5a38bc6da3d72621be003400b66f66a2b4c6d644fde05f680c2cb7ca8cf8dd6c", size = 7841822, upload-time = "2026-07-08T16:07:49.207Z" }, { url = "https://files.pythonhosted.org/packages/0b/9b/f1e68e861d4462e3e195a642c2b448e7b7d3fad5f209487162b9a2133d9b/torchvision-0.28.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:7e80f543b22503d9415e126db5f0ff3917036925e38560ee6b9ae38c571a4002", size = 7670718, upload-time = "2026-07-08T16:07:46.525Z" }, + { url = "https://files.pythonhosted.org/packages/f5/de/1494610ff54cbb154beb55033cc2cd50f3de04dac132fa2dd00e4f2b2556/torchvision-0.28.0-cp311-cp311-win_amd64.whl", hash = "sha256:9a45ea67235d965ef52187130d20002a4de20c54ea3d927a24286961d268dc37", size = 3814319, upload-time = "2026-07-08T16:07:37.153Z" }, { url = "https://files.pythonhosted.org/packages/15/49/c1cab1ecbb3ff1a380a3f99283db1dee61b8afe354f6352c643b65937130/torchvision-0.28.0-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:e9f54c30cd52e3ef7fd034cc69b7bb7e0964e1c8f8743e018ab92e95b40f9eee", size = 1856020, upload-time = "2026-07-08T16:07:52.182Z" }, { url = "https://files.pythonhosted.org/packages/f0/4c/95233776e2def960e5abb7a07931230a545f43717a56a1e1140162033598/torchvision-0.28.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:5cf78ebc401ce64ae19b8c55de866bb836797d559a4de9c25ccbe74cfa642d3a", size = 7842127, upload-time = "2026-07-08T16:07:53.446Z" }, { url = "https://files.pythonhosted.org/packages/93/e4/e9b2495d0d57b9f60d63c57d0a910410a81b4b073bf70917bef815291119/torchvision-0.28.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:028a3d481b37d785605620d7cdad897064c5a55bae2aa1f2658766333e291940", size = 7675040, upload-time = "2026-07-08T16:07:58.017Z" }, + { url = "https://files.pythonhosted.org/packages/7c/9c/55ed9cb6dfe3ee9c837df5cd0e758372e5829aa38b8dd71343aa632cc4e2/torchvision-0.28.0-cp312-cp312-win_amd64.whl", hash = "sha256:87dc16b2df427c1318ad335f1e2be2b3b15b2cf20f7934c83b0505a48425ee5d", size = 4085785, upload-time = "2026-07-08T16:07:50.928Z" }, { url = "https://files.pythonhosted.org/packages/20/55/08a726c14c67b37c8aca04b077766909f1c7ed23f76116884fe63b9bd033/torchvision-0.28.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:d483b4aa3f5237569053f749cd1a2b5bb548ca456e40461a5dd087f21149d123", size = 1856021, upload-time = "2026-07-08T16:07:45.386Z" }, { url = "https://files.pythonhosted.org/packages/db/8f/40beacd53809194f5259e590d1afaeaa8ad57da15f77c646e6560bcc4616/torchvision-0.28.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:bb6dd6918460ed89cc7644adcc2402991474d6933cf1ce92b390641cb233fddf", size = 7797014, upload-time = "2026-07-08T16:07:43.04Z" }, { url = "https://files.pythonhosted.org/packages/32/db/062cdb5a84380a60439775311fff34d89229760d2a50680393dc18699956/torchvision-0.28.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:ad7b3a439265cc3739a4ab5b4c998c0e38ea99c0ee7ca4dea35c5d0b099ec237", size = 7674669, upload-time = "2026-07-08T16:07:38.91Z" }, + { url = "https://files.pythonhosted.org/packages/f3/a6/b4081e2d04e1541abf82785ac9e5178a494c19330391f551356c8c18b7b3/torchvision-0.28.0-cp313-cp313-win_amd64.whl", hash = "sha256:7e9dd6f60d6e15f8dc27d4f877fdb6002fc70d70272412135f1c2ff9cfa08d3b", size = 4157380, upload-time = "2026-07-08T16:07:40.22Z" }, { url = "https://files.pythonhosted.org/packages/c5/b9/da40eca5bbe9596c12ae9899ab7abaf887f5e20f29d08b924b4633714821/torchvision-0.28.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:3bd9dba55224a9db4a2d77f6feaa5651770d8c8e86d3d0ddb0fa6bec54c8712b", size = 1856014, upload-time = "2026-07-08T16:07:44.282Z" }, { url = "https://files.pythonhosted.org/packages/06/d6/313aafd3df4eaf5f330211bd4e75b7598bddbfee4f55580d3b58536e1b20/torchvision-0.28.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:89f90e29b0966352811b12589f3a3c61943bf2bb9487b9d7bbec10efb1096bb5", size = 7796873, upload-time = "2026-07-08T16:07:30.907Z" }, { url = "https://files.pythonhosted.org/packages/b3/41/31f8e959ab8f942600b6357f8999c21d779d5fd3304b0fd204ff4b518239/torchvision-0.28.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:36beb0782976906069ca03d4c9aacaf4b6b838b06ed6c20960ea9c51cce7acdd", size = 7674634, upload-time = "2026-07-08T16:07:29.657Z" }, + { url = "https://files.pythonhosted.org/packages/15/15/4c5115253fd470672cdac0a1cf139e06b4f3e29d041238a2b255937f63be/torchvision-0.28.0-cp314-cp314-win_amd64.whl", hash = "sha256:3557cc7b539f46dabcda2b6f2b14017ccbeef024de466d4fc5835fc3f287f769", size = 4184005, upload-time = "2026-07-08T16:07:35.805Z" }, { url = "https://files.pythonhosted.org/packages/6a/80/822a6163da716f8a78141cf6678d74e26a572285d4ea866ef8aa657bb307/torchvision-0.28.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:09ce8f56e81f19b9c378ae7bb109f83f6659fd8bc3cd14241a48e4af46e9ed49", size = 1856011, upload-time = "2026-07-08T16:07:33.404Z" }, { url = "https://files.pythonhosted.org/packages/7f/d1/cd3f9463b39a790ec8c0c2f6e6c8061edb1562114d04fcdfa786ed889345/torchvision-0.28.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:62c7d110f86a039245b587e4fae60278c649f3bd42ff79cfbc1178eca4e72542", size = 7796742, upload-time = "2026-07-08T16:07:28.339Z" }, { url = "https://files.pythonhosted.org/packages/d9/82/3e0a7ad18e99831e2d7f4713d3be717b7159ff5a920862dd5c23c454aa71/torchvision-0.28.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:904cf89af220f8c6b2ed0296bb5065b474ce43b77558e48b2bf9de8b0ba17204", size = 7675526, upload-time = "2026-07-08T16:07:34.572Z" }, + { url = "https://files.pythonhosted.org/packages/18/d4/23aea03b28297bc66a4461f55ae4296368a9d85fa9a454bafcb2a5348bd7/torchvision-0.28.0-cp314-cp314t-win_amd64.whl", hash = "sha256:46f581979c010ad6da6bd85ee602aa707e1ff44312670223b7a0ee517ad06d47", size = 4291452, upload-time = "2026-07-08T16:07:32.236Z" }, ] [[package]] @@ -2696,8 +2658,8 @@ dependencies = [ { name = "distvae" }, { name = "einops" }, { name = "sentencepiece" }, - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, { name = "transformers" }, { name = "yunchang" }, ] @@ -2711,8 +2673,8 @@ name = "yunchang" version = "0.6.4" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "torch", version = "2.6.0+cu124", source = { registry = "https://download.pytorch.org/whl/cu124" }, marker = "sys_platform == 'win32'" }, { name = "torch", version = "2.13.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" }, + { name = "torch", version = "2.13.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "sys_platform == 'win32'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/f9/aa/307ef822c4dcd225658fa7a19b1986d6f7ed0c9600e3af0d81574a0bab5f/yunchang-0.6.4.tar.gz", hash = "sha256:9493ba28cd0f0daa3871f0c80a4876b866ead5db48e475708569d476804736f4", size = 40103, upload-time = "2026-01-13T08:14:19.5Z" } wheels = [ diff --git a/core/webui.bat b/core/webui.bat index f09f183..47bd38e 100644 --- a/core/webui.bat +++ b/core/webui.bat @@ -29,6 +29,8 @@ set "FORCE_REBUILD=0" set "SMART_MEMORY=0" set "USE_ACTIVE_ENV=0" set "RECREATE=0" +rem Empty means "decide from the GPU's compute capability at install time" (see :install_torch_args). +set "TORCH_CHOICE=%INLINE_TORCH_INDEX%" :parse if "%~1"=="" goto after_parse @@ -46,6 +48,7 @@ if /i "%~1"=="--models-dir" ( set "INLINE_MODELS_DIR=%~2" & shift & shift & go if /i "%~1"=="--data-dir" ( set "INLINE_DATA_DIR=%~2" & shift & shift & goto parse ) if /i "%~1"=="--install" ( set "RUN_INSTALL=1" & shift & goto parse ) if /i "%~1"=="--extra" ( set "EXTRAS=!EXTRAS!,%~2" & shift & shift & goto parse ) +if /i "%~1"=="--torch-index" ( set "TORCH_CHOICE=%~2" & shift & shift & goto parse ) if /i "%~1"=="--recreate" ( set "RECREATE=1" & shift & goto parse ) if /i "%~1"=="--use-active-env" ( set "USE_ACTIVE_ENV=1" & shift & goto parse ) if /i "%~1"=="--dev" ( set "DEV_MODE=1" & shift & goto parse ) @@ -125,23 +128,63 @@ echo Installing into the active environment: %ACTIVE_ENV% :install_torch_args rem PyPI's default torch is CPU-only on Windows, so installing blind generates on the CPU ~100x -rem slower with no error. When an NVIDIA GPU is present, resolve torch from the CUDA index. +rem slower with no error. Which CUDA index is right depends on the card: Blackwell (sm_120) has no +rem wheels before cu128, while cu126 is the last index still built for Maxwell..Volta (sm_50..sm_70). set "TORCH_ARGS=" +if defined TORCH_CHOICE goto install_torch_index where nvidia-smi >nul 2>nul || goto install_cpu nvidia-smi -L >nul 2>nul || goto install_cpu -rem unsafe-best-match: torchao is on the CUDA index too but only up to 0.9.0; without this uv's -rem first-index rule stops there and fails our torchao>=0.14 pin instead of finding it on PyPI. -set "TORCH_ARGS=--extra-index-url https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match" -echo NVIDIA GPU detected - installing the CUDA build of PyTorch. +rem compute_cap reads "12.0" on an RTX 50-series card. A driver too old to know the query leaves +rem CAP_MAJOR unset and lands on cu126, which covers the widest range of what people actually own. +set "CAP_MAJOR=" +for /f "usebackq tokens=1 delims=." %%c in (`nvidia-smi --query-gpu=compute_cap --format=csv,noheader 2^>nul`) do call :cap_max %%c +set "TORCH_CHOICE=cu126" +if defined CAP_MAJOR if !CAP_MAJOR! GEQ 10 set "TORCH_CHOICE=cu130" + +:install_torch_index +if /i "!TORCH_CHOICE!"=="cpu" goto install_cpu_forced +set "TORCH_URL=https://download.pytorch.org/whl/!TORCH_CHOICE!" +if /i "!TORCH_CHOICE:~0,4!"=="http" set "TORCH_URL=!TORCH_CHOICE!" +rem unsafe-best-match: torchao is on the CUDA index too, older there than our torchao>=0.14 pin on +rem some indexes; without this uv's first-index rule stops at that older copy instead of finding a +rem new enough one on PyPI. It also makes the +cuXXX local version outrank PyPI's plain one, which +rem is what pulls the CUDA build in rather than the CPU-only wheel PyPI serves on Windows. +rem no-sources-package: the pyproject pin names one fixed index, and the card decides here. +set "TORCH_ARGS=--extra-index-url !TORCH_URL! --index-strategy unsafe-best-match --no-sources-package torch" +echo NVIDIA GPU detected - installing the CUDA build of PyTorch (!TORCH_CHOICE!). goto install_pkgs + +:install_cpu_forced +echo Installing the default (CPU) build of PyTorch (--torch-index cpu). +goto install_pkgs + :install_cpu +set "TORCH_CHOICE=cpu" echo No NVIDIA GPU detected - installing the default (CPU) build of PyTorch. + :install_pkgs uv pip install --python "!TARGET_PY!" !TORCH_ARGS! -e ".[!EXTRAS!]" || goto fail uv pip install --python "!TARGET_PY!" inline-studio-frontend >nul 2>nul && echo Installed the prebuilt web UI (inline-studio-frontend). || echo Note: inline-studio-frontend not installed; the UI will build from source or run API-only. +rem A CPU-only wheel on a GPU box is silent at runtime and ~100x slower, so say it here rather than +rem let it through: it can still happen if PyPI ever outranks the CUDA index on version. +if /i "!TORCH_CHOICE!"=="cpu" goto install_done +"!TARGET_PY!" -c "import importlib, importlib.util, sys; spec = importlib.util.find_spec('torch'); sys.exit(0 if spec is None or importlib.import_module('torch').version.cuda else 1)" 2>nul && goto install_done +echo WARNING: the torch that got installed is a CPU-ONLY build. Generation would run on the +echo CPU, roughly 100x slower. Re-run with an explicit index, e.g. +echo .\webui.bat --install --torch-index cu126 + +:install_done echo Installed extras: !EXTRAS!. Start with: .\webui.bat exit /b 0 +rem Keeps the highest compute-capability major seen. Anything not a plain number is ignored: an +rem older driver answers an unknown query with an error string instead of failing outright. +:cap_max +echo(%~1| findstr /r /c:"^[0-9][0-9]*$" >nul || exit /b 0 +if not defined CAP_MAJOR set "CAP_MAJOR=%~1" +if %~1 GTR %CAP_MAJOR% set "CAP_MAJOR=%~1" +exit /b 0 + rem --- Pick the Python interpreter (and matching pip), in priority order ------------------------- rem Our own .venv outranks an env that merely happens to be activated, so a foreign venv can never rem absorb the on-demand installs in :ensure_frontend / :ensure_smart_memory_deps. @@ -302,6 +345,11 @@ echo --install create .venv (via uv) and install, then exit. An e echo reused, and an unrelated environment activated in your shell is never echo touched. echo --extra NAME add an install extra (repeatable): runtime, parallel, server, training +echo --torch-index WHICH with --install, override the PyTorch wheel index picked from your GPU's +echo compute capability. A short name (cu130, cu128, cu126), a full index +echo URL, or "cpu" to force the CPU-only build. Also settable as +echo INLINE_TORCH_INDEX. Use cu128 on a Blackwell card whose driver +echo predates CUDA 13. echo --recreate with --install, rebuild .venv from scratch (discards anything installed echo into it by hand) echo --use-active-env install into / run from the environment activated in this shell instead diff --git a/core/webui.sh b/core/webui.sh index edc201d..47fe665 100755 --- a/core/webui.sh +++ b/core/webui.sh @@ -29,6 +29,7 @@ FORCE_REBUILD=0 SMART_MEMORY=0 USE_ACTIVE_ENV=0 RECREATE=0 +TORCH_INDEX_CHOICE="${INLINE_TORCH_INDEX:-}" usage() { cat <<'EOF' @@ -62,6 +63,10 @@ Setup reused, and an unrelated environment activated in your shell is never touched. --extra NAME add an install extra (repeatable): runtime, parallel, server, training + --torch-index WHICH with --install, override the PyTorch wheel index picked from your GPU's + compute capability. A short name (cu130, cu128, cu126), a full index URL, + or "cpu" to force the CPU-only build. Also settable as INLINE_TORCH_INDEX. + Use cu128 on a Blackwell card whose driver predates CUDA 13. --recreate with --install, rebuild ./.venv from scratch (discards anything installed into it by hand, e.g. a ROCm build of PyTorch) --use-active-env install into / run from the environment activated in this shell instead of @@ -110,6 +115,7 @@ while [[ $# -gt 0 ]]; do --data-dir) export INLINE_DATA_DIR="${2:?--data-dir needs a path}"; shift 2 ;; --install) RUN_INSTALL=1; shift ;; --extra) EXTRAS="$EXTRAS,${2:?--extra needs a name}"; shift 2 ;; + --torch-index) TORCH_INDEX_CHOICE="${2:?--torch-index needs a name, URL or 'cpu'}"; shift 2 ;; --recreate) RECREATE=1; shift ;; --use-active-env) USE_ACTIVE_ENV=1; shift ;; --dev) DEV_MODE=1; shift ;; @@ -154,6 +160,45 @@ normalize_extras() { EXTRAS="$out" } +# Only Windows needs us to name a CUDA index: PyPI's torch is CPU-only there, while the Linux wheels +# already bundle CUDA. This script also runs under Git Bash / MSYS on Windows. +is_windows() { + case "$(uname -s 2>/dev/null || true)" in + MINGW*|MSYS*|CYGWIN*|Windows_NT) return 0 ;; + *) return 1 ;; + esac +} + +# The highest compute-capability major across the installed GPUs (12 for an RTX 50-series card). +# Fails when the driver is too old to answer the query, which the caller treats as "unknown". +gpu_compute_cap_major() { + local caps cap major best="" + caps="$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader 2>/dev/null)" || return 1 + while read -r cap; do + major="${cap%%.*}" + [[ "$major" =~ ^[0-9]+$ ]] || continue + if [[ -z "$best" || "$major" -gt "$best" ]]; then best="$major"; fi + done <<<"$caps" + [[ -n "$best" ]] || return 1 + printf '%s\n' "$best" +} + +# No single index covers every card: Blackwell (sm_100/sm_120) exists only from cu128 on, while cu126 +# is the last index still built for Maxwell..Volta (sm_50..sm_70). Unknown cards get cu126, the one +# that covers the widest range of what people actually own. +pick_torch_index() { + local major + major="$(gpu_compute_cap_major)" || { printf 'cu126\n'; return 0; } + if [[ "$major" -ge 10 ]]; then printf 'cu130\n'; else printf 'cu126\n'; fi +} + +torch_index_url() { + case "$1" in + http://*|https://*) printf '%s\n' "$1" ;; + *) printf 'https://download.pytorch.org/whl/%s\n' "$1" ;; + esac +} + if [[ "$RUN_INSTALL" -eq 1 ]]; then command -v uv >/dev/null 2>&1 || { echo "uv not found: https://docs.astral.sh/uv/" >&2; exit 1; } normalize_extras @@ -184,22 +229,51 @@ if [[ "$RUN_INSTALL" -eq 1 ]]; then fi # Pick the right torch wheel. PyPI's default torch is CPU-only on Windows (Linux wheels bundle # CUDA), so installing blind there yields a working install that generates on the CPU ~100x - # slower, with no error. When an NVIDIA GPU is present, resolve torch from the CUDA index. + # slower, with no error. Which CUDA index is right depends on the card - see pick_torch_index. TORCH_INDEX=() - if command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi -L >/dev/null 2>&1; then - # unsafe-best-match: torchao is on the CUDA index too but only up to 0.9.0; without this uv's - # first-index rule stops there and fails our torchao>=0.14 pin instead of finding it on PyPI. - TORCH_INDEX=(--extra-index-url https://download.pytorch.org/whl/cu124 \ - --index-strategy unsafe-best-match) + TORCH_CHOICE="$TORCH_INDEX_CHOICE" + GPU_PRESENT=0 + if command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi -L >/dev/null 2>&1; then GPU_PRESENT=1; fi + if [[ -z "$TORCH_CHOICE" ]]; then + if [[ "$GPU_PRESENT" -eq 1 ]]; then + if is_windows; then TORCH_CHOICE="$(pick_torch_index)"; fi + else + TORCH_CHOICE="cpu" + fi + fi + if [[ "$TORCH_CHOICE" == "cpu" && "$GPU_PRESENT" -eq 0 ]]; then + echo "No NVIDIA GPU detected - installing the default (CPU) build of PyTorch." + elif [[ "$TORCH_CHOICE" == "cpu" ]]; then + echo "Installing the default (CPU) build of PyTorch (--torch-index cpu)." + elif [[ -z "$TORCH_CHOICE" ]]; then echo "NVIDIA GPU detected - installing the CUDA build of PyTorch." else - echo "No NVIDIA GPU detected - installing the default (CPU) build of PyTorch." + # unsafe-best-match: torchao is on the CUDA index too but older there than our torchao>=0.14 pin + # on some indexes; without this uv's first-index rule stops at that older copy instead of + # finding a new enough one on PyPI. It also makes the +cuXXX local version outrank PyPI's plain + # one, which is what pulls the CUDA build in on Windows. + # no-sources-package: the pyproject pin names one fixed index, and the whole point here is that + # the card decides. + TORCH_INDEX=(--extra-index-url "$(torch_index_url "$TORCH_CHOICE")" \ + --index-strategy unsafe-best-match --no-sources-package torch) + echo "NVIDIA GPU detected - installing the CUDA build of PyTorch ($TORCH_CHOICE)." fi uv pip install --python "$TARGET_PY" "${TORCH_INDEX[@]}" -e ".[$EXTRAS]" # Pull the prebuilt web UI so there's no Node build (best-effort - it may not be published yet). uv pip install --python "$TARGET_PY" inline-studio-frontend >/dev/null 2>&1 \ && echo "Installed the prebuilt web UI (inline-studio-frontend)." \ || echo "Note: inline-studio-frontend not installed; the UI will build from source or run API-only." + # A CPU-only wheel on a GPU box is silent at runtime and ~100x slower, so say it here rather than + # let the resolve fail quietly - it can still happen if PyPI outranks the CUDA index on version. + if [[ "$TORCH_CHOICE" != "cpu" ]] && ! "$TARGET_PY" -c 'import importlib.util, sys +if importlib.util.find_spec("torch") is None: + sys.exit(0) +import torch +sys.exit(0 if torch.version.cuda else 1)' 2>/dev/null; then + echo "WARNING: the torch that got installed is a CPU-ONLY build. Generation would run on the" + echo " CPU, roughly 100x slower. Re-run with an explicit index, e.g." + echo " ./webui.sh --install --torch-index cu126" + fi echo "Installed extras: $EXTRAS. Start with: ./webui.sh" exit 0 fi diff --git a/requirements.txt b/requirements.txt index 4c9bcad..c38e5d5 100644 --- a/requirements.txt +++ b/requirements.txt @@ -8,12 +8,18 @@ # inline-studio-frontend is the prebuilt SPA wheel; Core serves it automatically (no Node needed). # # NVIDIA GPU users, read this: PyPI's default `torch` is a CPU-ONLY build on Windows (the Linux -# wheels bundle CUDA). Installing without the line below on Windows gives you a working install that -# generates on the CPU - roughly 100x slower, with no error. Uncomment it, or run -# `webui.sh --install`, which detects the GPU and does it for you. Core also warns at startup if it -# finds an NVIDIA GPU behind a CPU-only torch. +# wheels bundle CUDA). Installing without one of the lines below on Windows gives you a working +# install that generates on the CPU - roughly 100x slower, with no error. Uncomment the one that +# matches your card, or run `webui.sh --install`, which reads the compute capability and does it for +# you. Core also warns at startup if it finds an NVIDIA GPU behind a CPU-only torch, or behind a +# build with no kernels for the card. # -# --extra-index-url https://download.pytorch.org/whl/cu124 +# RTX 50-series and newer (Blackwell, sm_120) - needs a CUDA 13 driver: +# --extra-index-url https://download.pytorch.org/whl/cu130 +# ...on a Blackwell card with an older driver, use cu128 instead. +# +# Everything older (GTX 10-series through RTX 40-series, sm_50..sm_90): +# --extra-index-url https://download.pytorch.org/whl/cu126 # # `[all]` is the whole app in one line: engine + local model runtime + LoRA trainer + server. For a # hosted-only (fal) setup with no local GPU generation, use `inline-core[server]` instead - that