cuda version check based on hardware

README.md

@@ -37,7 +37,7 @@ It runs as a **single process on one port**: the Inline Core engine (Python) ser
 
 ## Get Started
 
-The built web UI ships as a Python package, so all you need is [Python 3.11+](https://python.org), no Node. **`--install --extra all` is the single command that installs everything** - the engine, the local model runtime, the LoRA trainer, and the UI. On an NVIDIA machine it detects the GPU and pulls the CUDA build of PyTorch for you.
+The built web UI ships as a Python package, so all you need is [Python 3.11+](https://python.org), no Node. **`--install --extra all` is the single command that installs everything** - the engine, the local model runtime, the LoRA trainer, and the UI. On an NVIDIA machine it reads the GPU's compute capability and pulls the CUDA build of PyTorch that has kernels for it, RTX 50-series included.
 
 **macOS / Linux:**
 
@@ -70,13 +70,29 @@ Prefer pip over the launcher? `pip install -r requirements.txt` (from the repo r
 
 Honest status - what's actually been run, versus what has a code path but no one has verified:
 
-| Hardware                | Status                                                                                              | Extra steps                                                                                                                                                                                                                                                                                                      |
-| ----------------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **NVIDIA, Linux**       | **Tested** - Z-Image Turbo 1024² on a T4 (16 GB); Krea 2 1024² and LoRA training on an L40S (48 GB) | None. `webui.sh --install` picks the CUDA build automatically.                                                                                                                                                                                                                                                   |
-| **NVIDIA, Windows**     | Supported, needs one step                                                                           | Run `.\webui.bat --install` (the Windows launcher; it detects the GPU). PyPI's default `torch` is **CPU-only on Windows**, so `--install` pulls the CUDA build for you, or install torch from `https://download.pytorch.org/whl/cu124`. Core warns at startup if it finds an NVIDIA GPU behind a CPU-only torch. |
-| **Apple Silicon (MPS)** | Code path exists, **untested**                                                                      | None. int8 quantisation doesn't apply on MPS, so a model too big for unified memory won't fit.                                                                                                                                                                                                                   |
-| **AMD (ROCm), Linux**   | **Untested** - reports welcome                                                                      | Needs a ROCm build of PyTorch - see [AMD (ROCm) setup](#amd-rocm-setup) below.                                                                                                                                                                                                                                   |
-| **CPU only**            | Works, very slow                                                                                    | `./webui.sh --cpu` (Windows: `.\webui.bat --cpu`)                                                                                                                                                                                                                                                                |
+| Hardware                | Status                                                                                              | Extra steps                                                                                                                                                                                                                                                                                                                                               |
+| ----------------------- | --------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **NVIDIA, Linux**       | **Tested** - Z-Image Turbo 1024² on a T4 (16 GB); Krea 2 1024² and LoRA training on an L40S (48 GB) | None. `webui.sh --install` picks the CUDA build automatically.                                                                                                                                                                                                                                                                                            |
+| **NVIDIA, Windows**     | Supported, needs one step                                                                           | Run `.\webui.bat --install` (the Windows launcher). PyPI's default `torch` is **CPU-only on Windows**, so `--install` reads your GPU's compute capability and pulls the matching CUDA build: `cu130` for RTX 50-series (Blackwell), `cu126` for everything older. Override it with `--torch-index` - see [RTX 50-series](#rtx-50-series-blackwell) below. |
+| **Apple Silicon (MPS)** | Code path exists, **untested**                                                                      | None. int8 quantisation doesn't apply on MPS, so a model too big for unified memory won't fit.                                                                                                                                                                                                                                                            |
+| **AMD (ROCm), Linux**   | **Untested** - reports welcome                                                                      | Needs a ROCm build of PyTorch - see [AMD (ROCm) setup](#amd-rocm-setup) below.                                                                                                                                                                                                                                                                            |
+| **CPU only**            | Works, very slow                                                                                    | `./webui.sh --cpu` (Windows: `.\webui.bat --cpu`)                                                                                                                                                                                                                                                                                                         |
+
+#### RTX 50-series (Blackwell)
+
+RTX 50-series cards (5060/5070/5080/5090 and the RTX PRO Blackwell line) are compute capability **sm_120**, and no PyTorch wheel built for CUDA 12.4 or 12.6 has kernels for them. `--install` handles this: it reads the compute capability off the driver and picks `cu130`, so a plain `.\webui.bat --install --extra all` is all you need.
+
+Two cases where you may want to say it yourself:
+
+```powershell
+rem Blackwell card, but a driver older than CUDA 13 (R580) - cu128 has sm_120 and a lower floor
+.\webui.bat --install --extra all --torch-index cu128
+
+rem Or set it once for the shell, same effect
+set INLINE_TORCH_INDEX=cu128
+```
+
+`--torch-index` takes a short name (`cu130`, `cu128`, `cu126`), a full index URL, or `cpu` to force the CPU-only build. `webui.sh` takes the same flag. If the installed build turns out to have no kernels for your card, Core says so by name at startup rather than leaving you with PyTorch's own `sm_120 is not compatible` warning.
 
 #### AMD (ROCm) setup
 
@@ -100,7 +116,7 @@ Then run `./webui.sh` as usual.
 
 Three gotchas:
 
-- **Don't run `uv sync` afterwards** - it re-resolves the environment against the lockfile and will pull the PyPI torch back over your ROCm build. Use `uv pip install --python .venv/bin/python` for follow-up installs.
+- **Don't run `uv sync` afterwards** - it re-resolves the environment against the lockfile and will pull the PyPI torch back over your ROCm build. Use `uv pip install --python .venv/bin/python` for follow-up installs. The same applies to a hand-picked CUDA index.
 - **Don't pass `--recreate`** - it rebuilds `.venv` from scratch and your ROCm torch goes with it. A plain `--install` re-run reuses the venv and is safe.
 - ROCm presents itself through `torch.cuda`, so the engine will treat it as a CUDA device and may largely work. But the dtype heuristics key off **NVIDIA** compute capability (`< 8.0` → fp16), which is meaningless on RDNA/CDNA, and the int8 (torchao) path is unverified on ROCm. If it works - or doesn't - [open an issue](https://github.com/inlineresearch/Inline-Studio/issues); that's the fastest way to get AMD properly supported.
 
@@ -164,7 +180,7 @@ The friendly launcher (in `core/`) maps flags onto the engine's `INLINE_*` envir
 
 </details>
 
-`webui.sh` also has `--install` / `--extra NAME` to set up the venv, plus `--recreate` (rebuild `.venv` from scratch) and `--use-active-env` (install into / run from the environment activated in your shell instead of `.venv`). New to Inline Studio? The [Getting Started guide](https://inlinestudio.art/getting-started) walks you through your first render.
+`webui.sh` also has `--install` / `--extra NAME` to set up the venv, plus `--torch-index WHICH` (`INLINE_TORCH_INDEX`) to override the PyTorch wheel index picked from your GPU's compute capability, `--recreate` (rebuild `.venv` from scratch) and `--use-active-env` (install into / run from the environment activated in your shell instead of `.venv`). New to Inline Studio? The [Getting Started guide](https://inlinestudio.art/getting-started) walks you through your first render.
 
 ## Features
 

core/pyproject.toml

@@ -101,13 +101,18 @@ all = [
 ]
 
 # PyPI ships CPU-only torch on Windows; pin the CUDA index there (pip: see requirements.txt).
+# cu126 rather than a newer index because it is the last one still built for sm_50..sm_70 (Maxwell
+# through Volta), so it is the safe default for a card we know nothing about. Blackwell (sm_120) has
+# no wheel here at all - webui.sh/.bat detect the compute capability and pass the matching index.
 [[tool.uv.index]]
-name = "pytorch-cu124"
-url = "https://download.pytorch.org/whl/cu124"
+name = "pytorch-cu126"
+url = "https://download.pytorch.org/whl/cu126"
 explicit = true
 
+# Launcher installs pass `--no-sources-package torch` so their detected index wins over this pin;
+# anything else added to this table would still apply to them.
 [tool.uv.sources]
-torch = [{ index = "pytorch-cu124", marker = "sys_platform == 'win32'" }]
+torch = [{ index = "pytorch-cu126", marker = "sys_platform == 'win32'" }]
 
 [project.scripts]
 inline-studio = "inline_core.server.__main__:main"

core/src/inline_core/device/detect.py

@@ -76,10 +76,58 @@ def cpu_only_torch_warning() -> str | None:
         "This install has a CPU-ONLY build of PyTorch, but an NVIDIA GPU was detected. "
         "Generation will run on the CPU and be roughly 100x slower. Reinstall torch from the CUDA "
         "index, e.g. `pip install --force-reinstall --extra-index-url "
-        "https://download.pytorch.org/whl/cu124 torch` (or re-run `webui.sh --install`)."
+        "https://download.pytorch.org/whl/cu126 torch` (or re-run `webui.sh --install`)."
     )
 
 
+def unsupported_arch_warning() -> str | None:
+    """A warning when the installed torch has no kernels for the GPU it is about to run on.
+
+    An RTX 50-series card (sm_120) under a wheel built for sm_50..sm_90 is the sharpest case: torch
+    reports a CUDA build and a visible device, so every check in ``cpu_only_torch_warning`` passes,
+    and the only clue the user gets is PyTorch's own late UserWarning followed by kernels that
+    cannot launch. Returns None on any uncertainty, same as the rest of this module.
+    """
+    try:
+        import torch
+
+        if getattr(torch.version, "hip", None):
+            return None  # the sm_ vocabulary is NVIDIA's; HIP reports gfx arches through it
+        get_arch_list = getattr(torch.cuda, "get_arch_list", None)
+        get_capability = getattr(torch.cuda, "get_device_capability", None)
+        device_count = getattr(torch.cuda, "device_count", None)
+        if get_arch_list is None or get_capability is None or device_count is None:
+            return None
+        if device_count() < 1:
+            return None
+        arches = [str(arch) for arch in get_arch_list() if str(arch).startswith("sm_")]
+        if not arches:
+            return None  # a CPU-only build; cpu_only_torch_warning owns that case
+        major, minor = get_capability(0)
+        target = f"sm_{major}{minor}"
+        # startswith, because a wheel lists tuned variants like sm_90a for the same architecture.
+        if any(arch.startswith(target) for arch in arches):
+            return None
+        name = _device_name(torch) or "The detected NVIDIA GPU"
+        return (
+            f"{name} is compute capability {target}, but this install's PyTorch only has kernels "
+            f"for {' '.join(arches)}. Generation will fail or fall back to the CPU. Re-run "
+            "`webui.sh --install` (Windows: `.\\webui.bat --install`) to pick the wheel index that "
+            "matches the card, or force one with `--torch-index cu130` - `cu128` if the driver "
+            "predates CUDA 13."
+        )
+    except Exception:  # noqa: BLE001 - a diagnostic must never break startup
+        return None
+
+
+def _device_name(torch: object) -> str | None:
+    try:
+        get_name = getattr(getattr(torch, "cuda", None), "get_device_name", None)
+        return str(get_name(0)) if get_name is not None else None
+    except Exception:  # noqa: BLE001
+        return None
+
+
 def _nvidia_gpu_present() -> bool:
     """Best-effort check for NVIDIA hardware that does NOT rely on torch's CUDA support - that is
     the whole point, since we are called precisely when torch cannot see the GPU.

core/src/inline_core/server/__main__.py

@@ -19,7 +19,7 @@
 import uvicorn
 
 from ..config import data_dir, server_host, server_port
-from ..device.detect import cpu_only_torch_warning
+from ..device.detect import cpu_only_torch_warning, unsupported_arch_warning
 from ..device.memory import MemoryPolicy
 from ..extensions.loader import LoadedExtension
 from ..graph.cache import InMemoryCache
@@ -55,7 +55,7 @@ def main() -> None:
         print(f"Extensions: {_extension_summary(extensions)}")
     # A CPU-only torch wheel on a CUDA machine is a silent ~100x slowdown, so say it loudly here
     # rather than letting the user conclude the engine is just slow.
-    torch_warning = cpu_only_torch_warning()
+    torch_warning = cpu_only_torch_warning() or unsupported_arch_warning()
     if torch_warning:
         print(f"WARNING: {torch_warning}")
     frontend_root = resolve_frontend_root()

core/tests/test_device_detect.py

@@ -78,7 +78,8 @@ def test_bf16_gate_uses_the_capability_rule_on_nvidia(monkeypatch) -> None:
     from inline_core.device.types import Device, DeviceKind
 
     dev = Device(DeviceKind.CUDA, 0)
-    for capability, expected in (((8, 6), True), ((9, 0), True), ((7, 5), False), ((7, 0), False)):
+    caps = (((12, 0), True), ((8, 6), True), ((9, 0), True), ((7, 5), False), ((7, 0), False))
+    for capability, expected in caps:
         monkeypatch.setitem(
             __import__("sys").modules,
             "torch",
@@ -110,6 +111,73 @@ def test_bf16_gate_asks_torch_directly_on_rocm(monkeypatch) -> None:
     assert detect.cuda_supports_bf16(dev) is True
 
 
+# What a cu124 wheel reports - the build every Windows install used to be pinned to.
+_CU124_ARCHES = ["sm_50", "sm_60", "sm_61", "sm_70", "sm_75", "sm_80", "sm_86", "sm_90"]
+
+
+def _fake_arch_torch(
+    *, capability: tuple[int, int], arches: list[str], name: str = "NVIDIA GeForce RTX 5070 Ti"
+) -> types.SimpleNamespace:
+    return types.SimpleNamespace(
+        version=types.SimpleNamespace(cuda="12.4", hip=None),
+        cuda=types.SimpleNamespace(
+            device_count=lambda: 1,
+            get_arch_list=lambda: arches,
+            get_device_capability=lambda i=0: capability,
+            get_device_name=lambda i=0: name,
+        ),
+    )
+
+
+def test_warns_when_the_wheel_has_no_kernels_for_the_card(monkeypatch) -> None:
+    """The RTX 50-series failure: torch is a CUDA build and the device is visible, so every check in
+    cpu_only_torch_warning passes and the user is left with PyTorch's own cryptic UserWarning."""
+    monkeypatch.setitem(
+        __import__("sys").modules,
+        "torch",
+        _fake_arch_torch(capability=(12, 0), arches=_CU124_ARCHES),
+    )
+    warning = detect.unsupported_arch_warning()
+    assert warning is not None
+    assert "sm_120" in warning
+    assert "RTX 5070 Ti" in warning
+    assert "--torch-index" in warning  # tells them how to fix it
+
+
+def test_silent_when_the_wheel_covers_the_card(monkeypatch) -> None:
+    monkeypatch.setitem(
+        __import__("sys").modules,
+        "torch",
+        _fake_arch_torch(capability=(8, 6), arches=_CU124_ARCHES),
+    )
+    assert detect.unsupported_arch_warning() is None
+
+
+def test_arch_warning_accepts_a_tuned_variant(monkeypatch) -> None:
+    """Wheels list per-architecture variants like sm_90a; that is still a match for sm_90."""
+    monkeypatch.setitem(
+        __import__("sys").modules,
+        "torch",
+        _fake_arch_torch(capability=(9, 0), arches=["sm_80", "sm_90a"]),
+    )
+    assert detect.unsupported_arch_warning() is None
+
+
+def test_arch_warning_silent_when_torch_cannot_answer(monkeypatch) -> None:
+    """An older torch has no get_arch_list, and a CPU-only build reports no sm_ arches at all -
+    cpu_only_torch_warning owns that second case. Neither may produce a false alarm here."""
+    monkeypatch.setitem(
+        __import__("sys").modules, "torch", _fake_torch(cuda_available=False, cuda_version="12.4")
+    )
+    assert detect.unsupported_arch_warning() is None
+    monkeypatch.setitem(
+        __import__("sys").modules,
+        "torch",
+        _fake_arch_torch(capability=(12, 0), arches=[]),
+    )
+    assert detect.unsupported_arch_warning() is None
+
+
 def test_silent_without_torch(monkeypatch) -> None:
     """A hosted-only (fal) install deliberately has no torch; that is not a misconfiguration."""
     import builtins