Skip to content

Fix misleading GPU probe diagnostics and add a GPU offload toggle (GPU.7, GPU.8) - #31

Merged
NCCU-Schultz-Lab merged 1 commit into
mainfrom
gpu-diagnostics-and-toggle
Jul 29, 2026
Merged

Fix misleading GPU probe diagnostics and add a GPU offload toggle (GPU.7, GPU.8)#31
NCCU-Schultz-Lab merged 1 commit into
mainfrom
gpu-diagnostics-and-toggle

Conversation

@NCCU-Schultz-Lab

Copy link
Copy Markdown
Collaborator

Two fixes found by actually running the M-GPU validation recipe on a local CUDA
box (RTX 5060 Ti, CUDA 13.0) — the code has been GPU-code-complete but unvalidated
since session 54.

GPU.7 — a broken CUDA install was reported as "gpu4pyscf not installed"

gpu4pyscf-cuda13x does not declare the NVIDIA CUDA math libraries as
dependencies, so a by-the-book install left import gpu4pyscf failing with
ImportError: Failure finding "libnvJitLink.so". QuantUI reported that as:

GPU offload not available
  reason: gpu4pyscf not installed (see README → 'Optional: GPU acceleration')

The package was installed. The message sent the user back to an install step
they had already completed, with the real cause discarded.

Cause: ModuleNotFoundError is a subclass of ImportError, so
except ImportError cannot tell "the package is absent" from "the package is
present but its import chain is broken". Both landed in the same branch — and that
branch logged nothing, so quantui log tail couldn't recover the cause either,
despite the docstring promising it would.

Now:

  • ModuleNotFoundError → "not installed", with the right pip install hint.
  • Any other ImportError → "installed but failed to import: {exc}", including the
    missing library name, plus a pointer that CUDA libraries are the usual cause.
  • Both paths log; the broken-chain case logs at WARNING.
  • probe_gpu() -> (available, name, reason) is now the single source of truth.
    The CLI previously re-implemented the probe to build its own reason string,
    which is precisely how it drifted from what the dispatcher actually decided. The
    Status badge reads the same triple.

is_gpu_available() keeps its exact 2-tuple contract; the cache moved onto the
shared probe and cache_clear / cache_info are forwarded, so existing callers
and tests are unaffected.

GPU.8 — warn when a detected GPU probably won't help

Quantum-chemistry SCF is FP64 throughout. Consumer and workstation GPUs gate
double precision to roughly 1/32–1/64 of their single-precision rate, so offload
on such a card can be slower than the CPU. Measured on this box (4096² DGEMM):

float32 float64
CPU (20 cores) 1 550 GFLOP/s 792
RTX 5060 Ti 15 346 GFLOP/s 346

Real B3LYP single points bore that out — 0.44×–0.91× of CPU wall time across
benzene/def2-SVP, benzene/def2-TZVP and ibuprofen/def2-TZVP. Before this change,
any detected CUDA device was presented as unqualified good news.

  • is_low_fp64_device(name) classifies by device name against the known
    strong-FP64 datacenter families. Unknown names are classified low-FP64
    deliberately: consumer hardware is the common student case, and a spurious
    advisory costs far less than a silent halving of throughput.
  • Advisory shown in the Status tab and by quantui gpu check.
  • Offload still auto-engages. We warn rather than override: FP64 weakness is a
    tendency, not a guarantee, and a name-based auto-skip would silently overrule
    the user on hardware the heuristic got wrong.
  • New persistent toggle — Status tab → Settings → "Use GPU when available",
    stored as compute.gpu_enabled. Additive settings section, schema version
    unchanged, unreadable settings never disable the GPU.
    QUANTUI_DISABLE_GPU=1 still wins over it, so scripted benchmarks are
    unaffected.

The toggle clears the detection cache on change — without that it would appear to
do nothing until restart, since the probe is cached for the process lifetime.

Testing

  • New tests/test_gpu_diagnostics.py — 28 tests: the ImportError/ModuleNotFoundError
    split (including the exact libnvJitLink.so failure), logging, the preserved
    is_gpu_available contract and its cache-clear behavior, the settings gate, and
    FP64 classification across real device names. Platform-independent; no GPU, no CUDA.
  • Two existing CLI tests were rewritten, deliberately.
    test_reports_missing_gpu4pyscf asserted that a bare ImportError produced
    "gpu4pyscf not installed" — it had encoded the bug. It now uses
    ModuleNotFoundError, and a new test locks the broken-chain case so the
    regression can't return. test_happy_path_when_gpu_detected was patching
    is_gpu_available; it now patches probe_gpu.
  • Verified on the real GPU: the advisory fires on the RTX 5060 Ti; toggling the
    setting off makes an actual B3LYP run return gpu_used=False; toggling it back
    on restores gpu_used=True.
  • Full suite: 1882 passed, 17 skipped. pre-commit clean at the pinned
    versions.

Not in this PR

The M-GPU speedup exit criterion still needs an H200 — it cannot be met on a
consumer card for the FP64 reason above. Everything upstream of it (detection,
migration, numerical agreement to 8 dp, gpu_usedperf_log → dashboard
pairing) is now validated locally, so that H200 session should be short.

🤖 Generated with Claude Code

GPU.7 — a gpu4pyscf install whose CUDA math libraries were missing failed
with ImportError: Failure finding "libnvJitLink.so", and QuantUI reported
it as "gpu4pyscf not installed", pointing the user back at an install step
they had already completed. ModuleNotFoundError is a subclass of
ImportError, so catching the latter conflated "absent" with "present but
broken". The two are now caught separately, the real exception reaches the
message, and both cases log. probe_gpu() returns (available, name, reason)
and is the single source of truth — the CLI previously re-derived its own
reason, which is how it came to contradict the dispatcher.

GPU.8 — SCF is FP64 throughout and consumer GPUs gate double precision to
~1/32-1/64 of FP32, so offload there can be slower than a many-core CPU
(measured: RTX 5060 Ti at 0.44x a 20-core CPU). Detected devices are now
classified by name; consumer-class cards get an advisory in the Status tab
and from `quantui gpu check`. Offload still auto-engages — we warn rather
than override, since FP64 weakness is a tendency and a name-based auto-skip
would silently overrule the user on hardware we guessed wrong about. A
persistent Status-tab checkbox (compute.gpu_enabled, additive settings
section) makes opting out one click; QUANTUI_DISABLE_GPU=1 still wins.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@NCCU-Schultz-Lab
NCCU-Schultz-Lab merged commit 7431999 into main Jul 29, 2026
5 checks passed
@NCCU-Schultz-Lab
NCCU-Schultz-Lab deleted the gpu-diagnostics-and-toggle branch July 29, 2026 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant