Windows: real GPU detection (AMD/Intel), VRAM, utilization %, and AMD temperature without external tools by Worlgun · Pull Request #57 · nathaninline/ajean

Worlgun · 2026-09-01T18:22:28Z

Problem

On Windows with an AMD or Intel GPU (no NVIDIA, no ROCm installed), the "MACHINE" panel's GPU/VRAM card always showed "(pas de GPU)" — handleVram only ever checks nvidia-smi -> amd-smi -> rocm-smi, and none of those three exist on a stock Windows + AMD/Intel setup. This was misleading: inference was running correctly on the GPU via the Vulkan backend the whole time, the UI just couldn't see it.

What this adds (3 commits, Windows-only — no-op elsewhere)

GPU identity + total VRAM via the configured engine's own --list-devices output (already parsed by parseListDevices for the model editor's device picker) — no external tool needed, and the list matches exactly what llama.cpp will use.
Live "used" VRAM + utilization % via the Windows GPU Adapter Memory / GPU Engine performance counters (system-wide, not per-process — --list-devices alone under-reports "used" when queried from a freshly spawned process while the real server already holds VRAM). Discrete vs. integrated adapters are matched by an architectural fact, not guesswork: integrated GPUs have no VRAM of their own, so Windows tracks their usage under Shared (not Dedicated) memory — Dedicated Usage stays ~0 for them permanently.
AMD hotspot/junction temperature via ADL (AMD Display Library, atiadlxx.dll, shipped by every AMD driver — the same API GPU-Z/HWiNFO use), loaded directly via syscall (no cgo). Windows exposes no WMI/perfmon counter for GPU temperature at all; struct layouts were taken from AMD's public SDK (github.com/GPUOpen-LibrariesAndSDKs/display-library, adl_structures.h/adl_defines.h) rather than guessed, since these calls misbehave silently if any struct size is even slightly off.

A 4th, small commit fixes a bug found after the first three: a PowerShell foreach loop that produces exactly one result collapses to a bare object instead of a one-element array, which broke the JSON contract with Go's json.Unmarshal on any machine where only one GPU adapter passes the significance filter — verified against a live single-adapter machine before/after.

Known limitations (documented in code)

Only the first AMD adapter ADL finds gets a temperature reading. ADL enumerates one entry per display output, not per physical card, so a machine with two real AMD GPUs would need dedup by bus/device/function to tell them apart — not implemented, since I couldn't test that configuration.
Hotspot (junction) temperature reads consistently hotter (~10°C in testing) than the "Edge" sensor Task Manager itself displays. Both are real, correct readings of different points on the die — hotspot is what actually drives thermal throttling and is what HWiNFO/GPU-Z treat as primary, which is why it's preferred here, but it won't match Task Manager's number exactly for the same card at the same moment.

Testing

Windows 11, AMD Radeon RX 7900 XTX (discrete) + Intel UHD Graphics 770 (integrated), Vulkan backend, rebased onto current main (v0.13.2). Verified:

Both GPUs show with correct name/total VRAM
"used" and "util" track real values (confirmed against Win32_PerfFormattedData_GPUPerformanceCounters_GPUAdapterMemory/GPUEngine directly, both idle and with a 27B model loaded — 0% idle, 96% during active generation)
AMD hotspot temperature matches AMD's own sensor reading in the same run
Intel iGPU correctly stays at util=0/temp=0 (no regression — Task Manager doesn't show Intel temperature either)
No crashes from the in-process ADL DLL calls across multiple engine restarts

Happy to adjust anything — this is my first contribution to the project, let me know if you'd want a different structure (e.g. squashed, or the edge/hotspot choice made configurable).

handleVram only ever checked nvidia-smi -> amd-smi -> rocm-smi to fill the "MACHINE" panel's GPU/VRAM card. None of those three exist on a stock Windows install with AMD or Intel graphics (no ROCm), so the panel always showed "(pas de GPU)" even while inference was correctly running on the GPU via the Vulkan backend. Adds a Windows-only fallback, vulkanVramGPUs: - GPU identity + total VRAM come from the configured engine's own `--list-devices` output (already parsed by parseListDevices for the model editor's device picker), so no external tool is needed and the list matches exactly what llama.cpp will actually use. - Live "used" VRAM is corrected via the Windows "GPU Adapter Memory" performance counter (system-wide, not per-process), since --list-devices probes free memory from a freshly spawned process and under-reports usage while another process (the real server) already holds VRAM. Discrete vs. integrated adapters are matched by checking for "intel" in the Vulkan device name, since integrated GPUs report their usage under Shared (not Dedicated) memory on Windows -- an architectural fact, not a coincidence of any particular machine. Tested on Windows 11 with an AMD Radeon RX 7900 XTX (discrete) + Intel UHD Graphics 770 (integrated), Vulkan backend. Both cards now show correct name/total/used in the MACHINE panel; used-VRAM tracks the real value (verified against Win32_PerfFormattedData_GPUPerformanceCounters_GPUAdapterMemory directly) both idle and with a 27B model loaded.

Follow-up to the previous commit (Windows VRAM detection): the MACHINE panel still showed util=0 and temp=0 for every GPU on Windows, since neither number is available through nvidia-smi/amd-smi/rocm-smi (absent) nor through --list-devices (Vulkan doesn't expose either). Utilization + accurate "used" (windowsAdapterStats, web_devices.go): Replaces the previous VRAM-only helper with one that reads BOTH the Windows "GPU Adapter Memory" and "GPU Engine" performance counters in a single PowerShell call, so the two numbers stay correctly paired per adapter. "util" is the max utilization across all engine types (3D, Compute, Copy...) for that adapter's LUID, matching what Task Manager shows. Adapter-to-device matching is still done by architecture, not guesswork: integrated GPUs have no VRAM of their own, so Windows tracks their memory under Shared (not Dedicated) Usage -- Dedicated Usage stays ~0 for them permanently, which is what separates "the discrete card" from "the iGPU" in the matching logic, regardless of machine or moment queried. Temperature (web_devices_adl_windows.go / _adl_other.go): Windows exposes GPU temperature through no WMI/perfmon counter at all -- Task Manager itself only gets it via a private DirectX telemetry API added for it in Windows 11 22H2, not reachable from a script. AMD's own answer to that is ADL (AMD Display Library, atiadlxx.dll, already shipped by every AMD driver): the same API GPU-Z/HWiNFO use. Loads it directly via syscall (no cgo), calls ADL2_Adapter_PMLog_Start + ADL2_New_QueryPMLogData_Get, and reads the hotspot/junction sensor (falls back to edge if hotspot isn't supported on a given card). Struct layouts were taken from AMD's public SDK (github.com/GPUOpen-LibrariesAndSDKs/display-library, adl_structures.h/adl_defines.h) rather than guessed -- these calls silently misbehave if any struct size is even slightly wrong, so two earlier from-memory attempts (int32 then int16 sensor pairs) were discarded once tested against a debug harness and found to return garbage, before pulling the real header settled it. Split into windows/other build-tagged files (not runtime.GOOS-gated inline like the rest of this fallback chain) because loading a Windows DLL via syscall doesn't compile on non-Windows GOOS at all -- the _other.go stub keeps the call site in web_devices.go unconditional. No-op on Linux/macOS, which already get real temperature via amd-smi/rocm-smi. Known limitations, both noted in code: - Reports only the first AMD adapter ADL finds. ADL enumerates one entry per display OUTPUT, not per physical card, so a machine with two real AMD GPUs would need dedup by bus/device/function to tell them apart -- not implemented, since it's not a configuration I could test against. - Hotspot (junction) temperature runs noticeably hotter than the "Edge" sensor Task Manager itself displays -- a real ~10C gap observed in testing, not a bug in either reading. Hotspot is what actually drives thermal throttling and is what enthusiast tools (HWiNFO, GPU-Z) treat as the primary number, which is why it's preferred here, but it will read higher than Task Manager's figure for the same card at the same moment. Tested on Windows 11, AMD Radeon RX 7900 XTX (discrete, ADL path) + Intel UHD Graphics 770 (integrated, correctly stays at util=0/temp=0 -- no regression, Task Manager doesn't show Intel temperature either). Verified util% tracks real load (0% idle, 96% during active generation) and hotspot temperature matches AMD's own sensor within the same run.

…terStats $r = foreach (...) { ... } in PowerShell unwraps to a bare scalar object instead of a one-element array when the loop produces exactly one result -- a well-known PowerShell footgun. On a machine where only one adapter passes the >5MB filter (e.g. right after a fresh boot, before anything else has touched the GPU), that meant $r stopped being an array right before ConvertTo-Json ever saw it, breaking Go's json.Unmarshal into []struct{...} silently (the error is swallowed, windowsAdapterStats just returns nil). Every "real" number layered on top of the base --list-devices reading -- used, util, and by extension the AMD ADL temperature call, which never even runs once this returns empty -- was quietly falling back to the earlier, much less accurate reading instead. Fix: wrap the loop body in @(...) to force array output regardless of how many adapters matched. Verified by hand against a live single-adapter machine before and after: {"items":{"dedicated":...}} (object, breaks unmarshal) vs {"items":[{"dedicated":...}]} (array, correct) for the identical underlying data.

…ency fix The temperature reading added in c27eb46 crashed the whole ajean-ui process (not just the engine -- same process) twice in real use on Windows, both times as a Windows access violation (0xc0000005) inside the ADL DLL call: 1. adlAMDHotspotTempC called syscall.NewCallback fresh on every invocation to build ADL's malloc callback. Go callbacks handed out this way are never freed for the life of the process and the runtime caps how many can exist (~2000) -- with /api/vram polled periodically by the UI, that budget ran out after about 108 minutes of uptime and the next call corrupted an unrelated callback slot. Fixed by hoisting the callback to a single package-level sync.OnceValue, computed once and reused for every call -- this part is a real, verified fix (kept in this commit for when temperature comes back). 2. After fixing (1), a deliberate concurrent-load test (50-80 parallel requests against /api/vram, the realistic worst case for a UI that polls this endpoint) crashed the process again, faster than before. This points at atiadlxx.dll itself not being safe to call from multiple goroutines at once -- ajean's HTTP server handles each request on its own goroutine, so two requests landing at the same moment both entering the DLL is a normal, expected occurrence, not an edge case. I don't yet have a fix I've verified under load (a mutex serializing all ADL calls is the obvious next attempt, untested). Rather than leave a native-DLL crash in a PR aimed at a shared codebase, this commit disables the AMD temperature reading -- applyWindowsDedicatedUsage now hardcodes temp=0/unsupported instead of calling adlAMDHotspotTempC -- while keeping the callback fix and all the ADL plumbing in place for whoever picks this back up (myself included). VRAM detection and utilization % are unaffected by either bug and have been stress-tested under the same concurrent load without issue. Sorry for the churn on an already-open PR -- I'd rather push the honest state (including what I broke and how) than leave something crash-prone in review.

Root-caused the second crash from the previous commit: a deliberate concurrent-load test (multiple parallel requests against /api/vram, the realistic case for a UI polling this endpoint) reproduced the access violation reliably. ajean's HTTP server runs every request on its own goroutine, so two requests both entering adlAMDHotspotTempC at the same moment is the normal case, not a rare edge case -- and AMD's ADL (atiadlxx.dll) predates goroutines by a decade and documents no thread-safety guarantee for concurrent callers. Fix: adlCallMu, a single mutex held for the DLL's whole create->start->query->destroy sequence, guarantees only one goroutine is ever inside the DLL at a time. The calls are fast (single-digit ms), so serializing them isn't a meaningful bottleneck even under heavy load. Re-enables the call this disabled last commit. Verified before pushing: - 3 waves of 100 concurrent requests against /api/vram (300 total) -- the exact pattern that crashed it twice before -- no crash, temp/used/ util all correct throughout - 550 further sequential requests (endurance/leak check) -- no crash, values stable - ajean-ui process handles/RSS flat across the whole run (1863 -> 1858 handles after the last 50 calls, RAM unchanged) -- no resource leak - chat generation unaffected during the concurrent runs (63 tok/s, consistent with pre-change numbers) - UI console clean on a fresh reload; the "no response after 30s" warnings seen mid-test were the browser's own poll queuing behind my hundreds of concurrent curl calls contending for the same mutex, not a crash or hang -- expected, and irrelevant at realistic (single-tab) polling rates

nathaninline · 2026-09-03T10:08:25Z

Nickel, la capture avec la RX 7900 XTX (VRAM + util + temp) et l'Intel UHD 770 détectées, c'était exactement ce qu'il me manquait pour valider (je n'ai pas de matériel AMD/Intel de mon côté). J'ai intégré la PR.

Ce qui m'a convaincu que c'était sûr à prendre : la voie Vulkan est un dernier repli, appelée seulement quand nvidia-smi/amd-smi/rocm-smi n'ont rien donné, donc zéro effet sur les configs NVIDIA existantes ; et tout échoue proprement en nil (DLL absente, PowerShell en erreur, moteur introuvable) sans jamais casser l'UI.

J'ai juste ajouté un recover() dans adlAMDHotspotTempC : adlDLL.Load() garantit la DLL mais pas que chaque fonction ADL y soit, donc sur une atiadlxx.dll très ancienne un LazyProc.Call pourrait paniquer. Avec le recover on retombe simplement sur « pas de température ».

Beau boulot sur la partie ADL (le mutex + le OnceValue sur le callback, les deux incidents que tu décris sont exactement le genre de piège qu'on ne voit qu'en charge réelle). Merci !

Worlgun · 2026-09-03T13:28:23Z

Avec plaisir ^^

…urnie par le contributeur) Sur un Windows non-NVIDIA, nvidia-smi/amd-smi/rocm-smi sont tous absents : la carte GPU/VRAM affichait « pas de GPU » alors que l'inference tournait via Vulkan. Repli ajoute a handleVram (DERNIER, apres nvidia/amd/rocm : aucun effet sur les configs NVIDIA) via --list-devices du moteur (identite + VRAM totale/libre, sans outil externe). Sur Windows, correction du "used"/"util" par les compteurs perf GPU Adapter Memory + GPU Engine (PowerShell), et temperature AMD via l'ADL (atiadlxx.dll, syscall). Degradation propre partout : DLL absente / PowerShell en erreur / moteur introuvable -> nil, l'UI reste comme avant. Ajout par rapport a la PR : recover() dans adlAMDHotspotTempC (une atiadlxx.dll ancienne sans une fonction ADL2 recente ferait paniquer LazyProc.Call). Valide : preuve reelle du contributeur (RX 7900 XTX + Intel UHD 770, VRAM/util/temp corrects) ; build Windows + cross Linux OK ; aucun effet sur le chemin NVIDIA (repli non atteint).

…il (nom, VRAM, utilisation, temperature AMD via Vulkan/ADL, sans effet sur les configs NVIDIA) ; synchronisation automatique du preset actif entre appareils sans rechargement ; fix conversation active qui revenait vide au redemarrage (lecture base indisponible confondue avec absence, desormais distinguee/reessayee/journalisee) ; second garde-fou quand le modele raisonne sans jamais repondre ; relevement de maxLogEvents pour ne plus tronquer le journal de rejeu sur un tour a tres long raisonnement Integre depuis les PR du fork (Worlgun) #57 #63 #61 #60 (partiellement), plus le fix de sync preset entre appareils.

Worlgun · 2026-09-03T22:28:21Z

Repris directement dans v0.13.6 (commit d357cfa). Fermeture — merci pour l'intégration !

Worlgun added 5 commits September 1, 2026 19:40

Worlgun closed this Sep 3, 2026

Navigation Menu

Windows: real GPU detection (AMD/Intel), VRAM, utilization %, and AMD temperature without external tools - #57

Windows: real GPU detection (AMD/Intel), VRAM, utilization %, and AMD temperature without external tools#57
Worlgun wants to merge 5 commits into
nathaninline:mainfrom
Worlgun:windows-gpu-detection

Worlgun commented Sep 1, 2026

Uh oh!

nathaninline commented Sep 3, 2026

Uh oh!

Worlgun commented Sep 3, 2026

Uh oh!

Worlgun commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Worlgun commented Sep 1, 2026

Problem

What this adds (3 commits, Windows-only — no-op elsewhere)

Known limitations (documented in code)

Testing

Uh oh!

nathaninline commented Sep 3, 2026

Uh oh!

Worlgun commented Sep 3, 2026

Uh oh!

Worlgun commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants