Success report: 2x RTX 3090, NVIDIA driver 620.70.06 (follow-up to #139): unlock still works, suspend OK

[Bot-AI-PC]

The unlock still works on NVIDIA driver 620.70.06. Follow-up to my earlier report #139 (same system, then on 615.71.10). I found no other 620 report.

• Cards: 2x EVGA RTX 3090 24 GB (10de:2204), desktop, NVLink bridge. Power limit stock 420 W on both (nvidia-smi: current = enforced = default = 420 W; unchanged for all tests below). Seasonic 1600 W PSU.
• Ubuntu 26.04.1, kernel 7.0.0-42-generic, nvidia-dkms-open 620.70.06 from NVIDIA's CUDA apt repo (DKMS built on the stock 620 source, no patches needed)
• gpu-unlock af28cbb (1.0.0), same install as in Compatibility report: 2x EVGA RTX 3090 (10de:2204), Ubuntu 26.04.1, dracut, driver 615.71.10 #139; the pinned 610.43.03 firmware is unchanged. I have not tried the current 2.0.0 (the last version I built was af28cbb, 1.0.0).
• Only the driver packages changed (615.71.10 -> 620.70.06), then a reboot. Both cards log gpu_unlock: unlocked, no NVRM errors.

Summary, both cards together, 615.71.10 -> 620.70.06 (same system, same tests as #139; only the driver packages changed):

Test	615.71.10	620.70.06	Change
Unlock on boot	both cards unlocked	both cards unlocked	same
BF16 GEMM, sum of cards	251.5 TFLOPS	251.9 TFLOPS	+0.2%
LoRA training, both loaded at once, sum	3,036 tok/s	3,073 tok/s	+1.2%
llama.cpp token generation, sum of cards	397.5 tok/s	402.9 tok/s	+1.4%
MTP generation, both at once, sum of medians	627.9 tok/s	623.9 tok/s	-0.6% (within noise)
Power under dual load, average sum	764 W	761 W	same
Max temperature	88 / 73 C	88 / 73 C	same
PCIe errors, Xid	none	none	same
Power limit	420 W on both	420 W on both	unchanged
Suspend/resume	worked (#139)	worked: unlocked again on both cards, same speed after resume

Details per card follow.

Same benchmarks as #139, 615.71.10 vs 620.70.06 (8192^3 GEMM, FP32 accumulate, torch 2.14.1+cu132):

	615.71.10	620.70.06
01:00.0 BF16 / FP16 / TF32 (TFLOPS)	129.6 / 126.3 / 68.8	129.7 / 126.0 / 68.5
03:00.0 BF16 / FP16 / TF32 (TFLOPS)	121.9 / 119.8 / 65.9	122.2 / 119.8 / 66.1
LoRA Mistral-7B step, 01:00.0 / 03:00.0	2.652 s / 2.694 s (#139)	2.580 s / 2.695 s
llama-bench Nemotron 3.5 Lightning 30B-A3B Q4_0, 01:00.0 pp512 / pp2048 / tg128	4683 / 4675 / 199.9 t/s (#139)	4784 / 4786 / 205.2 t/s
same, 03:00.0	4579 / 4507 / 197.6 t/s (#139)	4590 / 4500 / 197.7 t/s

The unlock gives the same speed-up on 620 as on 615. The small gain on 01:00.0 in LoRA and llama.cpp is within what I'd expect from run-to-run variation; the 615 numbers are from 2026-10-25, so I can't say it's the driver.

Both cards loaded at once (as in my earlier comment on #139: one LoRA process per card, started together, 70 steps each, stock 420 W, NVLink bridge, Seasonic 1600 W), 615.71.10 vs 620.70.06:

	01:00.0 615	01:00.0 620	03:00.0 615	03:00.0 620
Median step, both running	2.703 s	2.640 s	2.693 s	2.691 s
Power limit	420 W	420 W	420 W	420 W
Power draw, average / peak	406 / 420 W	399 / 420 W	358 / 367 W	362 / 369 W
Average SM clock	1733 MHz	1707 MHz	1659 MHz	1662 MHz
Temperature, average / max	n/a / 88 C	82 / 88 C	n/a / 73 C	69 / 73 C

03:00.0 draws only about 360 W under this load, below its 420 W limit, and reports SW Power Cap active while doing so (355 W measured in a solo run; 01:00.0 reports the same flag at 418 W, right at its limit). It was the same on 615 (357 W average, 367 W peak), so it is not new with 620. I do not know what holds it there; the two cards have different VBIOS versions.

No Xid, AER, DPC or "fallen off the bus" messages and all PCIe error counters at 0 on both cards after the run. NVLink still NV4, 4 links at 14.062 GB/s each.

MTP speculative decoding (llama-server, Nemotron 3.5 Lightning 30B-A3B Q4_0 + MTP drafter, --spec-draft-n-max 3, 128K context, 500 tokens, temp 0.3). Because single runs vary by about +-10 tok/s, this is 10 runs per prompt on 620.70.06 (after a discarded warm-up), tok/s, mean +- standard deviation:

Prompt	01:00.0 alone	03:00.0 alone	both at once: 01:00.0	both at once: 03:00.0
Code	307.8 +- 11.7	301.0 +- 10.4	309.8 +- 14.3	307.9 +- 11.9
Prose	244.9 +- 9.9	243.1 +- 6.9	246.8 +- 11.3	239.7 +- 9.9
Reasoning	337.8 +- 9.1	323.0 +- 8.6	330.9 +- 10.3	331.0 +- 6.5

Running both cards at once costs neither card any speed. For reference, my 3-run medians on 615.71.10 (2026-10-31) were: code 312 / 310-314, prose 250 / 240-253, reasoning 335 / 325-335 (01:00.0 / 03:00.0 alone). Those are within about one standard deviation of the 620 means above, so I see no difference between the drivers. 03:00.0 is 2-4% slower than 01:00.0 here, in line with its lower clock under the power behaviour described above.

Suspend/resume on 620: still unlocked after resume, on both cards.

systemctl suspend (S3, deep), woken with the power button. The module ran again for both cards before nvidia resumed them, exactly as on 615:

PM: suspend entry (deep)
NVRM: knvlinkCoreShutdownDeviceLinks_IMPL: Need to shutdown all links unilaterally for GPU1
nvidia 0000:03:00.0: selector read: SS0=0x67506044 expected_unlock=0x88888888
nvidia 0000:03:00.0: selector write: SS0=0x88888888
nvidia 0000:03:00.0: selector read: SS1=0x7 expected_unlock=0x8
nvidia 0000:03:00.0: selector write: SS1=0x8
nvidia 0000:01:00.0: selector read: SS0=0x44530050 expected_unlock=0x88888888
nvidia 0000:01:00.0: selector write: SS0=0x88888888
nvidia 0000:01:00.0: selector read: SS1=0x1 expected_unlock=0x8
nvidia 0000:01:00.0: selector write: SS1=0x8
nvidia 0000:03:00.0: gpu_unlock: unlocked again after resume
nvidia 0000:01:00.0: gpu_unlock: unlocked again after resume
NVRM: GPU1 nvAssertFailedNoLog: Assertion failed: !pGpu->getProperty(pGpu, PDB_PROP_GPU_IN_PM_CODEPATH) @ mem_mapper.c:49
PM: suspend exit

GEMM benchmark right after resume: 01:00.0 BF16 128.9 / FP16 124.8 / TF32 67.8 TFLOPS; 03:00.0 BF16 123.9 / FP16 120.8 / TF32 66.1 TFLOPS (before suspend: 129.7 / 126.0 / 68.5 and 122.2 / 119.8 / 66.1). NVLink still NV4, 4 links, PCIe error counters 0 on both cards.

The nvAssertFailedNoLog line from #139 is still there on 620, but only once (GPU1) instead of four times (once GPU0, three GPU1). Nothing failed and the numbers are normal.