Skip to content

Frontier 80GB

Jump to bottom
Consensus Protocol edited this page Jul 28, 2026 · 3 revisions

The 80 GB attempt on 10 GB cards

What this page covers. Two different attempts to give the 10 GB CMP 170HX (10de:2082) an 81920 MiB framebuffer instead of the 40960 MiB that ships, and why they came out differently:

  • The 80 driver branch, which is what almost every tester ran. As compiled it programs an incoherent register set, folds at exactly 40 GiB, and fails. This is the failure the rest of the wiki describes.
  • The driverless refire chain (a clean-room fire script, not a build of the 80 branch), which on 2026-07-23 to 2026-07-27 programmed the coherent set, reached 81920 MiB, and passed dense tagged write/readback tests with no fold at 77.5 GiB and again at 72 GiB. That result is real, unshipped, and carries hard limits.

The headline: shipping master gives a 10 GB card 40 GB, and 40 GB is the supported configuration. Nothing on this page is a recommendation to run 80 GB. The 80 branch never merged to master and its geometry dies above roughly 40 GiB; the driverless coherent set gets further but delivers roughly one CUDA context per fire before Xid 154 and about 79 % of peak bandwidth in the extended region.

!!! danger "Do not run the 80 branch on a card you care about" At 80 GB, kernels touching more than roughly 40 GB cause fatal GPU loss, independent of power limit. Reported Xid codes include Xid 31 (described as harmless) and Xid 154 after CUDA memory tests; the dominant reported symptom is hangs. Xid 31 alone was suggested by a bystander and was not corroborated as the signature by the operator with the failing card. The failure is not graceful: the allocation succeeds, the kernel launches, and the device is then gone until reset. No memory validation pass above 39 GB has ever been repeatable on this branch.

!!! warning "8 GB and 10 GB are different cards with different targets" An 8 GB card (10de:20c2) unlocks to 64 GB and that configuration is stable and in production. A 10 GB card (10de:2082) unlocks to 40 GB and that is also stable and is what shipping master gives you. The 80 GB configurations described here apply only to the 10 GB card and neither of them ships. Never substitute one profile for the other: see memory geometry.

The three geometries, side by side

Item 10 GB stock 10 GB shipping unlock 8 GB shipping unlock 80 branch, as actually built Coherent set, driverless refire
PCI ID 10de:2082 10de:2082 10de:20c2 10de:2082 10de:2082
Capacity 10240 MiB 40960 MiB 65536 MiB reports 81920 MiB reports 81920 MiB
CFG1 0x009a0204 0x02449000 0x02669000 0x02779000 0x02779000 0x02779000
CFG1 tier byte [23:16] 0x44 0x66 = 2048 MiB/FBPA 0x77 = 4096 MiB/FBPA 0x77 = 4096 MiB/FBPA 0x77
LMR 0x00100ce0 0x00000288 0x0000028A 0x0000020B 0x0000028A (40 GiB) 0x0000028B (80 GiB)
L2/LTC decode 0x17e2a0 native native native 0x70000300 (2 GB/ch) 0x10000300 (4 GB/ch)
targetFbBytes / fb_length native 0x0000000A00000000 0x0000001000000000 0x0000001400000000 (80 GiB) set by the fire script
Status shipping stable, ships stable, ships unstable, never merged experimental, no fold to 77.5 GiB, one context per fire

The bolded column is what ran on the card of everyone who installed the branch. The final column is a script-driven register state, not an installable build; the branch's own constants.yaml claims 0x0000028B but is never read by its build.

What the branch actually changes

Contrary to a widely repeated claim that the 80 branch "reaches the 80 GB geometry purely by changing constants" with "no patch file differing from master", a byte comparison shows otherwise. Patches 0002 through 0006 are identical to master. 0001-sec2-postbl-plm-ss-cfg.patch differs in exactly two lines:

-    cfg1Value = 0x02669000U;
+    cfg1Value = 0x02779000U;
-    targetFbBytes = 0x0000000A00000000ULL;
+    targetFbBytes = 0x0000001400000000ULL;

Plus changes to build.sh, install.sh and common/constants.yaml. Canonically: one patch file, two lines, plus three scripts.

The branch also extends the profile auto-detect so a reinstall on an already-80 GB card picks the right profile. Master maps >= 60000 MiB -> 8gb and 35000-59999 MiB -> 10gb; the 80 branch adds >= 75000 MiB -> 10gb.

The LMR that was never programmed

This is the most important finding on the page, and it was only established by reading the branch line by line rather than trusting its documentation.

80/common/constants.yaml carries:

lmr: "0x0000028B"
unlocked_mib: 81920

introduced by commit 3c53aca "Correct LMR for 80GB". Three other places disagree, and all three of them are on the path that actually gets compiled and run:

Location Value
80/driver/build.sh line 93 LMR="0x0000028A"
80/install.sh line 138 (printed to the user) Unlock geometry: 80GB (CFG1=0x02779000 LMR=0x0000028A)
80/driver/patches/0001-sec2-postbl-plm-ss-cfg.patch line 144 lmrValue = 0x0000028AU;

build.sh never reads constants.yaml. Geometry is hard-coded in the case block of build.sh and again in patch 0001; constants.yaml is documentation, not configuration. This matters when reading any branch of this project: editing constants.yaml alone changes nothing that gets compiled.

Worse, the build-time rewrite that could in principle have substituted a value is short-circuited. The dual-device guard in 80/driver/build.sh tests for the presence of 0x02779000U, 0x0000020BU, 0x0000028AU, 0x0000001000000000ULL and 0x0000001400000000ULL. All five are present in the patched C source, so the guard concludes both geometries are already baked in and the Python rewrite exits before substituting anything.

The consequence: commit 3c53aca "Correct LMR for 80GB" changed only inert metadata. Every tester who ran the 80 branch programmed CFG1 0x02779000, LMR 0x0000028A and fb_length 0x0000001400000000.

Why that combination is incoherent

The Local Memory Range register encodes capacity as a magnitude and a scale:

size_MiB = MAG[9:4] << SCALE[3:0]        equivalently   bytes = MAG << (SCALE + 20)

which decodes every value in real use exactly:

LMR MAG SCALE Size
0x00000208 32 8 8 GiB (8 GB card, stock)
0x00000288 40 8 10 GiB (10 GB card, stock)
0x0000020B 32 11 64 GiB (8 GB card, unlocked)
0x0000028A 40 10 40 GiB
0x0000028B 40 11 80 GiB

So the 80 branch as built tells three different parts of the machine three different things:

Layer What it was told
FBPA CFG1 tier 0x77 4096 MiB per partition, that is, address for 80 GiB
MMU Local Memory Range 0x0000028A the local memory range is 40 GiB
GspStaticConfigInfo.fb_length 0x0000001400000000 the framebuffer is 80 GiB

That mismatch predicts the observed behaviour exactly: nvidia-smi reports 80 GB because it reads the forced fb_length, the hardware decodes 40 GiB because that is what the LMR says, and an alias test folds at precisely 40 GiB. The three-way disagreement is the best explanation for the branch's fold, and the driverless runs below support it: programming the coherent 0x0000028B instead made the 40 GiB fold disappear. It does not explain everything, because the coherent set still loses the device after one CUDA context.

!!! question "Open problem: field width of LOWER_MAG" Everything in real use works under a 6-bit MAG at [9:4], but the width has never been read from dev_fb.h; 7 bits at [10:4] is the alternative. This is a header lookup, not an experiment. It is the last thing standing between the 0x28B-versus-0x50A question and a clean answer. Related: 0x40A and 0x50A are refuted candidate encodings, not observed ones. Under a 6-bit field (0x40A >> 4) & 0x3F = 0 and (0x50A >> 4) & 0x3F = 16, and 0x40A is separately recorded as a failed attempt on a 10 GB card on 2026-07-11.

What was observed on hardware with the 80 branch

What worked

  • The card boots with the 80 GB geometry applied. No rejected boot, no RmInitAdapter failure.
  • nvidia-smi reports roughly 81920 MiB, and cuda_memtest reports a CUDA global memory size of 85,545,582,592 bytes, which is 79.67 GiB rather than a full 80 GiB.
  • cudaMalloc of 77 GiB succeeds. The allocator hands back the memory.
  • Late PMA registration reports 79 GB and fb_length reports 80 GB, both confirmed.
  • The physical DRAM is present. A PRAMIN walk proved 80 distinct GiB out of 80. Whatever the wall is, it is not missing silicon. On this branch the wall behaves like address decode, since the fold sits exactly on the LMR boundary; the driverless coherent runs show a different wall once the decode is coherent.

What failed

Symptom Detail
Fatal GPU loss Kernels touching more than roughly 40 GB lose the device, independent of the power limit. Reported Xid codes include Xid 31 (described as harmless) and Xid 154 after CUDA memory tests; the dominant reported symptom is hangs. Xid 31 alone was suggested by a bystander and was not corroborated as the signature by the operator with the failing card
cuda_memtest Hangs above 39 GB
Model loading Failures above roughly 20 GB in practice for one tester; a second tester with multiple cards reported failures in the 40-60 GB band. Reported the same day as the hang
Per-page eviction test Collides at exactly 40 GiB, with PMA at 79 GB and fb_length at 80 GB both confirmed at the time

Note the gap between the register-level fold boundary (exactly 40 GiB) and the practical workload ceiling, which one tester put at about 20 GB for model loads while a second tester with multiple cards put it in the 40-60 GB band. The corpus does not explain either the gap or the disagreement between the two reporters. Anyone reading "80 GB nearly works" from the allocation ceiling should note that in practice the usable-memory outcome for inference was worse than the 40 GB profile, not better.

What was never observed on the 80 branch

  • No repeatable memory validation pass above 39 GB. cuda_memtest is reported to have completed once immediately after a reboot and to have failed every run after that.
  • No successful workload above roughly 40 GB.
  • No build of the branch carrying the coherent 0x28B LMR. The compiled path has always carried 0x0000028A. The coherent register set was reached on hardware, but by the driverless refire chain rather than by a driver build; see the next section.
  • No repeat of the per-page eviction test after any fix, on this branch.

!!! note "Superseded: the branch README's 'Working ✓'" The 80 branch README asserts 80 GB geometry on 10 GB cards is "Working ✓" with "expect ~81920 MiB". The same week's testing contradicted it. The author's own assessment on 2026-07-19 was far more modest: "I'd say even having 80GB show up in nvidia-smi without rejected boot is a success (for now)." "Working" in that README means "reports the size and boots", not "passes memory validation". Every adjudicated document that mentions this configuration records the README row as a documentation defect. Superseded by the shipping 40 GB profile.

The coherent 0x28B set on hardware, without the driver branch

!!! warning "Experimental, unshipped, and not a recommendation" Everything in this section comes from a clean-room refire chain that programs the registers from a script and then hands the card to a stock driver. It is not in master, not in the 80 branch, and not an install path. Shipping master gives a 10 GB card 40 GB, and 40 GB remains the supported configuration.

Between 2026-07-23 and 2026-07-27 the coherent triple was fired on 10 GB cards and logged:

[memory 80GB -> 81920 MiB] CFG1=0x02779000 LMR=0x028b
POST: CFG1=0x02779000 LMR=0x0000028b decode=0x10000300 CSTATUS=20/24
  => READY for NO-FLR stock-driver load (81920 MiB)

Note decode=0x10000300, the 4 GB-per-channel L2/LTC decode, present alongside the coherent LMR rather than the 0x70000300 seen under the branch.

What passed.

Observation Detail Date
Dense tagged write/readback, no fold 310 blocks x 256 MB = 77.5 GiB, blocks reading a WRONG tag: 0/310, after an FLR, an 80 GB fire, a GSP boot and a CPU-RM handoff 2026-07-23
Same result reviewed in-channel A dense fill writes a unique tag into every page and reads every page back, so an alias cannot slip through the way it can past a sparse probe. 77 GiB of surviving tags implies at least 77 GiB of distinct physical cells, past the 40 GiB fold point. This is an argument about method, not a second run 2026-07-24
Coherent fire reproduced Same register set fired from the v8 script on a second card, 81920 MiB reported, Gen2 link achieved in the same run 2026-07-24
Bandwidth across the boundary 1 GB memset at offsets 0, 1, 2, 4 GB ran at 98 % of a 1448 GiB/s peak; every offset from 8 GB through 76 GB ran at 79 % (about 1149 GiB/s). Nothing collapsed at 40 GB 2026-07-25
Largest no-fold run at stock timings 72 GiB, with the boot timings untouched 2026-07-27

What did not pass, and why this is not a recommendation.

  • One CUDA context per fire. Both operators reported Xid 154 after CUDA memory tests, leaving roughly one usable context per fire; one of them needed a cold power cycle rather than a driver reload before refiring.
  • The extended region is slower. 79 % of peak above the roughly 8 GB boundary, not parity.
  • The top of the range is untested. The dense test covered 77 of the 79.29 GiB reported, leaving about 2 GiB unverified, and the reported total is 79.3 GiB rather than a full 80.
  • A refresh fix did not rescue it. Writing FBPA_CONFIG4 to 0xc403001a (tREFI halved, landing on all 20 live FBPAs) flattened the bandwidth curve but dropped it to 54-61 % of peak and did not clear the instability for the operator who tried it.
  • Small sample. Two operators, a handful of days, no burn-in and no third rig.
  • The operator's own first reaction was doubt. Hours after the 2026-07-23 dense pass the same person wrote that it was "probably bogus", because the 4 GB-per-channel sizing appeared to activate only once a CUDA kernel was loaded and only for a single kernel. The method argument the next day answered the fold question; it did not answer that one.

!!! note "Was a hardware strap mod required?" One of the two cards had its strap resistors moved before these runs, which initially looked load-bearing. It was not: a probe showed CFG1 unchanged at 0x02449000 after the mod, and the second operator reported the refire script reaching 80 GB on an unmodded 10 GB card, with the same Xid 154 limit. Straps may still select memory timings; they do not select capacity.

Why reported size proves nothing

The rule adopted after this episode, and the reason the 80 GB claim collapsed:

!!! danger "CSTATUS reports measured hardware capacity, not allocatable capacity" Per-FBPA CSTATUS_RAMAMOUNT can read 40 or 80 GB without the upper region being usable at all. Every capacity claim must be backed by a write/read-back alias ("fold") test rather than a reported size. The rule was adopted after a reported 79.4 GiB folded above 40 GiB and a separately reported 4 GiB turned out to be a tooling bug.

For reference, the CSTATUS tiers on this hardware:

State Per-FBPA CSTATUS_RAMAMOUNT Total
Stock 170HX 0x200 (512 MiB) x 20 FBPAs 10 GB
40 GB tier 0x800 40 GB
64 GB tier 0x00001000 64 GB
A100 PCIe 40GB (stock) 0x000007ff x 20 40 GB
A100 PCIe 80GB (stock) 0x00000fff x 20 80 GB

The unlocked 170HX values are consistently one greater than the stock A100 equivalents (0x1000 versus 0x0fff, 0x800 versus 0x7ff). The register is annotated "clean, written by no patch": it is derived hardware state following CFG1, not something the unlock writes.

One useful precedent from stock NVIDIA silicon: the A100 PCIe 40GB and 80GB have byte-identical floorsweep fuses (FUSE_FBPA_DISABLE 0x00820368 = 0x00c03000, FUSE_FBP_DISABLE 0x00820364 = 0x00000840), both with 20 active FBPAs, and the entire 2x capacity difference is carried by CSTATUS_RAMAMOUNT and FBPA_CFG1_BROADCAST (0x02669000 versus 0x02779000). So the CFG1 half of the 80 branch is exactly what NVIDIA itself uses to make an 80 GB part. The LMR half is where the branch went wrong.

What else was tried against the 40 GiB wall

Approach Result
LTC companion write 0x1402b4 = 0x00a00030, post-boot The one remaining LTC register difference versus a real A100-80 was made to match. The fold still landed at exactly 40 GiB. Not fully closed: three untested hypotheses remained (post-boot LTC writes do not re-latch and 0xb4 must be written during boot; an unreleased update handles it; or the eviction test is a false fold caused by a PMA or page-table artefact)
Flip the L2/LTC address decode from 0x70000300 (2 GB/channel) to 0x10000300 (4 GB/channel) Mechanically achievable: a workload using atomic writes can make the decode register flip (described as fiddly), and a one-byte driver mod can hold it there without the workload. Flipping it under the branch's incoherent LMR did not help: "when it flips to 0x10000300 it doesn't actually fix the >40GB issue ... It's downstream unfortunately." That drove a pivot toward strap resistors and JTAG. It reads differently now: the later driverless runs carry decode=0x10000300 together with the coherent LMR, and those runs are the ones with no fold at 77.5 GiB and full-offset bandwidth. The decode is necessary, not sufficient
The L2 amap 0x17e22c as the >10 GB gate The team's working root-cause model for over a week. Disproved the same day it was written down: a run reached real 40 GB with 0x17e22c at its native 0x00280404 the whole time, never programmed. The shipping driver's total silence on 0x17Exxxx (no such address exists in master or in any branch snapshot) is the standing confirmation
Re-enable floorswept FBPs On the tested 10 GB card the delta is empty: FBP_DEFECTIVE (0x8205CC) = FBP_DISABLE (0x820364) = 0x840, and STATUS_HALF_FBPA = 0 means no half-capacity fuses to recover. That card is genuinely dead on those partitions, which also makes 80 GB its theoretical ceiling rather than 96 GB. The route is not closed in general (one community dump shows FBP_DISABLE = 0x852 against FBP_DEFECTIVE = 0x840), but the intended 8 GB-card test was never reported
MRS replay to reach a different memory strap without reflashing Mechanically works: baseline MR1 0x00100093, MR2 0x002000cf, MR3 0x003000ea; derived strap 7 MR1 0x0010009b, MR2 0x00200029, MR3 0x003000ef; replay back to strap 4 reproduced the exact boot values. The driver then refuses with RmInitAdapter failed! (0x62:0x40:2674). This is the most recent state of the >64 GB effort in the corpus (2026-07-27)
Physically moving the strap resistors On 2026-07-25 a tester moved a 10 GB card's straps to the 8 GB pattern; probe.sh showed CFG1 unchanged at 0x02449000. An unmodded card reached the same capacity by software alone. Straps may still matter for memory timing selection, but not for capacity

The paper's memory-error data

The Zenodo paper framed the CMP 170HX as an A100 die fuse-crippled on three axes: SM math rate throttled to 1/32, memory capacity 10 GB instead of 80 GB, and PCIe Gen1 instead of Gen4. The 1/32 SM rate and the Gen1-versus-Gen4 framing held up. The 10 GB to 80 GB memory figure did not: the verified geometry is 10 GB to 40 GB and 8 GB to 64 GB, and the figure was already being questioned in-channel on 2026-07-06.

The paper's stability data is the closest thing to an independent 80 GB stability measurement, and it is worth recording precisely because it points at a different failure mode than the address-decode fold:

Configuration Memory-stress errors Confidence
Full-speed memory, 80 GB configuration 2,796 errors medium
Same card at half capacity 0 errors medium
80 GB configuration with the refresh-interval field lowered (that is, refresh raised) 0 errors medium
Throughput cost of the refresh fix roughly 32 %, 64.6 versus 94.6 TFLOPS medium

!!! note "Superseded: the 60-70 % figure was never about compute" The original in-channel reading was that full-speed memory at 40 GB on a 10 GB card allowed only 60-70 % compute speed to stay stable, blamed on missing VRM phases sagging a rail. Within an hour on 2026-07-07 that was corrected: the card in the paper had 80 GB of poor-quality VRAM, and getting all of it working reliably required raising the DRAM refresh rate, which lowered available bandwidth to 60 %. Nothing to do with compute, nothing to do with power phases.

!!! question "Open problem: bandwidth or latency?" A third position was offered and never settled: that bandwidth, memory speed and bus width are all unchanged and it is latency that spikes inconsistently, with random latencies far higher, pushing 1 % lows out to 5 % lows. Two different mechanisms, the same headline number, and nobody ran the measurement that separates them. What would settle it: a streaming bandwidth benchmark and a pointer-chase latency benchmark run back to back at both refresh settings on the same card.

The refresh hypothesis, and the test that failed to confirm it

The proposed mechanism for the paper's error count and for the Xid 154 seen on the coherent set is DRAM retention, not decode: FBPA_CONFIG4 (0x9a02a0 broadcast, 0x9002a0 + n*0x4000 per-FBPA) carries the tREFI interval in bits [14:0], and it reads a rock-stable 0xc4030033 at both 40 GB and 80 GB, identical across the 10 GB CMP, the 8 GB CMP and the A100. Refresh does not scale with capacity, so a fire that doubles reachable rows (CSTATUS 0x800 to 0x1000) leaves roughly half the rows refreshed at the 2 GB-per-channel rate.

!!! warning "Attractive, but the direct test did not confirm it" The fix was landed: CONFIG4 = 0xc403001a (refresh 26, doubled) written through the HS ROP path, verified on all 20 live FBPAs. It did not clear the instability for the operator who tried it, and it cost bandwidth, dropping the offset sweep from 79-98 % of peak to 54-61 %. Treat refresh as the leading hypothesis for the retention story, not as an established root cause. The counter-argument is on memory subsystem: a retention failure should scatter errors by time and address, while the branch's fold sat on an exact power-of-two boundary.

Why 40 GB ships

Three reasons, in order of weight.

  1. 40 GB passes and 80 GB does not. The 10 GB to 40 GB unlock was independently confirmed working by two testers on 2026-07-19, one of them on two cards simultaneously, and has been in production use since. The 80 GB configuration hangs cuda_memtest above 39 GB and loses the GPU above roughly 40 GB.
  2. 80 GB yields less usable memory, not more. For inference workloads, a configuration that reports 80 GB and then faults somewhere between roughly 20 GB and 60 GB depending on the tester is strictly worse than one that reports 40 GB and delivers it. The driverless coherent set does reach real memory past 40 GiB, but at one CUDA context per fire it is not a serving configuration either.
  3. Structural corroboration that the 170HX is wired like an A100. MMU_NUM_ACTIVE_LTCS (0x00100EC0) reads 0x05001414 on the 170HX and FBHUB_NUM_ACTIVE_LTCS (0x00100800) reads 0x00000014, both giving 20 active LTCs. Both values are byte-identical on all three A100 SKUs, including the 80 GB part, so they place the 170HX in the A100 family but do not by themselves pick 40 GB over 80 GB. What picks 40 GB is that 40 GB is the geometry that passes.

The 80 branch survives as the record of the attempt, with unlocked_mib: 81920 still sitting in its constants.yaml.

The experiment that would settle it

!!! question "Open problem: does fixing the LMR in the branch fix the fold?" The 80 branch ships LMR 0x0000028A in the compiled path, which decodes to 40 GiB, and the fold boundary matches the LMR exactly. The driverless refire chain already answers the physics: with the coherent 0x0000028B the fold does not appear up to 77.5 GiB. What is still open is whether a driver build carrying 0x0000028B reproduces that, which is what would matter for anything installable.

**The change.** Two lines, in the two places that are actually read:

```text
driver/patches/0001-sec2-postbl-plm-ss-cfg.patch :  lmrValue = 0x0000028BU;
driver/build.sh                                  :  LMR="0x0000028B"
```

Editing `common/constants.yaml` is **not** sufficient and is what the original commit did.
Note also that adding `0x0000028BU` to the source may satisfy or disturb the dual-device
guard in `build.sh`, which currently tests for `0x02779000U`, `0x0000020BU`, `0x0000028AU`,
`0x0000001000000000ULL` and `0x0000001400000000ULL`; check that the Python rewrite still
takes the branch you expect.

**The test.** Repeat the per-page eviction test and check whether the fold moves off 40 GiB.

**Why it matters.** This one-line change has apparently never been compiled, even though the
register set it produces has been fired from a script. It is the difference between "the
coherent geometry only works from a bespoke fire chain" and "the driver can carry it", and it
is one constant change and one reboot.

The remaining unknowns, now that the fold question is largely answered:

  • What actually causes Xid 154 and the one-context-per-fire limit. This is the live blocker, not the fold. See the refresh discussion below.
  • Whether the per-page eviction test on the branch was itself a false fold caused by a PMA or page-table artefact. Listed as one of three hypotheses in 2026-07-19 and never checked directly, though the coherent-set dense runs make the fold look real and LMR-driven.
  • Whether the top roughly 2 GiB of the range is usable at all: the dense test stopped at 77 GiB of a reported 79.29 GiB.

Related shipping-driver behaviour worth knowing

Three things the shipping driver does that interact with any large-geometry experiment. All are gated on device ID 0x20C2 or 0x2082.

  • It overwrites GspStaticConfigInfo.fb_length with the profile fb_bytes and extends the last FB region's limit to targetFbBytes - 1 (patch 0001). This is the mechanism by which the 80 branch makes nvidia-smi report 80 GB.
  • It registers the region above 8 GiB with PMA late, at osinit.c time, via memmgrSec2DebugLateExtendHighPmaRegion, splitting the high reserved FB region at the 8 GiB boundary if necessary (patch 0003).
  • It clamps the BAR0 PRAMIN window back to the stock 8 GiB position whenever Ram.fbAddrSpaceSizeMb > 0x2000 (patch 0004).

!!! warning "The 'stock' size constant is hard-coded to 8 GiB for both card types" stockFbBytes = 0x200000000ULL /* 8GB */ appears in both patch 0001 and patch 0003, and the PRAMIN clamp compares against 0x2000 MB. On a 10 GB card the true stock size is 10240 MiB, so the late-PMA candidate search (limit >= stockFbBytes) and the region split both use an 8 GiB boundary rather than a 10 GiB one. This is shipping behaviour on hardware that demonstrably works, so it is apparently benign, but it is a real asymmetry, it is undocumented in the repository, and anyone debugging a large-geometry PMA problem should know about it. Checking it is a log-reading exercise: read the SEC2_DEBUG_LATE_PMA: region[...] and SEC2_DEBUG_HEAP: dmesg lines on a 40 GB-unlocked 10 GB card and see whether publicBytes accounts for the full 40 GiB.

Also relevant: BAR1 stays at 64 MiB even when the card advertises 81920 MiB of framebuffer, so large-BAR or full-VRAM host mapping is not available regardless of geometry. The Resizable BAR capability at [bb0 v1] advertises exactly one supported size per BAR (BAR0 16 MB, BAR1 64 MB, BAR3 32 MB).

Finally, the geometry does not survive a reset. CFG1, per-FBPA CFG1, CSTATUS, LMR, the FB-geometry PLMs and the AON LMR shadow 0x001180f0 are all cleared by FLR, while SS0, SS1 and FEAT_OVR_PLM 0x00823804 survive. That asymmetry is why the compute unlock shipped before the memory unlock, and it applies to the 80 GB configuration too.

Measured values

Quantity Value Conditions Confidence
80 branch as-built geometry CFG1 0x02779000, LMR 0x0000028A, fb_length 0x0000001400000000 code-verified across build.sh, install.sh and patch 0001 high
80 branch metadata geometry LMR 0x0000028B, unlocked_mib: 81920 common/constants.yaml, never read by the build high
Patch-file delta versus master one file, two lines (cfg1Value, targetFbBytes); 0002-0006 identical byte comparison high
Profile auto-detect addition >= 75000 MiB -> 10gb 80 branch install.sh high
Reported capacity roughly 81920 MiB (nvidia-smi); 85,545,582,592 bytes = 79.67 GiB (CUDA, via cuda_memtest) two different reporters, two different numbers high
cudaMalloc ceiling 77 GiB succeeds allocation only high
Fatal-loss threshold kernels touching more than roughly 40 GB, power-limit independent on hardware high
Xid code of the fatal loss not established. Xid 31 (called harmless) and Xid 154 after CUDA memory tests are both reported; the dominant reported symptom is hangs Xid 31 suggested by a bystander, not corroborated by the operator with the failing card low
cuda_memtest hangs above 39 GB 2026-07-19 high
Model loading failures above roughly 20 GB for one tester; 40-60 GB band for a second tester with multiple cards same day, 80 branch medium
Per-page eviction fold point exactly 40 GiB, with PMA 79 GB and fb_length 80 GB 610 build, 80 branch (incoherent LMR 0x28A) medium
Dense tagged write/readback, coherent LMR 0x28B 77.5 GiB, 310 of 310 blocks correct, no fold driverless refire chain, 2026-07-23; largest stock-timing no-fold run 72 GiB on 2026-07-27 medium
Bandwidth above the boundary, coherent LMR 0x28B 98 % of peak below roughly 8 GB, 79 % at every offset from 8 GB to 76 GB 1 GB memset sweep, one card, 2026-07-25 medium
Contexts per fire, coherent LMR 0x28B roughly one, then Xid 154 both operators, 2026-07-25 medium
PRAMIN distinct-GiB walk 80 of 80 physical DRAM is present; on the 80 branch the wall behaves like address decode high
LTC companion write 0x1402b4 = 0x00a00030 post-boot did not move the fold medium
L2 decode value 0x70000300 (2 GB/channel); target 0x10000300 (4 GB/channel) flipping it under the branch's incoherent LMR did not fix >40 GB; the later coherent runs sit at 0x10000300 with 77 GiB real and full-offset bandwidth, so it is necessary but not sufficient medium
L2/LTC amap 0x0017e22c native 0x00280404, never programmed, yet real 40 GB was reached disproves the amap model high
FBP_DEFECTIVE / FBP_DISABLE, tested 10 GB card 0x840 / 0x840 empty delta; STATUS_HALF_FBPA = 0, so 80 GB is that card's theoretical ceiling high
FBP_DISABLE, one community dump 0x852 against FBP_DEFECTIVE 0x840 the one recorded non-empty delta medium
CSTATUS tiers 0x200 stock / 0x800 at 40 GB / 0x1000 at 64 GB per active FBPA high
MRS replay result mechanically works, then RmInitAdapter failed! (0x62:0x40:2674) 2026-07-27, unresolved high
Memory-stress errors, 80 GB configuration 2,796 paper medium
Memory-stress errors, half capacity 0 same card, paper medium
Memory-stress errors, raised refresh 0 80 GB configuration, paper medium
Throughput cost of raised refresh roughly 32 %, 64.6 versus 94.6 TFLOPS paper medium
BAR1 size at any geometry 64 MiB unchanged even at an advertised 81920 MiB high

See also

Home

Start

Hardware

Unlock

Procedures

Operations

Frontier

History

Appendix

Clone this wiki locally