error updating replicas table: memory allocation failure

[tasleson]

I created a bcachefs with replicas=3 across 24 block devices totaling 27T of space. I mounted and ran a simple single threaded utility which writes out files of random sizes (512B - 8MB) with random data. After writing 4384 files with a total of 18267237367 bytes we hit the following:

[14256.715385] percpu: allocation failed, size=9352 align=8 atomic=1, atomic alloc failed, no space left
[14256.715405] bcachefs (1f577465-f9ab-41c0-b16e-6bb493507155): error updating replicas table: memory allocation failure
[14256.715428] bcachefs (1f577465-f9ab-41c0-b16e-6bb493507155 inum 1577058436 offset 262144): write error while doing btree update: ENOMEM_replicas_table

I tried running it again and we created 222 files with a total of 914278306 bytes before hitting an error again

[16099.074524] percpu: allocation failed, size=10736 align=8 atomic=1, atomic alloc failed, no space left
[16099.074542] bcachefs (1f577465-f9ab-41c0-b16e-6bb493507155): error updating replicas table: memory allocation failure
[16099.074586] bcachefs (1f577465-f9ab-41c0-b16e-6bb493507155 inum 67109095 offset 2318336): write error while doing btree update: ENOMEM_replicas_table

Fedora 39, 6.8.9-200 kernel

The user space error is [Errno 5] Input/output error

[koverstreet]

so - we'll have to either fix the percpu allocator to handle larger allocations, or change the way we do accounting to not use one big percpu allocation.

the former would be preferable, since option 2 will add a pointer deref to our accounting fastpaths