Memory allocation error in qemu arm chroot.

[petronny]

Hi,
I created an armv6h enviroment with qemu_arm_static on an x86_64 machine.
Inside this chroot, zstd get memory allocation error with --ultra -20. (--ultra -19 works.)
However, I have 186Gi RAM available.
Could anyone help to take a look?

BTW, how much RAM do --ultra -20 and --ultra -19 actually need?
Also qemu_arm_static has an option called QEMU_STACK_SIZE and now it's 8388608 byte (8MB). Is this relevant?

Log:

[alarm@alarm unzip]$ echo a > a.txt
[alarm@alarm unzip]$ echo a.txt | zstd -v -c -T0 --ultra -20 - > PKGBUILD.zst
*** zstd command line interface 32-bits v1.4.4, by Yann Collet ***
Note: 16 physical core(s) detected 
zstd: error 11 : Allocation error : not enough memory 
[alarm@alarm unzip]$ echo a.txt | zstd -v -c -T0 --ultra -19 - > PKGBUILD.zst
*** zstd command line interface 32-bits v1.4.4, by Yann Collet ***
Note: 16 physical core(s) detected 
/*stdin*\            :316.67%   (     6 =>     19 bytes, /*stdout*\)           
[alarm@alarm unzip]$ free -h
              total        used        free      shared  buff/cache   available
Mem:          188Gi       902Mi       183Gi       1.0Mi       4.5Gi       186Gi
Swap:            0B          0B          0B

---

PS. The chroot is created by devtools-qemu on Archlinux.
Anyone using Archlinux can also get a similiar chroot with this script and use arch-chroot to chroot into it.

[Cyan4973]

Hi @petronny ,

the amount of memory required is more or less multiplied by the number of cores to run in parallel (not exactly, but good enough). The amount of memory required for a single -20 thread can be pretty large, I measured up to 400 MB.

In your test, you use -T0, which means "use as many threads as cores", and the status message mentions detecting 16 cores. 16 * 0.4, that's still well below 186 GB.

In this test case though, one can see that the CLI is compiled in 32-bit mode.
Therefore, even if there is 186 GB of RAM on the x64 host, this is irrelevant for the 32-bit qemu arm6 environment : address space is necessarily limited to 4 GB (and effectively less than that due to additional layout constraints).

The input size is very small (6 bytes), so the compressor will effectively use only a tiny fraction of that amount. But the full memory will nonetheless be "pre-allocated" by the guest system. On a 64-bit environment, even with much less RAM, this would work : unused memory will remain untouched, hence not really allocated. But on 32-bit, it hits the maximum capacity of the address space, and it can't recover from that : it generates an OOM error.

The archlinux thread recommends reducing the nb of threads, rather than the compression level. But in my opinion, both methods are valid. Select whichever method suits your needs best.

[petronny]

Hi @Cyan4973,

Thank you for your explanation. That's very nice and clear.

In my case, the limit of the number of cores to run in parallel is 12 in the armv6h chroot.
And in an aarch64 chroot, the memory allocation error won't show up with --ultra -20.
So the answer should just be the limitation of the address space of 32-bit program.

I'll adjust the number of threads in my configurations for armv{6,7}h locally.