Building script in debug mode fails on 32 Bit GNU/Linux

[antrik]

The 3 GiB virtual address space available on 32 Bit systems is no longer sufficient for a debug build of Servo on x86:

rustc: /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/llvm/lib/Support/SmallVector.cpp:36: void llvm::SmallVectorBase::grow_pod(void*, size_t, size_t): Assertion `NewElts && "Out of memory"' failed.
error: Could not compile `script`.

In release mode, virtual memory usage is around 2600 MiB, so that's still fine for now...

[Coder206]

How much storage do you have on the VM?

[antrik]

It's not a VM, and there is plenty of swap space and disk space available. The machine has 8 GiB of RAM. But none of this really matters, as this is about address space exhaustion...

[Coder206]

@antrik Sorry about that, the error above is similar to one that I got when I ran out of disk space.

[antrik]

@Coder206 I see. I checked the memory usage though, and it is indeed approaching 3 GiB before it fails.

[zeusintuivo]

I made some research on the best approach to install servo form scratch. You might want to take a look at this thread #12356

[metux]

I've got the same problem. Machine as 8GB RAM, 64bit kernel, 32bit userland (devuan ascii).
Build allocates 3.8G (actually resient 2.7G) before crashing.

Is there anything I can do to reduce (per process) memory consumption ?
Why does it take so much at all ?

[antrik]

@metux last time I tried, disabling debug symbols was the only thing that seemed to help. However, that was quite some time ago. Pretty significant changes to the compiler have taken place since then -- so there might be other options now...

AIUI, the reason it takes so much memory is that the Rust front-end passes a humongous amount of intermediate code down to the LLVM code generator all at once, which results in LLVM building up huge temporary data structures while translating it to machine code.

[metux]

@metux last time I tried, disabling debug symbols was the only thing that seemed to help.

Already trying it (as somewhere this already had been suggested).
Now vsize keeps at 4070M (still running, not crashed yet). Two threads at 100%, three others idle.

AIUI, the reason it takes so much memory is that the Rust front-end passes a humongous amount of
intermediate code down to the LLVM code generator all at once, which results in LLVM building up
huge temporary data structures while translating it to machine code.

So, is it actually a LLVM problem ?

BTW: don't see any llvm process - it it running in the rustc process ?
Perhaps splitting that into separate processes could help ?

[metux]

Just crashed again, even non-dev doesnt work :(

[antrik]

@metux yes, rustc links libllvm.

Not sure it's fair to call it an LLVM problem... I think the Rust front-end could choose to pass down to work in smaller chunks? Don't know the details though. If you really want to get into it, the #rust-internals IRC channel is probably the place to go.

See also #1799 for the possibility of addressing (working around?) the issue in this specific instance on the Servo side.