Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Out of memory error when running on in-ram file system / GCF #225
I'm running into the fabled error:
when running crystal language binaries from within a Google Cloud Function, and when there is tons of free ram (generally 1.7 gigs). GCF uses a 2 GB in-ram file system, meaning if I create a 20 MB file, I am really using up 20 MB of ram, etc. Running
For some reason no matter what I do, boehm-gc is throwing this error via crystal. I am wondering if maybe this is a bug? If you give me a portable linux binary I can run it in GCF to try to debug this more.
If you are unfamiliar with crystal, they bind with boehm-gc here: https://github.com/crystal-lang/crystal/blob/master/src/gc/boehm.cr
Here is the same issue posted on the crystal repo:
referenced this issue
Jun 13, 2018
Thanks for reporting. Based on strace, I see 2 things that together lead to the error.
This means brk has failed to move the program data boundary for some reason, it might be OK (I don't know who calls mremap and probably there's some relation between mremap EFAULT and brk fail). To handle brk failures, BDWGC switches to using mmap after the first one.
3 stands for fd of /dev/zero.
So, to workaround this issue on GCF the BDWGC library should be compiled with "-D USE_MMAP_ANON" passed to CFLAGS. Could you please try this? (I know that you don't need it anymore but it would be nice to check at least.)
Alternatively I'm thinking on the run-time workaround - to switch to anonymous mmap if the first mmap(/dev/zero) failed with ENODEV.
Run-time detection is not really needed, it is OK to default to MAP_ANONYMOUS/MAP_ANON when available (at compile time).
Hmm. In case of linux platform, USE_MMAP_ANON is defined by default (if mmap is used).