Alpine has dropped jemalloc due to no support.

[shleeable]

There are related issues with alpine and libjemalloc running ruby.

I'd like to start using jemalloc on my ruby docker containers but it seems to be missing some engagement with the community over there.

[davidtgoldblatt]

I don't think there's much we can do here; the underlying issue is an incompatibility between jemalloc and the musl libc (that Alpine uses); musl would have to make a design decision to support replacing the allocator on its end. That would require extra work on musl's end, and it's not clear to me that the cost/benefit ratio is there for them; part of its pitch is its simplicity and lightweight design.

If you persuade them that they should support malloc replacement as a feature, I'm willing to help with bootstrapping issues on our end; but making this work isn't really just a question of writing code.

[shleeable]

Thanks for the additional detail.

[davidtgoldblatt]

No problem! Closing this for now, but happy to take a new issue if something changes that makes this feasible.

[Kaisrlik]

FYI: Recently, I wrote on musl mainling list and they confirm that musl supports usage of external malloc function.

[davidtgoldblatt]

OK, I'll add this to the queue. This is a very busy time for us, but I'll come back to it when I get a chance.

[thedanbob]

For what it's worth, I was able to build jemalloc 5.2.1 on alpine 3.11 with no issues and ruby seems to use it just fine with LD_PRELOAD. I haven't tested it extensively yet, but I did run a memory stress test script I found here.

Sample Dockerfile:

FROM ruby:2.7-alpine AS builder

RUN apk add build-base

RUN wget -O - https://github.com/jemalloc/jemalloc/releases/download/5.2.1/jemalloc-5.2.1.tar.bz2 | tar -xj && \
    cd jemalloc-5.2.1 && \
    ./configure && \
    make && \
    make install

FROM ruby:2.7-alpine

COPY --from=builder /usr/local/lib/libjemalloc.so.2 /usr/local/lib/
ENV LD_PRELOAD=/usr/local/lib/libjemalloc.so.2

[killua99]

The only thing, and that's because I'm not an expert I can't see ruby using jemalloc with @thedanbob docker build running ruby -r rbconfig -e "puts RbConfig::CONFIG['MAINLIBS']"

Perhaps I'm missing something else to make it work.

[thedanbob]

Apparently, ruby can't see jemalloc (even though it's working) when you use LD_PRELOAD. According to this comment you can check it this way:

MALLOC_CONF=stats_print:true ruby -e "exit"

[booleanbetrayal]

Apparently, ruby can't see jemalloc (even though it's working) when you use LD_PRELOAD. According to this comment you can check it this way:

MALLOC_CONF=stats_print:true ruby -e "exit"

Can confirm and think that this issue could probably be closed (?)

[davidtgoldblatt]

I believe that musl now supports malloc replacement, and my understanding is that jemalloc + musl now "just works". I was keeping this open until I had verified it myself, but I've seen enough people report it working that I figure it must be at least mostly OK.

Closing, but still happy to field reports of breakages if they arise.

[DirectorX]

@thedanbob

Apparently, ruby can't see jemalloc (even though it's working) when you use LD_PRELOAD. According to this comment you can check it this way:

MALLOC_CONF=stats_print:true ruby -e "exit"

try to execute ldconfig to refresh the cache

[neohunter]

On alpine 3.10 using docker image ruby:2.5.5-alpine i'm able to compile jemalloc, however when running

# set LD_PRELOAD=/usr/local/lib/libjemalloc.so.2
# ldconfig
# ruby -r rbconfig -e "puts RbConfig::CONFIG['MAINLIBS']"
# ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']"
-lpthread -lgmp -ldl -lcrypt -lm 
# MALLOC_CONF=stats_print:true ruby -e "exit"

Any command the output is always empty (unless CONFIG['LIBS'].

I guess jemalloc is not loaded :(

[thedanbob]

I tried the same and can confirm that trying to load jemalloc in ruby:2.5.5-alpine segfaults (even just LD_PRELOAD=/usr/local/lib/libjemalloc.so.2 /bin/sh). One workaround could be to build a ruby 2.5.5 image from alpine 3.11.

[neohunter]

What I did for now is use this docker image instead:

FROM swipesense/ruby-jemalloc:2.5-stretch-slim

It has already ruby 2.5.7 compiled with jemalloc.

It works like a charm, what I don't like however is taht is a much heavier image. But I just wanted to confirm if jamalloc offer me some performance improvement so it will work for now.

[srini38]

Was trying to enable leak profiling using jemalloc on Alpine Linux 3.11.11, but it just hangs. Without "--enable-debug --enable-prof" configure options to jemalloc, the command completes. Is this expected? Even tried Alpine 3.13.5 without any success. Any helpful pointers much appreciated.

/source/jemalloc-5.2.1 # cat /etc/alpine-release
3.11.11
/home/source/jemalloc-5.2.1 # ./configure --enable-debug --enable-prof && make -j10 && make install
/source/jemalloc-5.2.1 # MALLOC_CONF=prof_leak:true,lg_prof_sample:0,prof_final:true LD_PRELOAD=/usr/local/lib/libjemalloc.so.2 /bin/sh
^C

tried with ./configure --enable-debug --enable-prof --disable-prof-libgcc and the program segfaults. Issue seen in Alpine 3.13.5 as well

Reading symbols from ./sample...
(gdb) attach 8294
Attaching to program: /home/source/sample, process 8294
Reading symbols from /usr/local/lib/libjemalloc.so.2...
Reading symbols from /lib/ld-musl-x86_64.so.1...
Reading symbols from /usr/lib/debug//lib/ld-musl-x86_64.so.1.debug...
Reading symbols from /usr/lib/libstdc++.so.6...
(No debugging symbols found in /usr/lib/libstdc++.so.6)
Reading symbols from /usr/lib/libgcc_s.so.1...
(No debugging symbols found in /usr/lib/libgcc_s.so.1)
0x00007f5dde4f8ed6 in __syscall3 (a3=1024, a2=140041138758952, a1=0, n=0) at ./arch/x86_64/syscall_arch.h:29
29      ./arch/x86_64/syscall_arch.h: No such file or directory.
(gdb) c
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0x00007f5dde22beee in je_prof_backtrace (bt=0x7fffb37ab790) at src/prof.c:665
665             BT_FRAME(7)
(gdb) bt
#0  0x00007f5dde22beee in je_prof_backtrace (bt=0x7fffb37ab790) at src/prof.c:665
#1  0x00007f5dde1b3438 in prof_alloc_prep (tsd=0x5645b6c8f068, usize=16384, je_prof_active=true, update=true)
    at include/jemalloc/internal/prof_inlines_b.h:158
#2  0x00007f5dde1ba1af in imalloc_body (sopts=0x7fffb37ab880, dopts=0x7fffb37ab8b0, tsd=0x5645b6c8f068) at src/jemalloc.c:2116
#3  0x00007f5dde1ba6b4 in imalloc (sopts=0x7fffb37ab880, dopts=0x7fffb37ab8b0) at src/jemalloc.c:2265
#4  0x00007f5dde1ba7fa in je_malloc_default (size=16000) at src/jemalloc.c:2289
#5  0x00007f5dde1baa77 in malloc (size=16000) at src/jemalloc.c:2332
#6  0x00005645b4cc9231 in main ()
(gdb)

// Sample program from the WWW
#include <stdio.h>
#include <stdlib.h>

int main()
{
    int n, i, *ptr, sum = 0;

    printf("Enter number of elements: ");
    scanf("%d", &n);

    ptr = (int*) malloc(n * sizeof(int));

    // if memory cannot be allocated
    if(ptr == NULL)
    {
        printf("Error! memory not allocated.");
        exit(0);
    }

    return 0;
}

[BillyYarosh-WellSky]

Wanted to follow up and ask if anyone has gotten jemalloc running on a baseimage without jemalloc precompiled into ruby? Based on this conversation, it doesn't sound like we have. I have tried thedanbob's suggestion and that doesn't work. I've also seen that solution provided elsewhere, and seems ruby doesn't recognize or acknowledge the LD_PRELOAD variable.

[chubbson]

I think there is still no support of Jemalloc and Alpine.
I also tried thebandbob's suggestion this will result in segmentation fault building the container. Also tried 5.3.0 of Jemalloc, with same error. I did not follow that rabbit hole in deep.

[Aubermean]

There is support for Jemalloc in Alpine, at least on 3.18. Everything I read said it would be hard or near impossible but in reality it was as easy as this:

apt add --no-cache jemalloc
LD_PRELOAD=libjemalloc.so.2
MALLOC_CONF=stats_print:true ruby -e "exit"

Version 5 no less!

Version: "5.3.0"

[thedanbob]

Alpine restored support for jemalloc in 2022: https://gitlab.alpinelinux.org/alpine/aports/-/commit/e764db77406d02df1e6ee627daa808837c1a81ce

Edit: there's some useful info in this gist: https://gist.github.com/jjb/9ff0d3f622c8bbe904fe7a82e35152fc
I can confirm that the method of patching ruby with jemalloc support, mentioned in the last comment there, works on Alpine:

apk add ruby jemalloc patchelf
patchelf --add-needed libjemalloc.so.2 /usr/bin/ruby
apk del patchelf
MALLOC_CONF=stats_print:true ruby -e "exit"