Convert docker builds to use debian / glibc images, away from docker alpine / musl.

[dessalines]

Question

Some people are noting some performance and crash-related issues, that they say has been fixed by moving to glibc-based images.

I was against it before, since musl is supposed to be slimmer and faster, but it might have worse stability. Everything I read online is contradictory, so maybe we should go with how lemmy has been functioning in practice. What are yall's thoughts?

cc @Nutomic @phiresky @asonix @SleeplessOne1917

[SleeplessOne1917]

Just as theory is proven correct through practice, I think going with what has worked best in production is the best approach.

[Security-Chief-Odo]

I believe that glibc is more stable and performant than musl. I don't have a lot of metrics to back that up with regards to Lemmy though; I only started keep track after developing my own* glibc based container for Lemmy. However, I have noted before that using a glibc based container has anecdotally alleviated some ongoing issues; related to OOM and DNS.

Building against musl for the libc means each binary needs to include it’s own libc statically.

Reference:
Rust running slower in musl container
Alpine grief - especially DNS and build times
Known performance issues with Rust
Alpine and Runtime bugs

---

• That PR is for the Lemmy-ui, and I have since modified the Dockerfile. The Lemmy Dockerfile I created does some of the same things though for glibc base instead.

[phiresky]

I don't have a strong opinion on this. I've had problems a few years back with python in Alpine and have heard multiple times of the well-known DNS issues of musl, but that was apparently fixed recently.

Assuming the only pro of using musl/alpine is the smaller image size, it probably makes sense to go for glibc (which base image would that be though?). The cost of a larger image seems small and the potential benefit of eliminating weirdness sounds worth it.

[asonix]

The reason I personally build against musl is for ease of deployment. pict-rs has no dynamically linked dependencies and so can run on any Linux with the same binary

If this doesn't matter as much for you there's little reason not to use glibc

[SleeplessOne1917]

Rust has its own docker image. There seem to be slim variants as well. Maybe that would be good for running lemmy.

[Security-Chief-Odo]

Rust has its own docker image. There seem to be slim variants as well. Maybe that would be good for running lemmy.

Just noting those are based on glibc as well. In fact I do use the rust:slim-bookworm for Lemmy Dockerfile at this time.

[Nutomic]

Iirc the reason we decided to use Alpine images was to reduce the image size. Alpine is about 3 MB. For comparison, Debian-slim images are around 28 MB. Current Lemmy images are around 30 MB. So switching from Alpine to Debian-slim would increase image size roughly to 60 MB. I dont think this increase would bother anyone in practice.

So to me it makes sense to switch to glibc images. They are much more widely used and have less papercuts, plus as Odo mentioned glibc is supposed to be more performant. Additionally we could simplify the build process by using official Rust images, instead of the unofficial musl-rust images.

[marklr]

I've mentioned this on the Matrix chat previously, but alpine images simply don't run at all on my Arch host. Converting to a glibc-based image (documented here) solved it completely.

[dessalines]

@TheSilkky is this something you would be able to handle, given that you've built our existing dockerfile that also works for ARM?

[TheSilkky]

Is this actually necessary? It won't simplify the build process at all, it'll just make it worse. The reason I used the blackdex/musl-musl image is because it includes a C cross compilation toolchain (and static openssl and libpq), the official Rust Docker images do not include any C cross compilation toolchain which is required for Lemmy to actually cross compile to arm64. This makes the build process more complicated because now we will have build our own cross compilation toolchain and other dependencies like libpq if we want to include them statically.

[dessalines]

There might be someone who makes / publishes cross compile images for gcc too. That might even be easier to find than the musl ones.

[TheSilkky]

@dessalines idk I couldn't find any tbh that was just a quick search so

[marklr]

I would wager the majority of deployments aren't on arm64 -- and there are significant issues with musl. Is arm64 a significant deployment target? If so, why not base x64 builds on bookworm but still use the musl image for arm64?

[Leopere]

What exactly was breaking in alpine? Packages are often virtually the same it’s usually just how you get them that changes.