Lock rabbitmq Erlang node name to a stable hostname (zulip + stoat)#26
Merged
Conversation
RabbitMQ namespaces its Mnesia data dir by Erlang node name — mnesia/rabbit@<container-hostname>/. Neither zulip nor stoat previously set `hostname:` on the rabbitmq service, so any container recreation (image bump, env change, manual --force-recreate) got a new random hostname and started as a "virgin node", orphaning the old node's data on disk and making vhosts / users / durable queues / persistent messages inaccessible. Fix: set `hostname: rabbitmq` (zulip) and `hostname: rabbit` (stoat — matches the service key), matching each project's existing service-name convention. Verified end-to-end in an isolated /tmp test that matches the production hardening (cap_drop ALL, user: 100:101, ipc: private, tmpfs, security_opt, RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS): - Container hostname resolves to the configured string. - Erlang node name is rabbit@<configured-hostname>. - After `docker compose up -d --force-recreate`: vhost and user tag preserved; node name unchanged; Mnesia dir contains only rabbit@<stable>/ (no orphaned random-hostname dirs). - Zero error / fatal / panic / "virgin node" lines in logs. CHANGELOG documents the migration step required for existing deployers (rename the on-disk Mnesia subdirs). Fresh deployments are unaffected.
wnstfy
enabled auto-merge (squash)
May 21, 2026 12:50
Contributor
Trivy misconfiguration findings (HIGH + CRITICAL)✅ No HIGH or CRITICAL misconfigurations detected. |
Contributor
CVE deltaNo changes. Every image's CVE set is identical to the previous scan. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Both
zulip/docker-compose.ymlandstoat/docker-compose.ymlhad a latent data-loss bug: their rabbitmq service didn't sethostname:orRABBITMQ_NODENAME. RabbitMQ stores its Mnesia DB undermnesia/rabbit@<container-hostname>/, and any container recreation (image bump, env change, manual--force-recreate) got a new random hostname → broker started as a fresh "virgin node" → vhosts / users / durable queues / persistent messages all became inaccessible (data was still on disk but orphaned).Fix: set
hostname: rabbitmq(zulip) andhostname: rabbit(stoat — matches the service key naming used in that compose file).Verification (per the internal upgrade-testing methodology)
Tested in
/tmp/upgrade-tests/rmq-hostname-fix/with a compose matching the production hardening (cap_drop: ALL,user: 100:101,ipc: private,tmpfs,security_opt: no-new-privileges,RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS):rabbitmq(matches thehostname:directive)rabbit@rabbitmq(stable)docker compose up -d --force-recreaterabbit@rabbitmqupgrade_test_vhostpreservedadministratorpreservedrabbit@rabbitmq/(no orphaned random-hostname dirs)Migration for existing deployers
Anyone with a running deployment of
zuliporstoatfrom this repo needs to rename their existing Mnesia node directory before the next container restart, otherwise the new container with the stable hostname will start fresh and the existing vhosts / users / queues will be unreachable. CHANGELOG documents the exact rename procedure. Fresh deployments are unaffected.Test plan
scan,misconfiginformational checks pass.rabbitmq 4.3.0-alpineRenovate PR can be merged safely (verified earlier in this session).