GitHub - avandeputte/haproxy-manager: A small self-hosted web UI to manage an HAProxy configuration, obtain Let's Encrypt certificates, and run an active-passive cluster with Keepalived on a shared virtual IP

A small self-hosted web UI to manage an HAProxy configuration, obtain Let's Encrypt certificates, and run a cluster of any number of nodes with Keepalived on a shared virtual IP — one node active, the rest ready to take over, with settings and certificates syncing across all of them.

# from a package: .deb and .rpm on every release (Debian, Ubuntu, RHEL, Fedora)
sudo apt-get install -y ./haproxy-manager_1.91.2_all.deb

# or the install script, on any Debian-based server
curl -fsSL https://raw.githubusercontent.com/avandeputte/haproxy-manager/main/install.sh | sudo bash

# or in Docker (linux/amd64 and linux/arm64)
docker run -d --network host --cap-add NET_ADMIN --cap-add NET_BROADCAST --cap-add NET_RAW \
  -v ham-data:/var/lib/haproxy-manager -v ham-acme:/var/lib/acme.sh \
  -v ham-haproxy:/etc/haproxy -v ham-keepalived:/etc/keepalived \
  ghcr.io/avandeputte/haproxy-manager:latest

Then open http://<node>:8080.

_{Every screenshot here is the real application, driven by a browser against
a three-node cluster holding made-up data — see
tools/screenshots.}

Detailed guides


Installing on a server	requirements, what the installer does, options, updating, uninstalling, troubleshooting
Running in Docker	images, compose, networking, volumes, capabilities, limitations
Configuration	every setting, what is shared between nodes, environment variables, ports
Authentication	the UI login and 2FA, basic auth for services, single sign-on (OIDC), the trust model

The rest of this file describes what each part does and why.

First run

The first visit asks for a username and password, then offers a setup wizard with two branches:

Join an existing cluster — point it at any node already running and give that node's API key. It registers itself there and that node pushes the whole configuration, the cluster settings and the membership list back. You do not touch the other nodes.
Create a new cluster, or run standalone — set the virtual IP the nodes will share (or skip it and run alone). Other nodes join later by pointing at this one.

Either way it ends with this node's API key, which is what the other nodes need to reach it. "Set this up later" skips straight to the UI.

The gear beside your name at the foot of the menu opens the account dialog: change the password (optionally applying it to the other nodes), and choose Appearance — System, Light or Dark. System follows the operating system's own light/dark setting and changes with it. The choice is stored with the node's administrator so it survives a new browser, and also remembered locally so the page is painted in the right colours before it has asked the server anything.

On a phone

The same application, laid out for the screen it is on rather than shrunk to fit it. Below 840px the menu becomes a drawer behind a button in the bar, which stays put as the page scrolls; below 680px each row of a table becomes a block with its column names in front of the values, dialogs take the whole screen, and fields are large enough that a phone does not zoom in when you tap one. In between, a table too wide for the screen scrolls inside its own card. The page itself never scrolls sideways — a phone answers that by zooming the whole interface out until the widest thing on it fits.

Publishing a service

Each row is one public name routed to a pool. Publish a service opens the wizard, which can start from a recipe:

The normal way to use this is Services → Publish a service: give it the URL people will visit and the server behind it.

Public URL    https://app.example.com
Forward to    http://192.168.1.100:1781

From those two lines it creates the Real Server, Backend Pool, host Condition, routing Rule, the HTTPS listener, an ACME certificate, and an HTTP listener that redirects to HTTPS — then applies. Preview shows exactly what it will create, and the resulting haproxy.cfg, before anything is written.

Wildcard certificates are reused, not duplicated. If a certificate already covers the host — *.example.com for app.example.com, or an exact name — the wizard attaches it instead of requesting another one, and says so before you commit. An exact name wins over a wildcard that would also match, and *.example.com correctly does not cover example.com or a.b.example.com. Set Certificate to always request a new certificate to override, or no certificate to terminate TLS elsewhere.
Several public URLs on one service: put one per line and every name reaches the same servers, as a single rule (use_backend be_app if acl_host-app or acl_host-www). One certificate covers them all, and adding a name later extends it rather than requesting another. Names must agree on scheme and port, since they share a listener, and a URL with a path cannot be combined with others — a host and a path must both match, while several host names are alternatives.
Several targets, comma separated, are load balanced across. Each may be named: galera1=192.168.1.81:3306.
Raw TCP works too — give it tcp://0.0.0.0:3306 as the public URL and the wizard builds a TCP listener, a TCP-mode pool and the servers. TCP carries no host name, so one port serves exactly one pool; publishing a port that is already taken is refused rather than silently merged. Load balancing, source-IP stickiness (a stick table), health check logging and a separate check port are all part of the same form, so this comes out of it:
```
backend be_mariadb_galera_pool
    mode tcp
    balance source
    option mysql-check user haproxy post-41
    option log-health-checks
    stick-table type ip size 50k expire 30m
    stick on src
    server galera1 192.168.1.81:3306 check inter 3s port 3306
    server galera2 192.168.1.82:3306 check inter 3s port 3306
    server galera3 192.168.1.83:3306 check inter 3s port 3306
```
Note type ip: that is how HAProxy spells an IPv4 stick table. type ipv4 is rejected outright (unknown type 'ipv4').

A health check can be set up in the same step, and servers that fail it are taken out of rotation:

Check	What HAProxy does
ping	opens a TCP connection to the port (HAProxy has no ICMP ping)
HTTP request	`option httpchk`, with a path and an expected status
TLS handshake	`option ssl-hello-chk`
PostgreSQL login	`option pgsql-check` — the login handshake only, no password
MariaDB / MySQL login	`option mysql-check`, `post-41` for anything modern

The database checks expect the servers to speak that protocol, so point them at the database itself; the wizard says so when you pick one.

The check can use a different port and protocol from the traffic. Set a check port and HAProxy checks each server at its own address on that port, which is how a PostgreSQL cluster behind Patroni is fronted: route TCP to 5432 while an HTTP check asks each node's own API on 8008 whether it is the primary. The HTTP check can carry a method, an HTTP version and a Host header, and a pool can override the connect, server and check timeouts:

backend be_postgres_backend
    mode tcp
    balance source
    option httpchk
    http-check send meth GET uri /master ver HTTP/2 hdr Host localhost
    http-check expect status 200
    timeout connect 5s
    timeout server 30s
    stick-table type ip size 50k expire 30m
    stick on src
    server postgresql1 192.168.1.111:5432 check inter 3000 port 8008
    server postgresql2 192.168.1.112:5432 check inter 3000 port 8008
    server postgresql3 192.168.1.113:5432 check inter 3000 port 8008

A path works too: https://app.example.com/api routes only that prefix. More specific rules are placed ahead of broader ones, so a host+path rule is never swallowed by the host-only rule for the same name.
Publishing the same URL again edits it — repointing a service replaces its target rather than quietly adding a second server behind it.
Editing changes a service, it never clones it. Edit follows the service's own objects, so changing its URL, health check, balancing or targets updates the rule, pool, monitor and certificate it already has instead of leaving them behind beside a new set. New objects appear only where there was none before, and settings an edit does not mention are left alone. Objects shared with something else — a monitor another pool uses, an already-issued certificate — are never altered underneath it.
Overview is the landing page and lists every configured service alongside node health, certificates and the generated configuration. Services shows the same table on its own. Delete removes the objects that mapping alone was using.
Pause puts a service into maintenance mode: every request is answered with a clean 503 ("This service is down for maintenance"), while the servers and their health checks stay exactly as they were — pausing does not read as an outage, and Resume takes effect immediately instead of waiting for checks to pass again. The paused state is part of the shared configuration, so it survives a failover. It is also a switch on the pool's edit dialog, and — opted in — a switch in Home Assistant.

Requiring a sign-in

A service can ask its visitors to prove who they are, two ways — and both are enforced by HAProxy itself, so an unauthenticated request never reaches the servers behind it:

Basic authentication — a user name and password, checked from a userlist in the generated configuration. Users and groups are managed under Sign-in; a service admits groups rather than people, so access changes by moving someone in or out of a group. Passwords are stored only as SHA-512 crypt hashes, and a sign-in nobody can satisfy renders as http-request deny — refusing everyone is safer than quietly becoming public.
Single sign-on (OIDC) — the service sends visitors through an OpenID Connect provider (Authentik, Keycloak, Authelia, Pocket ID, Google, Entra), with a per-service allow-list of emails and @domains. One sign-in covers every protected service; HAProxy verifies the session on every request in pure configuration — an HMAC-signed cookie, no Lua, HAProxy 2.4+ — so the app stays out of the traffic path and a failover signs nobody out. The upstream app sees none of the login unless a per-service toggle passes the verified identity as X-Auth-Request-Email/Remote-User for apps that trust a proxy identity.

Source-address controls compose with either: Allowed networks (a CIDR allow-list, which works for tcp:// services too) and Skip the sign-in from (networks trusted without a password, typically the LAN).

The whole subject — the settings, the sign-in flow, provider setup, the trust model, and what to do when something refuses — lives in docs/authentication.md.

The Advanced · HAProxy menu group still exposes every object individually, for the cases the wizard does not cover (header rewriting, custom ACLs, per-object tuning). Every editor there carries a second tab, haproxy.cfg, showing exactly what Apply will write for that object — recomputed from the values as they stand in the form, not as they were saved, so an edit can be read before it is committed. Raw lines the fields do not cover go in each object's Extra directives; the file itself stays generated, because parsing a hand-written haproxy.cfg back into objects would be guesswork dressed up as a feature. The pages mirror the OPNsense plugins this UI was modeled on:

This UI	OPNsense `net/haproxy`
Public Services	Virtual Services → Public Services (frontends)
Backend Pools	Virtual Services → Backend Pools
Real Servers	Real Servers → Servers
Conditions	Rules & Checks → Conditions (ACLs)
Rules	Rules & Checks → Rules (actions)
Health Monitors	Rules & Checks → Health Monitors
Settings	Settings → Global / Default / Statistics

This UI	OPNsense `security/acme-client`
Accounts	Accounts
Challenge Types	Challenge Types (HTTP-01 / DNS-01)
Certificates	Certificates
Settings	Settings

(OPNsense's Automations have no equivalent here: what has to happen after a renewal happens on its own.)

How it works

All state lives in a single JSON file ($HAM_DATA_DIR/config.json). No database.
Timestamps are stored and sent in UTC, and shown in the browser's own timezone — certificate expiries, snapshots, the watchdog's actions, log lines. The server has no idea where the reader is; the browser is the one thing that does.
Apply renders haproxy.cfg, validates it with haproxy -c before writing anything, then writes the file (keeping a .bak) and reloads HAProxy. If Keepalived is enabled it renders and reloads keepalived.conf too.
Settings are checked before they are stored. The settings pages have a Validate button that renders the configuration those values would produce and runs haproxy -c (and keepalived -t where it applies) without saving anything, and Save refuses outright if the result would not work — so a mistyped directive cannot be stored and then block every Apply until someone finds it. The full checker output is shown either way.
Only the node holding the virtual IP issues and renews. HTTP-01 validation arrives at that address, so a passive node could not answer it, and with DNS-01 the nodes would race each other for the same certificate and burn the CA's rate limits. A passive node says so instead of trying, and refuses a manual Issue or Renew all now.
Everything a renewal needs happens by itself. Once a certificate is written, HAProxy is reloaded so it actually serves it — it keeps certificates in memory, so a new file changes nothing until it reloads — and the PEM is pushed to every other node, so a failover serves the current certificate rather than the one that node last saw. There is nothing to configure.
ACME issuance/renewal shells out to acme.sh. Certificates are written as combined fullchain + key PEMs into the HAProxy certificate directory (what HAProxy's crt expects). HAProxy is then reloaded and the certificate pushed to the other nodes, with nothing to configure. A built-in loop renews on the interval set in Settings → ACME Settings.
HTTP-01 challenges use a local acme.sh --standalone listener. When "HAProxy integration" is on, every HTTP Public Service automatically routes /.well-known/acme-challenge/ to it, and HTTP→HTTPS redirects skip that path — so you can keep port 80 fronted by HAProxy and still validate.
Apply checks that HAProxy is actually serving afterwards. systemctl returning success means the reload was accepted, not that HAProxy came back: a configuration that passes haproxy -c can still fail to start, because -c never binds a socket. So Apply asks HAProxy over its stats socket, and if it is not answering it puts the previous haproxy.cfg back, reloads again, and says so — loudly, and by notification. Without that a node can be left not listening at all, which looks like the node being down rather than like a configuration that was just applied.
Before a real certificate exists, Apply drops in a short-lived self-signed placeholder so HAProxy can start; the first successful issue replaces it. It covers every name the certificate is for, and is replaced when those names change — a service answering for two names with a stand-in covering one of them serves the wrong certificate on the other.
Certificate status is shown on Overview and under ACME → Certificates: whether the PEM on disk is a real certificate or still the placeholder, its issuer, its expiry date with the days remaining, and the outcome, timestamp and full acme.sh log of the last issue/renew attempt (the Log button).
The names on the record are not the names in the file. Adding a name to a certificate does not reissue it, so until it is issued again that name is served whatever was issued before — which a browser reports as a failure to connect, not as a certificate problem. The names are read from the PEM itself, so the Certificates page says not in the issued certificate and names them, and the publish wizard says so at the moment it extends one.

High availability

Any number of nodes. One holds the virtual IP and serves traffic; the others stand by with the same configuration, ready to take it.

Cluster health

Overview opens with a Cluster table: every node's role, HAProxy and Keepalived state, which virtual IPs it currently holds, its certificate health, its version and how long it took to answer. Each node asks the others directly, in parallel, so the view is live rather than remembered.

It calls out the conditions that are otherwise invisible until traffic stops:

No node holds the virtual IP — nothing is being served on it.
Two or more nodes hold it at once — split brain; they are not seeing each other's VRRP.
Nodes holding an older configuration than the rest — see below.
Nodes with unapplied changes, nodes running different versions, and nodes that did not answer (with the reason: unreachable, or the API key this node holds for it was rejected).

What is shared, and what is each node's own

The configuration has two containers, not one container with some objects marked. The shared sections are what every node has. local is what only this node has — and that includes whole objects, not just settings: the pool, server, health monitor, rule, conditions and certificate that publish this node's own management UI live there.

Sharing is therefore a copy rather than a filter: what is sent is the shared sections as they are, and what is compared is the same value. Nothing has to decide, on the way past, which objects belong to whom.

A listener is shared while the rule attaching this node's UI to it is not, so local.attach records that by the listener's name — every node has one called https-443, with a different id — and by position, since HAProxy takes the first matching use_backend and serves the first certificate to clients that send no SNI. Rendering merges the two.

The nodes agree, or they do not

Reachable is not the same as up to date. A node that was unreachable when a change was applied keeps the configuration it had, and until it takes the virtual IP nothing about it looks wrong.

So the shared configuration — everything except node-local settings and the objects a node owns alone — carries a revision: a counter that moves whenever that configuration changes, and a fingerprint of its contents. Every node reports both, the Cluster table shows them per node, and the header says configuration agreed or configuration differs at a glance.

Three things follow from it:

A node that is behind is named, with the revision it holds and the one the cluster is on.
A node cannot push a configuration older than the one already there. An isolated node that was edited and then reconnected would otherwise overwrite the current configuration with its own; it is refused, with both revisions in the message. To discard what is on the other node instead, Overwrite on the Cluster page lifts this node's revision above every other node's and pushes, so they end up on the same configuration and the same revision.
It heals itself. With Keep the nodes in step on, the background health check already asks every node how it is; any node reporting an older revision is brought up to date from the node holding the newest, and a node that has just started takes the newest configuration from the cluster before it can serve anything stale. Nothing is queued, so nothing is lost — the next round observes the same disagreement and acts on it again.

The Cluster page

Everything about the cluster lives on one page, split by what it applies to:

Cluster settings — the virtual IPs, virtual router ID, VRRP password, advertisement interval, initial state, nopreempt and HAProxy tracking. These must be identical everywhere, so they are part of the shared configuration and travel with a push.
This node — whether Keepalived runs here, the interface, this node's priority, its unicast addresses, the URL the others should use to reach it, and its API key. Never synced: these are meant to differ.
Other nodes — one entry per node with its URL and the API key configured on that node. A push also hands each node the membership list, including a way back to this one, so you maintain the list in one place. Keys are stored per peer and never sent back to the browser.
This node right now — the diagnostics described under Keepalived below.

An existing two-node setup is migrated automatically: the old single peer becomes the first entry, and the shared VRRP settings move out of the node-local section.

Only the active node is editable

The node holding the virtual IP is where the shared configuration is edited; the others are read-only and show a banner saying so. This stops nodes from diverging and then overwriting each other on the next push.

A passive node can still fix itself — interface, priority, unicast addresses, peer list, login, API key, updates, and Apply — because a node that cannot take the VIP has to be repairable. And when no node holds the VIP, the banner's Edit here anyway unlocks that node, so a broken cluster is never a lockout. The lock is enforced by the API, not just hidden in the UI.

The unlock belongs to the sign-in that asked for it: Lock again restores it, and so does signing out or back in. It is not stored, so it cannot be left on by accident, and another browser signed in to the same node is unaffected.

Keepalived

Keepalived runs on every node that has a virtual IP configured — it is not a per-node switch, so a node cannot sit in a cluster with VRRP quietly off. Unicast addresses are derived from the node list: each node asks the others for the address on their VRRP interface, excluding the virtual IP, which is why a DNS name is never used for this. There is nothing to type.
Keepalived runs VRRP on every node with a shared virtual IP. Bind your Public Services to that VIP. Keepalived's settings are node-local — set them separately on each node, and they must agree on the virtual router ID. For non-preempting failover set every node to BACKUP with a different priority and enable nopreempt; the highest-priority node holds the VIP, and a recovered node won't yank it back.
Tracking HAProxy decides when a node should give the virtual IP up. The default asks HAProxy through its admin socket whether it is serving, so an instance that is running but wedged — accepting connections and answering none — hands the address to a node that works. The alternative, process, only checks that something called haproxy exists, which a hung one still does. The check is a small script written next to keepalived.conf by the same Apply that writes it, and where there is no admin socket to ask it falls back to looking for the process rather than failing a healthy node. The Keepalived page diagnoses this node: whether the configured interface exists, whether the config was written, the VRRP state, and the keepalived -t output. The state is read from the journal when it is there to read — only the journal distinguishes FAULT from BACKUP — and otherwise worked out from whether the node holds the virtual IP, with the page saying which.
Sync is push-based: the node you edit pushes to the others. A renewed certificate is pushed automatically by the node that renewed it.

Node-local vs. synced

Synced between nodes	Node-local (never synced)
Real Servers, Backend Pools, Public Services	Keepalived (interface, VRID, priority, VIP)
Conditions, Rules, Health Monitors	The peer list and their API keys
HAProxy Settings	API key
ACME accounts, challenges, certificates, automations	Administrator login
Deployed certificate PEM files

Reaching the UI over HTTPS

Settings → Web UI access publishes this management UI through HAProxy itself, so it answers at a name you choose:

Serve the UI through HAProxy   [x]
Address                        https://proxy.example.com

It builds the same objects the publish wizard would — a pool pointing at 127.0.0.1:8080, a host rule, the HTTPS listener, an HTTP→HTTPS redirect and a certificate — and reuses a wildcard that already covers the name. Turning it off removes them again. A host name already used by another service is refused rather than quietly stolen from it.

Set it on each node, with that node's own name. The setting and everything it creates are node-local: they are stripped from what a node sends its peers and preserved when a configuration arrives, so proxy1 never turns up on the other nodes. The service appears in the Services list marked as managed, with no Edit or Delete — change it from this page, or turn it off.

Once it works, set HAM_LISTEN=127.0.0.1 in the service unit and restart, so the plain-HTTP port is no longer reachable from anywhere but HAProxy. The page says so while the UI is still listening on all addresses.

Requesting a certificate

Certificates → Request a certificate asks for the domains and then, for the two things a certificate needs, lets you either reuse what is already there or create it in the same step:

ACME account — an existing one, or a new one with its e-mail and CA (letsencrypt_test issues untrusted certificates with no rate limits, which is what you want while setting things up).
Challenge type — an existing one, or a new HTTP-01 or DNS-01.

Preview shows exactly which objects will be created or reused before anything is saved, and Request it now runs acme.sh immediately and shows its log. It warns about the mistakes that are otherwise only visible in a failed issuance: a wildcard with an HTTP-01 challenge (only DNS-01 can validate one), a DNS-01 challenge with no API hook, and an account with no e-mail address.

DNS API hooks

The DNS API hook field lists every hook the acme.sh on this node actually provides — 191 of them, by provider name — and picking one shows the credentials it needs, with a button that fills the variable names into the credentials box:

CloudFlare needs:
  CF_Key    — API Key
  CF_Email  — Your account email
or instead:
  CF_Token  — API Token
  CF_Account_ID — Account ID
  CF_Zone_ID — Zone ID. Optional.

That list is parsed from acme.sh itself rather than hard-coded, so it stays correct as acme.sh adds providers. The field still accepts anything typed, so an unknown or newer hook name is passed through unchanged.

Statistics

Statistics reads HAProxy's admin socket (show stat) and refreshes every five seconds:

Listeners — status, current/max/total sessions, request rate, bytes in and out, denied requests and errors.
Each pool — its own status, how many servers are up, and per server: state (UP / DOWN / MAINT / DRAIN / NOLB) with how long it has held it, active or backup, weight, sessions, queue, traffic, the last health check result (L7OK, L4CON, …) with its duration, failed-check and flap counts, and total downtime.

A server with health checking switched off reports no check and counts as up, because HAProxy still routes to it.

Traffic history

The Statistics page shows what is happening now; the Traffic card on it shows what happened. Once a minute each node records, per pool, how many requests it served and how many server errors it returned, and keeps a day of it — enough to answer when did this start, which a live gauge cannot.

The Services page carries the same thing as a sparkline per service, with server errors drawn over the requests, because the question is always whether they happened at the same time.

It is per node and only covers time the app was running: a gap in the line is a gap in the recording, not in the traffic. Counts are per minute rather than totals, and a counter that goes backwards is treated as HAProxy having restarted rather than as negative traffic.

The traffic this app generates itself is not counted. The once-a-minute URL probes go through HAProxy on purpose — that is what makes them honest — so HAProxy counts them like anyone else's requests, and a service nobody visits would show a steady line of the app talking to itself. The history subtracts what the probes put through, including the 401 a sign-in answers a probe with, so zero visitors reads as zero. (HAProxy's backend health checks were never in these numbers — HAProxy accounts for them separately.)

The charts cannot overstate a trickle. A sparkline scaled purely to its own peak turns a flat 1 request a minute into a solid block that reads as more traffic than a real rush on the row above it. The scale therefore never drops below 10 requests a minute: a trickle draws as the low band it is, and anything actually busy still gets its own scale. Server errors are drawn on the same scale as the requests, for the same reason.

Metrics for Prometheus

GET /metrics speaks the Prometheus exposition format: per-pool request and error counters straight from HAProxy, servers up per pool, certificate expiry timestamps (and whether the deployed file is still the self-signed stand-in), the URL probes' verdicts, cluster agreement, and the watchdog's view of each service. Everything is read from state the app already keeps, so a scrape costs what a page load costs. Note the request counters are HAProxy's own and include this app's URL probes — the subtraction only applies to the built-in traffic history.

It requires the node's API key, because service names and certificate expiries are not for whoever can reach the port:

scrape_configs:
  - job_name: haproxy-manager
    authorization:
      credentials: <the API key from Cluster - This node>
    static_configs:
      - targets: ["proxy1:8080", "proxy2:8080", "proxy3:8080"]

Scrape every node: each answers for itself, and ham_node_active says which one holds the virtual IP.

Home Assistant

Point Notifications → Home Assistant at an MQTT broker and the entities appear in Home Assistant by themselves — MQTT discovery, no YAML, no polling, and nothing extra installed on either side. The Test button connects and publishes before anything depends on the settings.

Each node publishes a small device of its own: whether it holds the virtual IP, and whether its HAProxy is answering. The node that is serving also publishes the cluster's view, as one shared device:

a problem sensor per service, honouring that service's Alert when setting — a Patroni pool at 1 of 3 shows healthy here too
a connectivity sensor per published URL, from the URL probes
days to expiry per certificate, with its domains as attributes
configuration drift and nodes reachable, when there is a cluster
requests per minute per service, from the traffic history (with this app's own probes already subtracted)

On failover the new active node simply continues publishing the same topics, so the entities carry on rather than duplicating per node.

The connection is held open for the sake of the will: the broker flips this node's availability topic to offline the moment the process dies, and Home Assistant greys the entities out — the one state a dead process cannot report for itself, and the reason MQTT beats polling here.

Allow control from Home Assistant (off by default) adds a maintenance switch per service: flipping it pauses the service with a clean 503, exactly like Pause on the Services page, and flipping it back resumes. Leave it off unless you want it — anyone who can publish to the broker holds this power the moment it is on, so keep the broker behind credentials you trust.

No broker, or no wish for MQTT? The webhook destination under Notifications posts JSON that a Home Assistant webhook trigger can consume directly — zero code, though it only carries alerts, not entities.

Notifications

Notifications sends when something needs a person. Nothing extra is installed for any of it: the whole feature uses the standard library and the requests package that is already there.

Destination	Notes
Email (SMTP)	STARTTLS, SSL or plain; authentication optional
Pushover	severity maps to Pushover priority (quiet / normal / high)
Webhook	`POST {subject, message, severity, event, node, time}`, custom headers

The webhook is the escape hatch: it posts JSON, so a few lines of script can forward an alert to anything not listed above.

Test each destination from the page: it sends a real message, so it is proven before it is needed.

When a service loses its servers

HAProxy is already health-checking every server, and it is the thing actually deciding where traffic goes — so that is what the alerts come from. A service whose servers all fail their checks is reported as down, one that loses some of them as degraded, naming the servers and what their last check said. When it recovers, that is reported too.

A paused service says nothing about its servers. Pausing a service is usually the prelude to taking its backend down and working on it, at which point the servers fail their checks — so a paused service is left out of these alerts entirely. When it is resumed and its servers pass again, the next round picks up from wherever it left off.

What losing a server means is the service's own business. Not every pool means the same thing by a failing check. A load-balanced pool losing one of three is degraded — but a Patroni pool is designed to have exactly one server passing, because the health check is what does the routing: "down to 1 of 3" is its healthy state, and alerting on it is crying wolf on every round. So each service carries an Alert when setting, in the publish wizard beside the health check:

a server is lost (the default) — any failing server is a warning, a pool with none left is an error
no server is left — only the outage is news; servers failing their checks is normal running. The right choice for leader-election pools, and what the Patroni recipes now set by themselves. Recovery is reported the moment one server passes again, worded for what it is — "serving again", not a false "all servers healthy".
never — this service looks after itself

Only the node holding the virtual IP says anything about services. Every node runs the same checks, so three nodes would send three copies of every alert — and a passive node's view is not the one that matters: it is not carrying the traffic, and a server it cannot reach may be perfectly reachable from the node that is. Faults about a node itself — its HAProxy stopped, its certificate could not be renewed — still come from that node, because nobody else can see them.

It alerts on changes, not on conditions

A recovery is always delivered, whatever the severity threshold says. Recovery messages are informational and the threshold defaults to warning, so without that exception someone would be told what broke and never that it came back — which is worse than silence, because it leaves them believing it is still broken. A recovery for something that was never reported is not sent.

The watchdog runs every twenty seconds. Anything that reported a state would arrive thousands of times a week, so alerts fire on transitions and an unresolved problem is repeated only every repeat_hours (6 by default) until it clears. Six watchdog rounds against one dead service produce two messages — "was restarted", then "is healthy again" — not six.

One alert waits deliberately: the nodes holding different configurations. Saving a change makes the cluster disagree by design — the other nodes catch up when Apply pushes to them — so the moment of divergence is nearly always the middle of ordinary work. That alert is sent only once the disagreement has stood for 30 minutes, longer than an edit-then-Apply should take, and its subject says how long — so the reader knows it is not the save they made a minute ago. The Cluster page still shows the disagreement immediately; it is only the email that waits.

What it can tell you about, each switchable:

Certificates — issued, or failed with the reason
Watchdog — a service restarted, beyond repair, or unrestartable because its configuration is broken (the message names the offending line)
Apply — refused by validation, or HAProxy did not reload
Cluster — a node stopped answering, came back, or split brain
Updates — a new version is published

min_severity sets the floor: error for breakage only, warning to include repairs, info to include recoveries and new versions.

Settings are shared, so configure them on one node and they propagate — each node then alerts about its own troubles. Note that this means SMTP passwords and Pushover tokens travel in the sync payload: run peer sync over HTTPS, or keep it on a trusted network. Because every node watches every other one, a node that vanishes is reported by each of its peers — which also tells you who lost sight of it.

Watchdog

Each node supervises its own services. Watchdog shows what it sees and what it has done.

The point is the distinction between stopped and hung. systemctl is-active answers "is the process there", which a wedged process passes while serving nothing — so each service gets a probe that makes it do something:

Service	Liveness probe	Restarted when
HAProxy	`show info` on its stats socket	the service is stopped or failed, or it does not answer within 5s
Keepalived	the service is running when the cluster wants it	it is stopped or failed while this node should be running it
This app	a real HTTP request to its own listener	see below

It restarts deliberately, not reflexively:

Never against a configuration that cannot work. If haproxy -c rejects the file, restarting is a loop that hides the fault, so it stops and says which line is wrong.
Never a service you disabled. A masked or disabled unit is taken as "leave this alone" — a node in maintenance stays in maintenance.
Never endlessly. Three restarts per fifteen minutes by default; after that it stops and reports, so a failing service stays visible instead of flapping.
Everything it does is logged, so the Logs page carries the history.

The published URLs are asked, the way a browser would ask

Every check above looks at a piece: the health checks watch the backend servers, the watchdog watches the processes, the VRRP tracking script watches the admin socket. All of them can be green while https://app.example.com answers nobody — DNS pointing at the wrong machine, another host claiming the address, a listener that lost its certificate. So once a minute, the node holding the virtual IP requests every published URL exactly as a visitor would: resolve the name, connect, speak TLS, ask.

Three answers. It answers — any HTTP status counts, including the 401 of a service behind a sign-in, and the 503 of a pool whose servers are down (that one is already alerted on by the health checks, so the probe stays quiet about it). It answers but the certificate does not verify — expired, the wrong name, or an issuer this machine does not trust; a visitor would get a warning page, so this is a warning here. No answer at all — including a name DNS cannot resolve, which is reported as exactly that. TCP services are a connection attempt to their port.

Failures show beside the URL on the Services page and go out as notifications (the service event), with what failed and what the name resolved to. Only changes are reported, and recovery closes the loop. Turn it off under Settings → Watchdog if your names only resolve from outside your network.

Two machines using one address

Every few minutes the watchdog asks the network whether anything else answers for the addresses this node holds — arping -D, which asks without claiming, so a reply can only come from somebody else. If one does, the Watchdog page says so and a notification goes out.

It is worth checking because it is invisible from every layer above the network: the address is configured on this node, the socket is listening on this node, and a client reaches whichever machine won the last ARP exchange. The symptom is a node that answers from some places and not others, and comes and goes for no visible reason — which looks like almost anything except two machines claiming one address. A cluster that deliberately moves addresses between machines is exactly where it happens.

Nothing here can fix it: one of the two has to stop using the address.

Watching the app itself

A watchdog inside a process cannot restart that process, so systemd does it. The unit sets WatchdogSec=90, and the app pings systemd only when a real request to its own listener succeeds. That catches the failure that matters: every worker thread blocked, process healthy, UI answering nothing. Pinging from a timer would report health from inside a process that serves none.

Verified by stopping the process with SIGSTOP — systemctl is-active still said active, and systemd restarted it on the deadline:

systemd[1]: haproxy-manager.service: Watchdog timeout (limit 1min 30s)!
systemd[1]: haproxy-manager.service: Failed with result 'watchdog'.
systemd[1]: haproxy-manager.service: Scheduled restart job, restart counter is at 1.

In Docker there is no systemd: supervisord restarts the app if it exits, and the image's HEALTHCHECK reports whether the UI answers, but nothing restarts a hung container unless your orchestrator acts on that health status.

Node health is collected here too

The watchdog polls every node on a schedule and keeps the result, so the UI reads a snapshot instead of asking each node while you wait. The Cluster panel shows the snapshot's age and has a Refresh button for a live round. With one unresponsive node: 5.1s to collect, 3ms to read.

HAM_CLUSTER_POLL (15s) sets the collection interval; HAM_CLUSTER_MAX_AGE (60s) is the age beyond which a request collects it inline rather than show something stale.

Logs

Logs merges four sources into one timeline, newest at the bottom:

Source	Where it comes from
Web UI	this app's own log — sign-ins, every configuration change and who made it, apply results, certificate outcomes, sync results
HAProxy	`journalctl -u haproxy`, falling back to `/var/log/haproxy.log` or `/var/log/syslog`
acme.sh	acme.sh's own log, plus the recorded outcome of every issuance
Keepalived	`journalctl -u keepalived`, with the same fallback

Tick the sources you want, filter by level, search the text, and choose how many lines to keep. Follow re-reads every five seconds and stays pinned to the bottom; untick it to scroll back without the view jumping. Download saves exactly what you are looking at, filters and all, as plain text.

Timestamps are the node's own, and lines that carry none sort to the end rather than to 1970. Requests are logged with the object's name but never the request body, so passwords, API keys and DNS credentials do not reach the log.

The app writes its own log to /var/lib/haproxy-manager/haproxy-manager.log (mode 0600, rotated at 4 MB, three kept) and to standard output, so journalctl -u haproxy-manager shows the same lines.

In the Docker image there is no journal, so a small collector binds /dev/log and tees it to both the container log and /var/log/ham-syslog.log, which is what the viewer reads.

Updates

The app carries a version (VERSION, starting at 1.0) and asks GitHub for the published one once a day. When a newer version exists, a chip appears in the header and Settings → Updates offers a one-click update.

The whole cluster from one node. Where there are other nodes, Updates offers Update the other N nodes as well, ticked by default — visiting each node to press the same button is the thing worth avoiding. They are told first, while the node you are on is still running to tell them, and each restarts when its own update finishes. Every node runs the same installer against the same source, so there is nothing to hand over, only the instruction. A node that does not take it is named with the reason and stays on the version it had; nothing retries it, because an update is something a person started and the node that missed it is their decision to make. The Cluster page shows the version each node ends up on, and warns when they are not all the same.

The update runs install.sh --update --yes on the node under systemd-run, in its own transient unit. That detail matters: as a child of the service it would be killed halfway, because restarting haproxy-manager.service takes down everything in that service's cgroup. Progress is streamed into /var/lib/haproxy-manager/update.log and shown live in the UI; the page keeps polling across the restart. Your configuration, certificates and login are kept, and HAProxy keeps serving traffic — only the management UI restarts.

One-click update applies to the installer-managed (systemd) install. In a container the button explains that you should pull a new image instead.

To publish a new version: bump VERSION, push, and every node offers it within a day. The check asks the GitHub API rather than raw.githubusercontent.com, because raw is behind a CDN that keeps serving the old file for up to five minutes after a push — long enough for a check straight after a release to report the previous version. Raw is the fallback if the API is unreachable or rate limited. HAM_VERSION_URL / HAM_INSTALL_URL point the whole mechanism at a fork or a private mirror.

The update fetches a script over the network and runs it as root. It is pinned to the repository above, and reaching it already requires an administrator login — the same login that can run arbitrary commands through an ACME automation — but if you would rather not have that path at all, leave the button alone and update with install.sh --update over SSH.

If a configuration change goes wrong

config.json is written by rename, so it is never half-written, and the previous version is kept beside it as config.json.bak. haproxy.cfg and keepalived.conf keep a .bak too, written before each Apply.

The management UI is served by the app itself on port 8080, not through HAProxy, so http://<node-address>:8080 reaches it even when HAProxy is misrouting — which is the way back in if a published address stops answering. To put HAProxy back the way it was:

cp /etc/haproxy/haproxy.cfg.bak /etc/haproxy/haproxy.cfg
systemctl reload haproxy

If the UI's own service is what is broken, Settings → Web UI access → Save rebuilds it from the stored setting — the pool, the rule, the conditions for both its addresses and the certificate — and re-attaches it to the listener. Set Certificate to request a new one if the listener has lost the one it was serving: an HTTPS listener with no certificate fails validation, and Apply refuses rather than writing it, which leaves the node showing unapplied changes until it is put right. Apply names that case when it happens.

Configuration history

Settings → History lists every state the shared configuration has passed through on this node — the last 50, newest first, each entry saying what it changed ("haproxy.backends: 1 added, 1 changed"). A snapshot is taken whenever the shared configuration actually changes, including when a peer pushes a configuration over this node's — that one deliberately does not count as a change of this node's own, so counting revisions would miss it, and it is precisely the case worth being able to undo.

Diff vs now names the objects that stand between then and now — added, removed, or changed — the same way the Cluster page compares two nodes. Restore puts a state back as a new change: it takes the next revision rather than the old one, so the rest of the cluster sees it as the newest configuration, which happens to have older contents. Nothing is applied or synced until you press Apply, so the result can be reviewed first. Node-local settings — Keepalived, the login, the API key — are untouched.

The snapshots live on each node's own disk (history/ in the data directory, mode 0600 like the configuration itself); each node remembers what it saw.

Backup & Export

Settings → Backup & Export covers two different jobs:

Generated files — download the haproxy.cfg and keepalived.conf this configuration renders to, exactly as Apply would write them. Downloading changes nothing on the node.
Configuration backup — a JSON file holding everything the UI manages (Real Servers, Backend Pools, Public Services, Conditions, Rules, Health Monitors, HAProxy Settings, every ACME object, and the users and groups a service can ask visitors to sign in with — without their passwords). Restoring replaces all of those and leaves node-local settings — Keepalived, Sync, the login, the API key — untouched, so the same file can seed a second node. A restored user keeps no password, and one already on the node keeps the one it has. Nothing is applied until you press Apply, so you can review the result first.

The backup deliberately contains no secrets: no API key, no login, no private keys from the certificate directory, and no DNS-provider credentials, EAB keys, or single sign-on client secret. Certificates move between nodes over Sync, or are re-issued; a restore onto the same node fills the stripped secrets back in from what is stored, and a restore elsewhere asks for them again. That makes the file safe to keep off the node.

Security

A second factor, if you want one. The account dialog can require a six-digit code from an authenticator app at every sign-in — standard TOTP, which any app produces. Enrolment only completes once the phone proves it holds the secret, so an unscanned QR code can never lock the account, and eight single-use recovery codes are shown once at setup. The pushed administrator record carries the second factor too, so a failover node asks for the same code. Lost phone, no recovery codes: app.py disable-2fa on the node's own shell — deliberately physical, because whoever can run that already owns the machine. The QR code is drawn by a small vendored encoder, verified module-for-module against a reference implementation; no dependency was added.
Sign in with a username and password. The installer creates the administrator and prints the generated password (also written to /var/lib/haproxy-manager/admin-credentials.txt, mode 0600); change it from the account dialog, behind the gear beside your name. Passwords are stored only as a PBKDF2-SHA256 hash, the session is an HMAC-signed HttpOnly / SameSite=Strict cookie that expires after 12 hours, and repeated failures lock that address out briefly. The login is node-local — set it on each node.
The API key (Cluster → This node) is for machines, not people: the peer must present it before it may push configuration here, and scripts can send it as X-API-Key instead of signing in.
If no administrator exists yet, the UI asks you to create one on first visit. Until then only the calls that create it answer — everything else returns 401 — so a node waiting to be set up does not hand its configuration to whoever reaches it first.
Every API endpoint requires a session or the API key. Of 87 routes exactly three answer without either: /api/login, /api/whoami (which unauthenticated returns nothing but whether an administrator exists), and /api/setup, which refuses once an administrator exists. This is verified by a test that walks every route and checks the rest refuse an anonymous caller. On a node with no administrator yet, /api/setup/state answers too, so the browser can tell it must offer the setup wizard; it returns 401 the moment an administrator exists. The sign-in page and its icons are served without a session too, because the page has to render before anyone can sign in; they come from a fixed list of filenames, not from the directory. The /.ham-sso/ routes for single sign-on are public by design — they exist to authenticate strangers — but they answer only on the configured sign-in host, act only on HMAC-signed parameters, and HAProxy routes nothing else to them.
The administrator login is node-local, but it can be copied. Each node stores its own; changing the password on one leaves the others as they were, which is a problem you tend to discover during a failover. The account dialog therefore offers Apply to the other nodes whenever there are any. What travels is the stored PBKDF2 salt and digest, over the peer channel, authenticated with the receiving node's API key — never the password, and never from a browser. If a node does not take it, the dialog says which one and why, and stays open: that node keeps the old login until you fix it.
Put the UI behind TLS (or an SSH tunnel / reverse proxy). Over plain HTTP both the password and the session cookie cross the network in the clear.
The service runs as root because it writes /etc/haproxy, /etc/keepalived and reloads services. Restrict who can reach port 8080.
Served by waitress, a production WSGI server — deliberately as a single process with a thread pool. The app keeps state in process globals (the lock that makes configuration writes atomic, the failed-sign-in counters, the renewal timer), so running several worker processes would give each its own copy and let concurrent edits overwrite one another. Raise HAM_THREADS if you need more concurrency; do not put a multi-process server in front of it. If waitress is missing the app still starts, on the development server, and says so in the log.
Request bodies are capped at 16 MB, and config.json — which holds the API key, session secret, peer keys and password hash — is written mode 0600.

Install

Debian-based distributions (Debian 12/13, Ubuntu 22.04/24.04), on every node:

curl -fsSL https://raw.githubusercontent.com/avandeputte/haproxy-manager/main/install.sh | sudo bash

From a checkout, sudo ./install.sh installs those files instead of downloading. If HAProxy Cluster Manager is already installed the same command detects it and offers to update, remove (keeping config.json and certificates), purge (removing those too), or cancel; piped from curl with no terminal to ask on, it updates in place and says so.

It installs haproxy, keepalived, python3-flask, python3-requests, python3-waitress, openssl, socat and iproute2 from apt, a pinned acme.sh with no cron of its own (the manager drives renewals), enables net.ipv4.ip_nonlocal_bind so HAProxy can bind a VIP this node does not hold, creates the administrator and API key, and installs the systemd unit.

Nothing in HAProxy's configuration is touched until you press Apply. The first Apply overwrites /etc/haproxy/haproxy.cfg, keeping a .bak.

→ Full installation guide — every option, what happens in what order, where each file lives, updating, uninstalling and troubleshooting.

Docker

Multi-architecture images (linux/amd64 and linux/arm64) are published to the GitHub Container Registry:

docker pull ghcr.io/avandeputte/haproxy-manager:latest   # or :1.46 to pin
docker compose up -d                                     # on every node

The image is all-in-one: the manager, HAProxy, Keepalived and acme.sh in one container. There is no systemd inside a container, so supervisord runs the processes and a small systemctl shim (docker/systemctl) translates the calls the app makes. HAProxy runs in master-worker mode and is reloaded with SIGUSR2, so Apply does not drop established connections.

Host networking is the intended mode — Keepalived's VRRP and the virtual IP need a real interface — and Keepalived needs NET_ADMIN, NET_BROADCAST and NET_RAW. One container per node.

One thing a container cannot do: restart a hung manager. On a systemd host WatchdogSec handles that; supervisord only restarts a process that exits. The image's HEALTHCHECK reports it, but something has to act on that. For a production cluster, the native install is the better fit.

→ Full Docker guide — images and tags, compose, networking modes, volumes, environment, health, logs, upgrading and limitations.

If the UI feels slow

Almost always it is waiting on another node, not on itself. A cluster member that is hung — accepting connections but not answering — is far worse than one that is cleanly down, because a refused connection fails instantly while a hung one has to time out.

Knob	Default	What it does
`HAM_PEER_CONNECT_TIMEOUT`	`3`	how long to wait for a node to accept a connection
`HAM_PEER_READ_TIMEOUT`	`5`	how long to wait for it to answer a health query
`HAM_PUSH_READ_TIMEOUT`	`90`	how long to wait for it to accept and apply a pushed configuration
`HAM_CLUSTER_POLL`	`15`	how often the watchdog collects every node's health in the background
`HAM_CLUSTER_MAX_AGE`	`60`	age at which a stale snapshot is collected inline instead
`HAM_THREADS`	`16`	waitress worker threads

Two things worth knowing:

Address peers by IP, not by name. DNS resolution happens before any of the timeouts above start counting, so a slow or unavailable resolver stalls a peer query for as long as /etc/resolv.conf allows. It is also the wrong dependency: the name may be published by the very cluster that is in trouble.

DNS is not cached on a stock Debian or Ubuntu server unless something is installed to do it: nsswitch.conf says hosts: files dns, and glibc has no cache of its own, so every lookup goes to the network. The resolver defaults are timeout:5 attempts:2, so a nameserver that does not answer costs ten seconds per lookup, every time — and a failed lookup is precisely the thing nothing can cache.

If you must use names, do one of these:
```
# 1. put the cluster in /etc/hosts -- checked before DNS, always instant
printf '10.0.0.1 proxy1\n10.0.0.2 proxy2\n' >> /etc/hosts

# 2. or fail fast instead of hanging
printf 'options timeout:1 attempts:1\n' >> /etc/resolv.conf

# 3. or install a caching resolver, and check it is being used
apt-get install -y systemd-resolved && resolvectl statistics
```
The Cluster page marks any peer that is addressed by name.
Apply waits for the push. With auto-sync on, Apply returns only once every peer has taken the configuration or timed out, so a wedged node can keep the button spinning for HAM_PUSH_READ_TIMEOUT. The rest of the UI stays responsive throughout; only that request is waiting.

Set them in the systemd unit (systemctl edit haproxy-manager):

[Service]
Environment=HAM_PEER_READ_TIMEOUT=3
Environment=HAM_THREADS=24

Environment overrides

HAM_DATA_DIR · HAM_CERT_DIR · HAM_HAPROXY_CFG · HAM_KEEPALIVED_CFG · HAM_ACME_HOME · HAM_ACME_SH · HAM_LISTEN · HAM_PORT · HAM_THREADS · HAM_LOG_FILE · HAM_DEBUG=1 (verbose logging) · HAM_STATS_SOCK · HAM_PEER_CONNECT_TIMEOUT · HAM_PEER_READ_TIMEOUT · HAM_PUSH_READ_TIMEOUT · HAM_CLUSTER_POLL · HAM_CLUSTER_MAX_AGE · HAM_WATCHDOG_PROBE_TIMEOUT · HAM_WATCHDOG_SELF_TIMEOUT · HAM_VERSION_URL · HAM_INSTALL_URL · HAM_DRY_RUN=1 (skip systemctl calls, for development).

The app also has a small maintenance CLI, used by the installer and the Docker entrypoint so neither has to reimplement password hashing:

python3 app.py show-admin                      # print the configured username
printf '%s' "$PW" | python3 app.py set-admin admin -    # set the login (stdin)

Note

This is a configuration front-end, not a fork of the OPNsense plugins — it borrows their structure and workflow but generates plain haproxy.cfg / keepalived.conf and drives acme.sh directly.

Name		Name	Last commit message	Last commit date
Latest commit History 137 Commits
.github		.github
deploy		deploy
docker		docker
docs		docs
ham		ham
packaging		packaging
static		static
tools		tools
.dockerignore		.dockerignore
.gitattributes		.gitattributes
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
VERSION		VERSION
app.py		app.py
docker-compose.yml		docker-compose.yml
install.sh		install.sh

Uh oh!

Folders and files

Latest commit

History

Repository files navigation