PR: #55 (feat/27-docker-compose)
Files: deploy/compose/docker-compose.yml volumes block
The compose file declares:
Alloy uses /var/lib/alloy/data as its WAL for buffering scraped series when the Grafana Cloud remote_write endpoint is unavailable (network partition, Grafana Cloud outage, expired API token). With no WAL retention limits configured, Alloy will buffer indefinitely until disk space is exhausted.
Combined with the missing log rotation noted in a separate issue, two concurrent disk-filling vectors exist on the same 40 GB Hetzner host. Disk exhaustion kills both containers and the signing process.
Suggested fix: Configure WAL retention bounds in alloy-config.alloy:
prometheus.remote_write "grafana_cloud" {
endpoint { ... }
wal {
truncate_frequency = "1h"
min_wal_time = "5m"
max_wal_time = "4h"
}
}
This bounds buffered data to at most 4 hours of metrics, limiting WAL disk usage to a few hundred MB under worst-case remote_write failure. Document the expected disk footprint in the deploy README section.
Refs #55
PR: #55 (feat/27-docker-compose)
Files: deploy/compose/docker-compose.yml volumes block
The compose file declares:
Alloy uses
/var/lib/alloy/dataas its WAL for buffering scraped series when the Grafana Cloudremote_writeendpoint is unavailable (network partition, Grafana Cloud outage, expired API token). With no WAL retention limits configured, Alloy will buffer indefinitely until disk space is exhausted.Combined with the missing log rotation noted in a separate issue, two concurrent disk-filling vectors exist on the same 40 GB Hetzner host. Disk exhaustion kills both containers and the signing process.
Suggested fix: Configure WAL retention bounds in
alloy-config.alloy:This bounds buffered data to at most 4 hours of metrics, limiting WAL disk usage to a few hundred MB under worst-case remote_write failure. Document the expected disk footprint in the deploy README section.
Refs #55