Skip to content

[deploy] No HEALTHCHECK — charon reports healthy before RPC connection and metrics port are ready #290

@obchain

Description

@obchain

PR: #55 (feat/27-docker-compose)
Files: Dockerfile (no HEALTHCHECK directive); deploy/compose/docker-compose.yml depends_on block

The Dockerfile has no HEALTHCHECK directive. The compose alloy service specifies:

depends_on:
  - charon

Without condition: service_healthy, Docker resolves this dependency at container start (process fork), not at service readiness. The charon binary performs several async init steps before binding :9091:

  • WS connection to BSC RPC
  • Chain ID verification
  • Block listener backoff/reconnect loop start
  • Prometheus server bind

Alloy will start scraping charon:9091 at scrape_interval=15s immediately after compose startup. All scrape attempts before the Prometheus server binds will fail with connection refused, producing Alloy error logs and a gap in early-operation metric data.

More critically, docker compose ps and automated deploy scripts that check container status will report charon as healthy immediately after the process starts, before BSC RPC connectivity is established. This produces false confidence during rolling restarts and post-deploy verification.

Suggested fix: Add a HEALTHCHECK that probes the metrics endpoint:

HEALTHCHECK --interval=10s --timeout=5s --start-period=30s --retries=3 \
    CMD curl -sf http://localhost:9091/metrics > /dev/null || exit 1

Add curl to the runtime apt-get install line. Update compose:

depends_on:
  charon:
    condition: service_healthy

Refs #55

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinglayer:devopsCI / deploy / infra / telemetrypr-reviewFindings from PR review processpriority:p1-coreCore MVP scopestatus:readyScoped and ready to pick up

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions