Skip to content

v1.0.2: Fix Docker Smoke Test with Native Health Check

Choose a tag to compare

@denhamparry denhamparry released this 03 Dec 22:34
· 64 commits to main since this release
421a97e

What's Changed

This patch release fixes the root cause of the Docker workflow smoke test timing issue by using Docker's native HEALTHCHECK mechanism instead of host-side health checks.

Fixed

  • Docker workflow smoke test now uses native health check (#50)
    • Root cause identified: Docker network bridge initialization delay (13+ seconds in CI)
    • Solution: Use Docker's internal HEALTHCHECK (runs inside container)
    • Result: Avoids network bridge delay entirely
    • Expected timing: 7-15 seconds depending on system speed
    • Leverages existing: Dockerfile HEALTHCHECK definition

Root Cause Analysis

The Real Problem

The issue was NOT nginx startup (which is ready in ~1 second), but Docker's network bridge initialization taking 13+ seconds in CI environments.

Timeline from failed runs:

Container starts:  22:25:09.8038
Nginx READY:       22:25:09       ← Ready in < 1 second!
Health check #1:   22:25:14.9262  ❌ Fails (bridge not ready)
Health check #5:   22:25:22.9546  ❌ Still failing after 13s

Why Network Bridge Is Slow

Docker needs time to:

  1. Create iptables NAT rules for port mapping (8888:80)
  2. Initialize network namespace
  3. Set up bridge interface
  4. Establish port forwarding

This is common in CI with resource contention.

Technical Solution

Before (v1.0.1): Host-Side Checks

sleep 5
curl http://localhost:8888/health  # Waits for bridge (13+ seconds)

❌ Checks HOST→Container connectivity (waits for bridge)

After (v1.0.2): Container-Side Checks

docker inspect --format='{{.State.Health.Status}}' test-server
# ✅ Checks INSIDE container (no bridge delay)

Uses the existing Dockerfile HEALTHCHECK:

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD wget --no-verbose --tries=1 --spider http://localhost:${PORT}/health

Benefits

Aspect v1.0.1 (Retry Logic) v1.0.2 (Native Health Check)
Checks from ❌ Host side ✅ Inside container
Bridge delay ❌ Affected (13+ seconds) ✅ Avoided
Speed ⚠️ 15s fixed timeout ✅ 7-15s variable
Reliability ⚠️ Still fails sometimes ✅ Highly reliable
Uses existing HEALTHCHECK ❌ No ✅ Yes
Root cause ❌ Masked ✅ Solved

Expected Performance

  • Fast systems: ~7-10 seconds (5s start-period + 2-5s for check)
  • Slow systems: ~10-15 seconds (allows for slower initialization)
  • Broken containers: Fails after 30s with clear error and logs

Upgrade Notes

From v1.0.1

Simply update to v1.0.2. The smoke test will now:

  1. Wait for Docker's internal health check to pass
  2. Verify from host only AFTER container is healthy
  3. Show progress updates every 2 seconds
  4. Timeout after 30 seconds with logs if health check fails

Testing

After upgrading, the workflow should:

  • ✅ Pass consistently (100% success rate expected)
  • ✅ Complete smoke test in 7-15 seconds
  • ✅ Show clear progress feedback
  • ✅ Fail gracefully with logs if container is broken

Related Issues

  • #47 - Original smoke test timing issue
  • #50 - Root cause fix (this release)
  • #49 - First attempt with retry logic (replaced)

Full Changelog

v1.0.1...v1.0.2: v1.0.1...v1.0.2

Version History

v1.0.0 (2025-12-03) - Initial public release
  ↓
v1.0.1 (2025-12-03) - Added retry logic (masked root cause)
  ↓
v1.0.2 (2025-12-03) - Fixed root cause (network bridge delay)

🤖 Generated with Claude Code