Skip to content

[Feature] Automated Multi-Service Deployment with Docker Compose and Production-Ready CI/CD Pipeline #98

@GaneshPatil7517

Description

@GaneshPatil7517

APIS currently has no Dockerfile or docker-compose.yml in the repository. The only Docker path available to users is a manually committed image (3akare/hyphaes-stable) built by hand inside a container this is not reproducible, not auditable, and breaks the open-source supply chain.

Additionally, the existing CI/CD workflows (.github/workflows/makefile.yml) have critical issues:

  1. Triggered on the add-license-1 branch instead of main - meaning CI never runs on the default branch
  2. Uses deprecated MongoDB 4.0 key import and bionic repositories (Ubuntu 18.04 EOL)
  3. No test execution the pipeline only does make build → make run → make stop with no validation
  4. MongoDB port inconsistency mongodb/start.sh binds to port 27018, but documentation states all services were standardized to 27017

This means new contributors cannot reliably build, test, or deploy the system, creating a significant barrier to adoption and contribution.

Proposed Solution
1. Multi-Stage Dockerfiles for Each Service Type

Dockerfile Services Covered Base Image
docker/java-services.Dockerfile apis-main, apis-ccc, apis-log, apis-web eclipse-temurin:11-jre
docker/python-services.Dockerfile apis-emulator, apis-main_controller, apis-tester python:3.10-slim
docker/service-center.Dockerfile apis-service_center (Django + SQLite) python:3.10-slim

2. docker-compose.yml Orchestration

services:
  mongodb:
    image: mongo:6.0
    ports: ["27017:27017"]
    healthcheck: ...

  apis-main-1:
    build: { context: ., dockerfile: docker/java-services.Dockerfile, target: apis-main }
    depends_on: { mongodb: { condition: service_healthy } }
    # ... (repeat for apis-main-2, 3, 4)

  apis-emulator:
    build: { context: ., dockerfile: docker/python-services.Dockerfile, target: emulator }
    ports: ["4390:4390"]

  apis-web:
    build: { context: ., dockerfile: docker/java-services.Dockerfile, target: apis-web }
    ports: ["4382:4382"]

  # ... remaining services

Single command to bring everything up: docker compose up --build

3. Fix CI/CD Pipeline

  • Retarget workflows to main branch with PR triggers
  • Upgrade MongoDB to 6.0+ with proper keyserver
  • Add service health checks use curl probes against ports 4382, 4390, 10000, 8000 after make run
  • Add integration smoke tests that validate energy exchange API responses
  • Standardize MongoDB port to 27017 across all config files

4. Resolve MongoDB Port Inconsistency
Audit and fix all configuration files referencing port 27018 to use 27017, matching the documentation.

Deliverables Checklist

  • Multi-stage Dockerfiles for Java services, Python services, and Service Center
  • docker-compose.yml with health checks, dependency ordering, and named volumes
  • .dockerignore to optimize build context
  • Fix CI workflow branch triggers (main + PRs)
  • Upgrade CI MongoDB version and keyserver
  • Add post-deployment health check step to CI
  • Standardize MongoDB port (27017) across all configs
  • Update docs/INSTALL_DOCKER.md with the new docker compose workflow
  • Add docs/DEVELOPMENT.md with local development setup instructions

Impact

Area Before After
New contributor onboarding Manual 10+ step process Single docker compose up
Build reproducibility Depends on pre-built image from one person Fully reproducible from source
CI/CD Runs on wrong branch, no tests Runs on all PRs with health validation
Supply chain security Opaque Docker image Auditable Dockerfiles in repo
OpenSSF Scorecard Partial Improved (reproducible builds, CI on default branch)

References

  1. OpenSSF Scorecard APIS
  2. LF Energy Project Guidelines
  3. Current Docker doc: docs/INSTALL_DOCKER.md
  4. Governance: GOVERNANCE.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions