# CI/CD Pipelines & Orchestration

This module covers the automation scripts, GitHub Action workflow definitions, and cross-registry image synchronization tooling of the LabNow Docker ecosystem.

---

## 1. Orchestration Script (`tool.sh`)

Located at the repository root, `tool.sh` coordinates local and automated builds. It configures registries, tags, and helper functions.

### Namespace & Branch-Based Tagging Style
The script determines the registry namespace and tags dynamically based on the current Git branch:
- **`CI_PROJECT_BRANCH`**: Defaults to `$GITHUB_HEAD_REF` or fallback to `"main"`.
- **`NAMESPACE_SUFFIX`**:
  - If the branch is `main`, the suffix is empty: `""`.
  - Otherwise, it is formatted as: `"0" + {first segment of branch before "/"}` (e.g. `feat/add-login` -> suffix is `0feat`).
- **`CI_PROJECT_NAMESPACE`**: Concatenates project namespace with the branch suffix (e.g. `LabNow-ai` -> `LabNow-ai0feat`).
- **`TAG_SUFFIX`**: Short hash suffix `-$(git rev-parse --short HEAD)`.

### Core Build Functions
- **`build_image <img_name> <tag> <dockerfile_path> [build-args]`**:
  Builds the image using `--compress --force-rm=true`. It tags the output as `${REGISTRY_DST}/${IMG_NAMESPACE}/${img_name}:${tag}` and creates a timestamped version `Y.m.d.HM${TAG_SUFFIX}`.
- **`build_image_no_tag`**: Builds the image without adding the timestamped version tag (typically used for intermediate build stages).
- **`alias_image <img1> <tag1> <img2> <tag2>`**: Retags a compiled local image to another repository name or alias.
- **`push_image [keyword]`**: Logs into the destination registry via stdin, selects local images matching `[keyword]`, and pushes them.
- **`clear_images`**: Clears historical local images to prevent Docker daemon disk starvation.

---

## 2. GitHub Actions Workflows

Automated builds are split into two workflows located under `.github/workflows/`:

### A. Core Workflow (`build-docker.yml`)
- **Trigger**: Pushes or PRs targeting `main` branch (ignoring changes solely in markdown files).
- **Jobs**:
  - `job-base`: Compiles the core `atom` and `base` images.
  - `job-python`/`job-core`/`job-py-std`/`job-node`/`job-rust`/`job-jdk` etc.: Depend on `job-base` and run concurrently to build their specific language environments.
  - `job-docker_kit`: Builds the `docker-kit` image containing `image-syncer`.
  - `sync_images`: Runs final registry migrations once dependencies are ready.

### B. GPU Workflow (`build-docker-gpu.yml`)
- **Trigger**: Same trigger conditions.
- **Jobs**:
  - `job-cuda_128` / `job-cuda_126`: Compiles CUDA-specific atom wrappers, builds conda-base wrappers, and builds final cuda base images.
  - `job-torch_cuda128` / `job-tf2` / `job-paddle_cuda126` etc.: Depend on the respective CUDA jobs and compile python profiles for Deep Learning stacks.
  - `job-core-cuda`: Builds the full stack CUDA environment.

---

## 3. Registry Mirroring & Syncing (`docker-kit`)

Built using `docker_docker_kit/docker-kit.Dockerfile`, the `docker-kit` image packages the tools required to mirror built images across registries.

### Key Tools in `docker-kit`
- **`yq`**: Installed via `setup_yq` for YAML files processing.
- **Docker Compose**: Sourced from `script-setup-docker.sh` -> `setup_docker_compose`.
- **image-syncer**: Sourced from `setup_docker_syncer` and mapped to `/opt/utils/image-syncer/`.

### Synchronizer Scripts (`image-syncer/`)
- **`run_jobs.py`**: A python script that reads registry authentication credentials from `auth.json`, parses workflow files to identify target images, and starts mirroring processes.
- **`run_sync.py`**: Executes parallel `image-syncer` shell invocations to migrate images from the primary destination registry to secondary mirror registries (e.g. Quay.io to regional mirrors).