Add CI/CD Verification for Complete Build Pipeline

[jonpspri]

Problem Statement

The project has a well-defined complete build pipeline that should cleanly produce a production-ready Docker image:

make venv install install-dev
make autoflake isort black pre-commit
make doctest test lint-web flake8 bandit interrogate pylint verify 
make smoketest
make docker-prod

However, there is currently no automated verification that this end-to-end sequence executes successfully. While individual components have CI checks (pytest, docker-image, lint workflows), the complete integrated pipeline is not tested in CI, creating a risk that the full build sequence could break without detection.

Current State

Existing CI Coverage:

• .github/workflows/pytest.yml - Tests & coverage only
• .github/workflows/docker-image.yml - Docker build with security scans (Hadolint, Dockle, Grype, Trivy)
• .github/workflows/lint.yml, lint-web.yml, bandit.yml - Individual quality checks
• .pre-commit-config.yaml - Pre-commit hooks for security, formatting, and validation

Gaps:

• No CI workflow that runs the complete sequence from setup through Docker build
• No pre-commit hook to verify the full pipeline locally before push
• Individual checks may pass while the integrated sequence fails

Proposed Solution

1. Add GitHub Actions Workflow

Create .github/workflows/full-build-pipeline.yml to verify the complete build sequence:

name: Full Build Pipeline

on:
  pull_request:
    branches: ["main"]
  push:
    branches: ["main"]
  schedule:
    - cron: '0 6 * * 1'  # Weekly Monday 06:00 UTC

jobs:
  full-pipeline:
    name: Complete Build Pipeline
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      
      - name: Setup Python
        uses: actions/setup-python@v6
        with:
          python-version: '3.11'
      
      - name: Install uv
        uses: astral-sh/setup-uv@v5
        
      - name: Environment Setup
        run: |
          make venv install install-dev
          
      - name: Code Quality & Formatting
        run: make autoflake isort black pre-commit
        
      - name: Comprehensive Testing & Verification
        run: make doctest test lint-web flake8 bandit interrogate pylint verify
        
      - name: Smoke Tests
        run: make smoketest
        
      - name: Production Docker Build
        run: make docker-prod

2. Add Pre-Commit Hook (Optional)

Add a local pre-commit hook to .pre-commit-config.yaml that runs a fast subset of the pipeline:

- repo: local
  hooks:
    - id: verify-build-pipeline
      name: 🏗️ Verify Build Pipeline Subset
      description: Runs quick verification that core build steps work
      entry: bash -c 'make autoflake isort black && make doctest test'
      language: system
      pass_filenames: false
      always_run: true
      stages: [manual]  # Run only with --hook-stage manual to avoid slowing down commits

Acceptance Criteria

• GitHub Actions workflow runs the complete build pipeline sequence
• Workflow triggers on PR, push to main, and weekly schedule
• Failures in any step fail the entire workflow
• (Optional) Pre-commit hook available for local validation with pre-commit run --hook-stage manual --all-files
• Documentation updated to reference the CI workflow

Benefits

• Early Detection: Catch integration issues before they reach production
• Build Reproducibility: Verify that the documented build process works consistently
• Developer Confidence: Ensure the complete toolchain works together
• Documentation Validation: Serves as executable documentation of the build process

Additional Context

• Current branch: chore/github-build-action-updates suggests this work may already be in progress
• The make docker-prod target should be the final validation that a production-ready artifact can be created
• Consider adding build artifacts upload if needed for debugging failed builds