From 2b8c31dc9de03729d936c7dc525b365a19810611 Mon Sep 17 00:00:00 2001 From: Miguel Martinez Date: Tue, 4 Nov 2025 00:36:45 +0100 Subject: [PATCH 1/2] chore: replace runbooks for claude skill Signed-off-by: Miguel Martinez --- .claude/skills/upgrading-chart/SKILL.md | 132 +++++++ .../upgrading-chart/chart-upgrade-process.md | 148 ++++++++ .../skills/upgrading-chart/files-modified.md | 68 ++++ .../upgrading-chart/image-upgrade-process.md | 54 +++ .claude/skills/upgrading-golang/SKILL.md | 90 +++++ .../upgrading-golang/files-to-update.md | 56 +++ CLAUDE.md | 2 +- docs/runbooks/chart-upgrade-process.md | 354 ------------------ docs/runbooks/golang-upgrade-process.md | 113 ------ 9 files changed, 549 insertions(+), 468 deletions(-) create mode 100644 .claude/skills/upgrading-chart/SKILL.md create mode 100644 .claude/skills/upgrading-chart/chart-upgrade-process.md create mode 100644 .claude/skills/upgrading-chart/files-modified.md create mode 100644 .claude/skills/upgrading-chart/image-upgrade-process.md create mode 100644 .claude/skills/upgrading-golang/SKILL.md create mode 100644 .claude/skills/upgrading-golang/files-to-update.md delete mode 100644 docs/runbooks/chart-upgrade-process.md delete mode 100644 docs/runbooks/golang-upgrade-process.md diff --git a/.claude/skills/upgrading-chart/SKILL.md b/.claude/skills/upgrading-chart/SKILL.md new file mode 100644 index 000000000..91a32f95d --- /dev/null +++ b/.claude/skills/upgrading-chart/SKILL.md @@ -0,0 +1,132 @@ +--- +name: upgrading-chart +description: Upgrades Helm chart dependencies (PostgreSQL, Vault) in the Chainloop project, including vendorized charts, container images, and CI/CD workflows. Use when the user mentions upgrading Helm charts, Bitnami dependencies, PostgreSQL chart, or Vault chart. CRITICAL - Major version upgrades are FORBIDDEN and must be escalated. +--- + +# Upgrading Helm Chart Dependencies + +This skill automates the upgrade process for Helm chart dependencies in the Chainloop project. Supports PostgreSQL and Vault (both Bitnami charts). + +## CRITICAL RESTRICTIONS + +**Version Upgrade Rules**: +- Patch upgrades (1.2.3 → 1.2.4): ALLOWED +- Minor upgrades (1.2.x → 1.3.x): ALLOWED +- Major upgrades (1.x.x → 2.x.x): **FORBIDDEN - STOP IMMEDIATELY** + +**MANDATORY**: If major version upgrade is detected, STOP the process and inform the user that manual review is required. + +## Upgrade Types + +The skill supports two upgrade types: + +1. **Specific Image Upgrade**: Update container image to specific version (chart unchanged) +2. **Chart Minor Version Upgrade**: Update chart to latest minor version (may include image updates) + +**IMPORTANT**: Container images are ONLY updated as part of chart upgrades, never independently (unless Type 1). + +## Process + +### 1. Identify Upgrade Type + +Ask the user which type of upgrade they want: +- Type 1: Specific image version upgrade +- Type 2: Latest minor chart version upgrade + +Also ask which chart: `postgresql` or `vault` + +### 2. Pre-Upgrade Validation + +Check current state: +```bash +cat deployment/chainloop/charts//Chart.yaml | grep "^version:" +cat deployment/chainloop/charts//Chart.yaml | grep "^appVersion:" +``` + +### 3. Version Compatibility Check + +For any version change, validate that major version remains the same: +```bash +CURRENT_MAJOR=$(echo "$CURRENT_VERSION" | cut -d. -f1) +TARGET_MAJOR=$(echo "$TARGET_VERSION" | cut -d. -f1) + +if [ "$CURRENT_MAJOR" != "$TARGET_MAJOR" ]; then + echo "FORBIDDEN: Major version upgrade detected" + exit 1 +fi +``` + +If major version upgrade detected, STOP and escalate. + +## Type 1: Specific Image Upgrade + +See [image-upgrade-process.md](image-upgrade-process.md) for detailed steps. + +**Summary**: +1. Locate target container image in [Bitnami Containers](https://github.com/bitnami/containers) +2. Find commit with release message pattern +3. Extract APP_VERSION from Dockerfile +4. Update `deployment/charts//Chart.yaml` appVersion +5. Update `.github/workflows/build_external_container_images.yaml` commit hash + +## Type 2: Chart Minor Version Upgrade + +See [chart-upgrade-process.md](chart-upgrade-process.md) for detailed steps. + +**Summary**: +1. Locate target chart version in [Bitnami Charts](https://github.com/bitnami/charts) CHANGELOG.md +2. Validate minor version upgrade only +3. Download and extract target chart +4. Check for image changes (compare Chart.yaml) +5. If images changed, update container image references +6. Vendorize chart update (copy files) +7. Update dependencies in correct order +8. Update main chart dependency version +9. Clean up temporary files + +## Verification + +After any upgrade type, run: +```bash +# Lint charts +helm lint deployment/charts/ +helm lint deployment/chainloop + +# Template validation +helm template deployment/charts/ +helm template deployment/chainloop + +# Local testing +cd devel && docker compose up + +# Verify image consistency +grep -r "appVersion\|image.*tag" deployment/charts// +``` + +## Files Modified + +See [files-modified.md](files-modified.md) for complete list. + +## Troubleshooting + +Common issues: +- **Image Version Mismatch**: Verify APP_VERSION matches Chart.yaml appVersion +- **Build Failures**: Check commit reference in build workflow +- **Dependency Conflicts**: Verify dependencies updated in correct order (vendorized first, then main chart) + +## Rollback + +If issues occur: +```bash +git checkout HEAD -- deployment/ +find deployment/ -name "Chart.lock" -delete +cd deployment/chainloop && helm dependency build +cd ../../devel && docker compose down && docker compose up +``` + +## Important Notes + +- Dex is self-managed and follows a separate process (not covered by this skill) +- Always use commit hashes for reproducibility +- Dependencies must be updated in correct order: vendorized chart first, then main chart +- Container images are found in Bitnami Containers repo, charts in Bitnami Charts repo diff --git a/.claude/skills/upgrading-chart/chart-upgrade-process.md b/.claude/skills/upgrading-chart/chart-upgrade-process.md new file mode 100644 index 000000000..042f0bf69 --- /dev/null +++ b/.claude/skills/upgrading-chart/chart-upgrade-process.md @@ -0,0 +1,148 @@ +# Chart Minor Version Upgrade Process (Type 2) + +This process upgrades a Helm chart to the latest minor version, potentially including image updates. + +## Step 1: Locate Target Chart Version + +1. Navigate to [Bitnami Charts](https://github.com/bitnami/charts) +2. Open `bitnami//CHANGELOG.md` +3. Find latest minor version (ensure no major version change) +4. Note target chart version + +## Step 2: Version Validation + +**MANDATORY**: Verify minor upgrade only: +```bash +CURRENT_CHART_VERSION="" +TARGET_CHART_VERSION="" + +CURRENT_MAJOR=$(echo "$CURRENT_CHART_VERSION" | cut -d. -f1) +TARGET_MAJOR=$(echo "$TARGET_CHART_VERSION" | cut -d. -f1) + +if [ "$CURRENT_MAJOR" != "$TARGET_MAJOR" ]; then + echo "FORBIDDEN: Major version upgrade detected" + exit 1 +fi +``` + +## Step 3: Download and Extract Target Chart + +```bash +# Pull chart to temporary location +helm pull bitnami/ --version --untar --untardir /tmp + +# Examine structure +ls -la /tmp// +``` + +## Step 4: Check for Image Changes + +```bash +# Compare current vs target chart +diff deployment/charts//Chart.yaml /tmp//Chart.yaml + +# Look for changes in: +# - appVersion field +# - images section (if present) +# - dependencies +``` + +## Step 5: Update Container Images (if changed) + +**Execute only if images changed in target chart**: + +1. **Locate new image versions**: + - Pattern format: `---r` + - Example: `15.3.0-debian-12-r1` + +2. **Get APP_VERSION from Dockerfile**: + - Navigate to `bitnami/containers///-/Dockerfile` + - Extract `APP_VERSION` environment variable + +3. **Update build configuration**: + ```bash + # Update commit hash in build workflow + vi .github/workflows/build_external_container_images.yaml + ``` + +## Step 6: Vendorize Chart Update + +```bash +# Replace vendorized chart with new version +cp -r /tmp//* deployment/charts// + +# Update Chart.yaml if images changed +vi deployment/charts//Chart.yaml +# Set appVersion to APP_VERSION from Bitnami Containers + +# Update values.yaml if needed +# Replace docker.io/bitnami/* with Chainloop registry paths +vi deployment/charts//values.yaml +``` + +## Step 7: Update Dependencies (CRITICAL ORDER) + +**Dependencies must be updated in this specific order**: + +```bash +# 1. Update vendorized chart dependencies FIRST +cd deployment/charts/ +helm dependency update +helm dependency build + +# 2. Update main chart dependency version +cd ../../chainloop +vi Chart.yaml # Update dependency version to match vendorized chart + +# 3. Update main chart dependencies +helm dependency update +helm dependency build + +cd ../.. +``` + +## Step 8: Clean Up + +```bash +# Remove temporary files +rm -rf /tmp/ + +# Verify working directory +git status +``` + +## Step 9: Verification + +```bash +# Lint charts +helm lint deployment/charts/ +helm lint deployment/chainloop + +# Template validation +helm template deployment/charts/ +helm template deployment/chainloop + +# Local testing +cd devel && docker compose up + +# Verify image consistency +grep -r "appVersion\|image.*tag" deployment/charts// +grep -r "" .github/workflows/build_external_container_images.yaml +``` + +## Files Modified + +- `deployment/charts//Chart.yaml` - Chart version and appVersion +- `deployment/charts//values.yaml` - Image references +- `deployment/charts//Chart.lock` - Dependency lock (regenerated) +- `deployment/charts//templates/*` - All chart templates (vendorized) +- `deployment/chainloop/Chart.yaml` - Main chart dependency version +- `deployment/chainloop/Chart.lock` - Main dependency lock (regenerated) +- `.github/workflows/build_external_container_images.yaml` - Image build references (if images changed) + +## Important Notes + +- Always vendorize first, then update main chart dependencies +- Chart.lock files are regenerated by `helm dependency build` +- Image registry paths must use Chainloop registry (not docker.io/bitnami) +- Commit hashes ensure reproducible builds diff --git a/.claude/skills/upgrading-chart/files-modified.md b/.claude/skills/upgrading-chart/files-modified.md new file mode 100644 index 000000000..c0d7b1fbb --- /dev/null +++ b/.claude/skills/upgrading-chart/files-modified.md @@ -0,0 +1,68 @@ +# Files Modified During Chart Upgrades + +This reference lists all files that may be modified during Helm chart dependency upgrades. + +## Type 1: Specific Image Upgrade + +### Chart Files +- `deployment/charts//Chart.yaml` - appVersion only + +### CI/CD Configuration +- `.github/workflows/build_external_container_images.yaml` - commit hash reference + +**Total files**: 2 files + +## Type 2: Chart Minor Version Upgrade + +### Vendorized Chart Files +- `deployment/charts//Chart.yaml` - Chart version and appVersion +- `deployment/charts//values.yaml` - Default configuration and image references +- `deployment/charts//Chart.lock` - Dependency lock file (regenerated) +- `deployment/charts//templates/*` - All Helm template files +- `deployment/charts//README.md` - Chart documentation +- Other chart files as needed + +### Main Chart Files +- `deployment/chainloop/Chart.yaml` - Dependency version reference +- `deployment/chainloop/Chart.lock` - Main chart dependency lock (regenerated) + +### CI/CD Configuration (if images changed) +- `.github/workflows/build_external_container_images.yaml` - Image build commit references + +**Total files**: Variable (all vendorized chart files + 2-3 main files) + +## Common Chart Names + +- `postgresql` - PostgreSQL database +- `vault` - HashiCorp Vault for secrets + +## File Locations Summary + +``` +deployment/ +├── charts/ +│ ├── postgresql/ # Vendorized PostgreSQL chart +│ │ ├── Chart.yaml # Chart metadata +│ │ ├── Chart.lock # Dependencies (regenerated) +│ │ ├── values.yaml # Configuration +│ │ └── templates/ # Helm templates +│ └── vault/ # Vendorized Vault chart +│ ├── Chart.yaml +│ ├── Chart.lock +│ ├── values.yaml +│ └── templates/ +└── chainloop/ # Main Chainloop chart + ├── Chart.yaml # Main chart with dependencies + └── Chart.lock # Main dependencies (regenerated) + +.github/ +└── workflows/ + └── build_external_container_images.yaml # Image build config +``` + +## Key Points + +- `.lock` files are always regenerated by `helm dependency build` +- Template files are completely replaced during vendorization +- Image registry paths should reference Chainloop registry, not `docker.io/bitnami` +- Build workflow only modified if container images change diff --git a/.claude/skills/upgrading-chart/image-upgrade-process.md b/.claude/skills/upgrading-chart/image-upgrade-process.md new file mode 100644 index 000000000..34cc1d791 --- /dev/null +++ b/.claude/skills/upgrading-chart/image-upgrade-process.md @@ -0,0 +1,54 @@ +# Specific Image Upgrade Process (Type 1) + +This process upgrades a container image to a specific version without changing the chart version. + +## Step 1: Locate Target Container Image + +1. Navigate to [Bitnami Containers](https://github.com/bitnami/containers) +2. Find image folder: `bitnami/` +3. Check commit history: `https://github.com/bitnami/containers/commits/main/bitnami/` +4. Find commit with message pattern: `Release ----r` + - Example: `Release postgresql-15.3.0-debian-12-r1` +5. Note the commit hash +6. Open the Dockerfile in that commit +7. Extract the `APP_VERSION` environment variable value + +**Example Dockerfile location**: +``` +bitnami/containers///-/Dockerfile +``` + +## Step 2: Update Chart appVersion + +Edit `deployment/charts//Chart.yaml`: + +```yaml +# Update only the appVersion field to match APP_VERSION from Dockerfile +appVersion: "X.X.X" + +# Keep chart version unchanged +version: "Y.Y.Y" +``` + +## Step 3: Update Build Configuration + +Edit `.github/workflows/build_external_container_images.yaml`: + +Update the commit hash reference for the specific image to point to the commit identified in Step 1. + +## Step 4: Verify Changes + +```bash +# Check consistency +grep "appVersion" deployment/charts//Chart.yaml +grep "" .github/workflows/build_external_container_images.yaml + +# Lint and test +helm lint deployment/charts/ +cd devel && docker compose up +``` + +## Files Modified + +- `deployment/charts//Chart.yaml` - appVersion only +- `.github/workflows/build_external_container_images.yaml` - commit hash diff --git a/.claude/skills/upgrading-golang/SKILL.md b/.claude/skills/upgrading-golang/SKILL.md new file mode 100644 index 000000000..5f260b810 --- /dev/null +++ b/.claude/skills/upgrading-golang/SKILL.md @@ -0,0 +1,90 @@ +--- +name: upgrading-golang +description: Upgrades Go version across the entire Chainloop codebase including source files, Docker images, CI/CD workflows, and documentation. Use when the user mentions upgrading Go, golang version, or updating Go compiler version. +--- + +# Upgrading Golang Version + +This skill automates the comprehensive Go version upgrade process across all components of the Chainloop project. + +## Process + +### 1. Confirm Target Version + +Ask the user for the target Go version (e.g., "1.25.0"). + +### 2. Get Docker Image Digest + +Pull the official golang Docker image and extract its SHA256 digest: + +```bash +docker pull golang:X.XX.X +``` + +Extract the SHA256 digest from the output (format: `sha256:abc123...`). + +### 3. Update Source Code + +Update the `go` directive in: +- `./go.mod` + +**IMPORTANT**: Do NOT update `./extras/dagger/go.mod` per project policy. + +Pattern to replace: +```go +go X.XX.X +``` + +### 4. Update Docker Images + +Update all Dockerfiles with the new version and SHA256 digest. See [files-to-update.md](files-to-update.md) for the complete list. + +Pattern to replace: +```dockerfile +FROM golang:X.XX.X@sha256:OLD_DIGEST AS builder +``` + +With: +```dockerfile +FROM golang:X.XX.X@sha256:NEW_DIGEST AS builder +``` + +### 5. Update GitHub Actions + +Update `go-version` in all workflow YAML files. See [files-to-update.md](files-to-update.md) for the complete list. + +Pattern to replace: +```yaml +go-version: "X.XX.X" +``` + +### 6. Update Documentation + +Update the version reference in `./CLAUDE.md` under "Key Technologies": +```markdown +- **Language**: Go X.XX.X. To know how to upgrade go version, see docs/runbooks +``` + +### 7. Verify Changes + +Run verification commands: +```bash +make test +make lint +``` + +If errors occur, address them before completing the upgrade. + +### 8. Final Checks + +- Ensure all license headers are updated (2024 → 2024-2025 or add current year) +- Run `buf format -w` if any proto files were affected +- Run `wire ./...` if any constructor dependencies changed +- Verify `go.mod` changes with `go mod tidy` + +## Important Notes + +- Always use SHA256 digests for Docker images for security and reproducibility +- The dagger module (`./extras/dagger/go.mod`) must NOT be updated +- Test thoroughly as Go upgrades can introduce breaking changes +- Multiple components use Go: CLI, Control Plane, and Artifact CAS diff --git a/.claude/skills/upgrading-golang/files-to-update.md b/.claude/skills/upgrading-golang/files-to-update.md new file mode 100644 index 000000000..431e10cb8 --- /dev/null +++ b/.claude/skills/upgrading-golang/files-to-update.md @@ -0,0 +1,56 @@ +# Files to Update During Go Version Upgrade + +This reference lists all files that must be updated when upgrading Go versions. + +## Source Code + +### Go Modules +- `./go.mod` - Main project go.mod + +**DO NOT UPDATE**: +- `./extras/dagger/go.mod` - Dagger module (per project policy) + +## Docker Images + +### Dockerfiles +- `./app/artifact-cas/Dockerfile` +- `./app/artifact-cas/Dockerfile.goreleaser` +- `./app/controlplane/Dockerfile` +- `./app/controlplane/Dockerfile.goreleaser` +- `./app/cli/Dockerfile.goreleaser` + +Update pattern in all: +```dockerfile +FROM golang:X.XX.X@sha256:DIGEST AS builder +``` + +## CI/CD Workflows + +### GitHub Actions +- `./.github/workflows/lint.yml` +- `./.github/workflows/test.yml` +- `./.github/workflows/release.yaml` +- `./.github/workflows/codeql.yml` +- `./docs/examples/ci-workflows/github.yaml` + +Update pattern in all: +```yaml +go-version: "X.XX.X" +``` + +## Documentation + +### Project Documentation +- `./CLAUDE.md` - Update "Key Technologies" section: + ```markdown + - **Language**: Go X.XX.X. To know how to upgrade go version, see docs/runbooks + ``` + +## Summary + +**Total files to update**: 13 files +- 1 go.mod file +- 5 Dockerfiles +- 5 GitHub Actions workflows +- 1 example workflow +- 1 documentation file diff --git a/CLAUDE.md b/CLAUDE.md index c99e14b15..88ddaa4bc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -135,7 +135,7 @@ make migration_lint # Lint migration files ## Key Technologies -- **Language**: Go 1.25.0. To know how to upgrade go version, see docs/runbooks +- **Language**: Go 1.25.0 - **API**: gRPC with HTTP/JSON gateway, Protocol Buffers with buf - **Database**: PostgreSQL with Ent ORM, Atlas for migrations - **Authentication**: OIDC, JWT tokens diff --git a/docs/runbooks/chart-upgrade-process.md b/docs/runbooks/chart-upgrade-process.md deleted file mode 100644 index 1bdbb26fa..000000000 --- a/docs/runbooks/chart-upgrade-process.md +++ /dev/null @@ -1,354 +0,0 @@ -# Chainloop Chart Dependencies Upgrade Runbook - -## Overview - -This runbook provides step-by-step instructions for upgrading Helm chart dependencies in the Chainloop project. Follow this process to ensure safe, consistent upgrades while maintaining system stability. - -### Dependencies Managed -- **PostgreSQL**: Database backend (Bitnami chart) -- **Vault**: Secret storage (Bitnami chart) -- **Dex**: OIDC provider *(self-managed, separate process)* - ---- - -## CRITICAL RESTRICTIONS - -### Version Upgrade Rules -| Upgrade Type | Example | Status | -|--------------|---------|--------| -| Patch upgrade | `1.2.3` → `1.2.4` | ALLOWED | -| Minor upgrade | `1.2.x` → `1.3.x` | ALLOWED | -| Major upgrade | `1.x.x` → `2.x.x` | **FORBIDDEN - STOP IMMEDIATELY** | - -**MANDATORY CHECK**: Any major version upgrade attempt must **STOP** the process and escalate for manual review. - ---- - -## Upgrade Types - -### Chart Version Upgrade Only -- **Purpose**: Upgrade chart to latest minor version -- **Scope**: Update chart version and potentially images -- **Use Case**: Feature updates, security patches, regular maintenance -- **Rule**: Container images are **ONLY** updated as part of chart upgrades, never independently - ---- - -## Pre-Upgrade Validation - -### Step 1: Identify Current State -```bash -# Check current chart version -cat deployment/chainloop/charts//Chart.yaml | grep "^version:" - -# Check current app version -cat deployment/chainloop/charts//Chart.yaml | grep "^appVersion:" -``` - -### Step 2: Version Compatibility Check -```bash -# Example validation -CURRENT_MAJOR=$(echo "$CURRENT_VERSION" | cut -d. -f1) -TARGET_MAJOR=$(echo "$TARGET_VERSION" | cut -d. -f1) - -if [ "$CURRENT_MAJOR" != "$TARGET_MAJOR" ]; then - echo "FORBIDDEN: Major version upgrade detected" - echo "Current: $CURRENT_VERSION → Target: $TARGET_VERSION" - exit 1 -fi -``` - -### Step 3: Stop Conditions -**STOP IMMEDIATELY if**: -- Major version change detected -- Breaking changes require manual intervention -- Dependencies conflict - ---- - -## Reference Resources - -### Bitnami Charts Repository Structure -``` -bitnami/charts/ -└── / - ├── Chart.yaml # Chart metadata and version - ├── CHANGELOG.md # Version history and changes - ├── values.yaml # Default configuration - └── templates/ # Helm templates -``` - -### Bitnami Containers Repository Structure -``` -bitnami/containers/ -└── / - └── / - └── -/ - ├── Dockerfile # Contains APP_VERSION - └── ... -``` - -### Finding Resources -| Resource | Location | Purpose | -|----------|----------|---------| -| Chart versions | [Bitnami Charts](https://github.com/bitnami/charts) + `CHANGELOG.md` | Find available chart versions | -| Container images | [Bitnami Containers](https://github.com/bitnami/containers) + commit history | Find image versions and commit hashes | -| Release tags | Commit messages: `Release ----r` | Identify specific releases | - ---- - -## Type 1: Specific Image Upgrade Process - -### Step 1: Locate Target Container Image -1. Navigate to [Bitnami Containers](https://github.com/bitnami/containers) -2. Find image folder: `bitnami/` -3. Check commit history: `https://github.com/bitnami/containers/commits/main/bitnami/` -4. Find commit with message: `Release ----r` -5. Note the commit hash and APP_VERSION from Dockerfile - -### Step 2: Update Image References -```bash -# Edit Chart.yaml - update appVersion only -vi deployment/charts//Chart.yaml - -# Update appVersion field to match container's APP_VERSION -# Keep chart version unchanged -``` - -### Step 3: Update Build Configuration -```bash -# Edit build workflow to reference the correct commit -vi .github/workflows/build_external_container_images.yaml - -# Update commit hash for the specific image -``` - ---- - -## Type 2: Latest Minor Version Chart Upgrade Process - -### Step 1: Locate Target Chart Version -1. Navigate to [Bitnami Charts](https://github.com/bitnami/charts) -2. Open `bitnami//CHANGELOG.md` -3. Find latest minor version (ensure no major version change) -4. Note target chart version - -### Step 2: Version Validation -```bash -# MANDATORY: Verify minor upgrade only -CURRENT_CHART_VERSION="" -TARGET_CHART_VERSION="" - -# Validate major version compatibility -# If major version differs, STOP PROCESS -``` - -### Step 3: Download and Extract Target Chart -```bash -# Pull chart to temporary location -helm pull bitnami/ --version --untar --untardir /tmp - -# Examine the new chart structure -ls -la /tmp// -``` - -### Step 4: Check for Image Changes -```bash -# Compare current vs target chart images -diff deployment/charts//Chart.yaml /tmp//Chart.yaml - -# Look for changes in: -# - appVersion field -# - images section (if present) -# - dependencies -``` - -### Step 5: Update Container Images (if changed) -**Execute only if images changed in target chart**: - -1. **Locate new image versions**: - ```bash - # For each changed image, find in Bitnami Containers - # Pattern: ---r - # Example: 15.3.0-debian-12-r1 - ``` - -2. **Get APP_VERSION from Dockerfile**: - ```bash - # Navigate to bitnami/containers///-/ - # Extract APP_VERSION from Dockerfile - ``` - -3. **Update build configuration**: - ```bash - # Update commit hash in build workflow - vi .github/workflows/build_external_container_images.yaml - ``` - -### Step 6: Vendorize Chart Update -```bash -# Replace vendorized chart with new version -cp -r /tmp//* deployment/charts// - -# Update Chart.yaml if images changed -vi deployment/charts//Chart.yaml -# Set appVersion to APP_VERSION from Bitnami Containers - -# Update values.yaml if needed -# Replace docker.io/bitnami/* with Chainloop registry paths -vi deployment/charts//values.yaml -``` - -### Step 7: Update Dependencies -```bash -# CRITICAL: Update dependencies in correct order - -# 1. Update vendorized chart dependencies first -cd deployment/charts/ -helm dependency update -helm dependency build - -# 2. Update main chart dependency version -cd ../../chainloop -vi Chart.yaml # Update dependency version to match vendorized chart - -# 3. Update main chart dependencies -helm dependency update -helm dependency build - -cd ../.. -``` - -### Step 8: Clean Up -```bash -# Remove temporary files -rm -rf /tmp/ - -# Verify working directory is clean -git status -``` - ---- - -## Verification & Testing - -### Local Verification -```bash -# 1. Lint charts -helm lint deployment/charts/ -helm lint deployment/chainloop - -# 2. Template validation -helm template deployment/charts/ -helm template deployment/chainloop - -# 3. Local testing -cd devel && docker compose up - -# 4. Integration testing -# Run your specific integration test suite -``` - -### Image Consistency Checks -```bash -# Verify consistency across: -# - Chart.yaml appVersion -# - values.yaml image tags -# - Build workflow commit references - -grep -r "appVersion\|image.*tag" deployment/charts// -grep -r "" .github/workflows/build_external_container_images.yaml -``` - ---- - -## Files Modified - -### Chart Files -- `deployment/charts//Chart.yaml` - Chart version, appVersion -- `deployment/charts//values.yaml` - Image references -- `deployment/charts//Chart.lock` - Dependency lock -- `deployment/chainloop/Chart.yaml` - Main chart dependencies -- `deployment/chainloop/Chart.lock` - Main dependency lock - -### CI/CD Configuration -- `.github/workflows/build_external_container_images.yaml` - Image build references - ---- - -## Troubleshooting - -| Issue | Symptoms | Solution | -|-------|----------|----------| -| **Image Version Mismatch** | Services fail to start | Verify APP_VERSION matches Chart.yaml appVersion | -| **Build Failures** | CI/CD fails to build images | Check commit reference contains required image versions | -| **Image Pull Failures** | Deployment fails | Ensure all image tags are consistent and updated | -| **Dependency Conflicts** | Helm dependency errors | Check compatibility between chart versions | -| **Missing Container Images** | Image not found errors | Check Bitnami Containers history for renamed/removed images | - -### Debug Commands -```bash -# Check image references -grep -r "image:" deployment/charts// - -# Verify build configuration -grep -A5 -B5 "" .github/workflows/build_external_container_images.yaml - -# Check dependency status -helm dependency list deployment/chainloop - -# Validate chart syntax -helm lint deployment/charts/ --strict -``` - ---- - -## Emergency Procedures - -### Rollback Steps -1. **Immediate**: Revert git changes - ```bash - git checkout HEAD -- deployment/ - ``` - -2. **Clean state**: Remove lock files and rebuild - ```bash - find deployment/ -name "Chart.lock" -delete - cd deployment/chainloop && helm dependency build - ``` - -3. **Verify**: Test rolled-back version - ```bash - cd devel && docker compose down && docker compose up - ``` - -### Escalation Criteria -**Contact team lead when**: -- Major version upgrade is required -- Breaking changes affect core functionality -- Multiple dependency conflicts arise -- Data migration is required - ---- - -## Process Checklist - -### Pre-Upgrade -- [ ] Version compatibility verified (no major version change) -- [ ] Current state documented -- [ ] Backup/rollback plan confirmed - -### During Upgrade -- [ ] Target version located and validated -- [ ] Image changes identified and updated -- [ ] Charts vendorized correctly -- [ ] Dependencies updated in correct order -- [ ] Build configuration updated (if needed) - -### Post-Upgrade -- [ ] Local testing passed -- [ ] Chart linting clean -- [ ] Image consistency verified -- [ ] Integration tests passed -- [ ] Documentation updated -- [ ] Temporary files cleaned up diff --git a/docs/runbooks/golang-upgrade-process.md b/docs/runbooks/golang-upgrade-process.md deleted file mode 100644 index c044ea43b..000000000 --- a/docs/runbooks/golang-upgrade-process.md +++ /dev/null @@ -1,113 +0,0 @@ -# Go Version Upgrade Process - -This document outlines the comprehensive process for upgrading Go versions across the Chainloop project. - -NOTE - -- do not upgrade dagger module - -## Overview - -The Chainloop project uses Go in multiple components requiring updates across: -- Source code (go.mod files) -- Docker images -- CI/CD pipelines (GitHub Actions) -- Documentation - -## Step-by-Step Process - -### 1. Get Latest Go Version and Docker Image Digest - -```bash -# Check latest Go version at https://go.dev/doc/devel/release -# Get Docker image SHA256 digest -docker pull golang:1.24.6 -# Note the SHA256 from output: sha256:2c89c41fb9efc3807029b59af69645867cfe978d2b877d475be0d72f6c6ce6f6 -``` - -### 2. Update Source Code - -Update all `go.mod` files in the project: -- `./go.mod` - Main project -- `./extras/dagger/go.mod` - Dagger module - -Change the `go` directive to new version: -```go -go 1.24.6 -``` - -### 3. Update Docker Images - -Update all Dockerfiles with new version and SHA256: -- `./app/artifact-cas/Dockerfile` -- `./app/artifact-cas/Dockerfile.goreleaser` -- `./app/controlplane/Dockerfile` -- `./app/controlplane/Dockerfile.goreleaser` -- `./app/cli/Dockerfile.goreleaser` - -Replace `FROM` lines with: -```dockerfile -FROM golang:1.24.6@sha256:2c89c41fb9efc3807029b59af69645867cfe978d2b877d475be0d72f6c6ce6f6 AS builder -``` - -### 4. Update GitHub Actions - -Update go-version in all workflow files: -- `./.github/workflows/lint.yml` -- `./.github/workflows/test.yml` -- `./.github/workflows/release.yaml` -- `./.github/workflows/codeql.yml` -- `./docs/examples/ci-workflows/github.yaml` - -Find and replace: -```yaml -go-version: "1.24.4" # old version -``` -with: -```yaml -go-version: "1.24.6" # new version -``` - -### 5. Update Documentation - -Update version reference in `./CLAUDE.md` - "Key Technologies" section: -```markdown -- **Language**: Go 1.24.6 -``` - -### 6. Test and Verify - -```bash -make test # Ensure compatibility -make lint # Check code quality -``` - -## Important Notes - -1. **SHA256 Verification**: Always use SHA256 digests for Docker images for security and reproducibility -2. **Test Thoroughly**: Go upgrades can introduce breaking changes -3. **Multiple Components**: CLI, Control Plane, and Artifact CAS all use Go -4. **Dagger Module**: Has separate go.mod that needs updating -5. **Development Environment**: Compose files use pre-built images, don't need Go updates - -## Files Updated in This Process - -### Source Code -- `./go.mod` - -### Docker Images -- `./app/artifact-cas/Dockerfile` -- `./app/artifact-cas/Dockerfile.goreleaser` -- `./app/controlplane/Dockerfile` -- `./app/controlplane/Dockerfile.goreleaser` -- `./app/cli/Dockerfile.goreleaser` - -### CI/CD Workflows -- `./.github/workflows/lint.yml` -- `./.github/workflows/test.yml` -- `./.github/workflows/release.yaml` -- `./.github/workflows/codeql.yml` -- `./docs/examples/ci-workflows/github.yaml` - -### Documentation -- `./CLAUDE.md` \ No newline at end of file From 9f7c3b430fdf31f501625120130b2dcc2c90e856 Mon Sep 17 00:00:00 2001 From: Miguel Martinez Date: Tue, 4 Nov 2025 00:49:30 +0100 Subject: [PATCH 2/2] chore: upgrade Go to 1.25.3 and Atlas to 0.38.0 Updated Go from 1.25.0 to 1.25.3 and Atlas from 0.36.0 to 0.38.0 across all components. Changes include: - Updated go.mod with Go 1.25.3 - Updated all Dockerfiles with new Go image digest - Updated GitHub Actions workflows with Go 1.25.3 - Updated CLAUDE.md documentation - Updated Atlas Docker image in Dockerfile.migrations - Updated Atlas CLI version in common.mk for make init - Enhanced upgrading-golang skill to include Atlas upgrade instructions Signed-off-by: Miguel Martinez --- .claude/skills/upgrading-golang/SKILL.md | 54 +++++++++++++++++-- .../upgrading-golang/files-to-update.md | 29 ++++++++-- .github/workflows/codeql.yml | 2 +- .github/workflows/lint.yml | 4 +- .github/workflows/release.yaml | 2 +- .github/workflows/test.yml | 2 +- CLAUDE.md | 2 +- app/artifact-cas/Dockerfile | 2 +- app/artifact-cas/Dockerfile.goreleaser | 2 +- app/cli/Dockerfile.goreleaser | 2 +- app/controlplane/Dockerfile | 2 +- app/controlplane/Dockerfile.goreleaser | 2 +- app/controlplane/Dockerfile.migrations | 8 +-- common.mk | 2 +- docs/examples/ci-workflows/github.yaml | 2 +- go.mod | 2 +- 16 files changed, 94 insertions(+), 25 deletions(-) diff --git a/.claude/skills/upgrading-golang/SKILL.md b/.claude/skills/upgrading-golang/SKILL.md index 5f260b810..455700233 100644 --- a/.claude/skills/upgrading-golang/SKILL.md +++ b/.claude/skills/upgrading-golang/SKILL.md @@ -9,9 +9,11 @@ This skill automates the comprehensive Go version upgrade process across all com ## Process -### 1. Confirm Target Version +### 1. Confirm Target Versions -Ask the user for the target Go version (e.g., "1.25.0"). +Ask the user: +1. What Go version they want to upgrade to (e.g., "1.25.3") +2. Whether they also want to upgrade Atlas migrations Docker image (if yes, ask for target Atlas version, e.g., "0.38.0") ### 2. Get Docker Image Digest @@ -65,7 +67,51 @@ Update the version reference in `./CLAUDE.md` under "Key Technologies": - **Language**: Go X.XX.X. To know how to upgrade go version, see docs/runbooks ``` -### 7. Verify Changes +### 7. Update Atlas Docker Image and CLI (Optional) + +If the user requested an Atlas upgrade: + +**7a. Pull the Atlas Docker image and extract its SHA256 digest:** + +```bash +docker pull arigaio/atlas:X.XX.X +``` + +Extract the SHA256 digest from the output (format: `sha256:abc123...`). + +**7b. Update `./app/controlplane/Dockerfile.migrations`:** + +Pattern to replace: +```dockerfile +# from: arigaio/atlas:X.XX.X +# docker run arigaio/atlas@sha256:OLD_DIGEST version +# atlas version vX.XX.X +FROM arigaio/atlas@sha256:OLD_DIGEST as base +``` + +With: +```dockerfile +# from: arigaio/atlas:X.XX.X +# docker run arigaio/atlas@sha256:NEW_DIGEST version +# atlas version vX.XX.X +FROM arigaio/atlas@sha256:NEW_DIGEST as base +``` + +**7c. Update `./common.mk` for `make init`:** + +Update the Atlas CLI installation version in the `init` target: + +Pattern to replace: +```makefile +curl -sSf https://atlasgo.sh | ATLAS_VERSION=vX.XX.X sh -s -- -y +``` + +With the new version (note: use `v` prefix for the version): +```makefile +curl -sSf https://atlasgo.sh | ATLAS_VERSION=vX.XX.X sh -s -- -y +``` + +### 8. Verify Changes Run verification commands: ```bash @@ -75,7 +121,7 @@ make lint If errors occur, address them before completing the upgrade. -### 8. Final Checks +### 9. Final Checks - Ensure all license headers are updated (2024 → 2024-2025 or add current year) - Run `buf format -w` if any proto files were affected diff --git a/.claude/skills/upgrading-golang/files-to-update.md b/.claude/skills/upgrading-golang/files-to-update.md index 431e10cb8..6e568b168 100644 --- a/.claude/skills/upgrading-golang/files-to-update.md +++ b/.claude/skills/upgrading-golang/files-to-update.md @@ -12,7 +12,7 @@ This reference lists all files that must be updated when upgrading Go versions. ## Docker Images -### Dockerfiles +### Dockerfiles (Golang) - `./app/artifact-cas/Dockerfile` - `./app/artifact-cas/Dockerfile.goreleaser` - `./app/controlplane/Dockerfile` @@ -24,6 +24,25 @@ Update pattern in all: FROM golang:X.XX.X@sha256:DIGEST AS builder ``` +### Atlas Files (Optional) +- `./app/controlplane/Dockerfile.migrations` - Docker image for migrations +- `./common.mk` - CLI tool installation in `make init` + +Update patterns: + +**Dockerfile.migrations:** +```dockerfile +# from: arigaio/atlas:X.XX.X +# docker run arigaio/atlas@sha256:DIGEST version +# atlas version vX.XX.X +FROM arigaio/atlas@sha256:DIGEST as base +``` + +**common.mk:** +```makefile +curl -sSf https://atlasgo.sh | ATLAS_VERSION=vX.XX.X sh -s -- -y +``` + ## CI/CD Workflows ### GitHub Actions @@ -48,9 +67,13 @@ go-version: "X.XX.X" ## Summary -**Total files to update**: 13 files +**Total files to update for Go**: 13 files - 1 go.mod file -- 5 Dockerfiles +- 5 Dockerfiles (Golang) - 5 GitHub Actions workflows - 1 example workflow - 1 documentation file + +**Optional Atlas upgrade**: 2 files +- 1 Dockerfile (Atlas migrations) +- 1 Makefile (Atlas CLI in make init) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 2c3d425a2..e2b958ca3 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -48,7 +48,7 @@ jobs: - name: Set up Go uses: actions/setup-go@be3c94b385c4f180051c996d336f57a34c397495 # v3.6.1 with: - go-version: "1.25.0" + go-version: "1.25.3" # Initializes the CodeQL tools for scanning. - name: Initialize CodeQL diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 4a89d38c5..789921978 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -26,7 +26,7 @@ jobs: steps: - uses: actions/setup-go@be3c94b385c4f180051c996d336f57a34c397495 # v3.6.1 with: - go-version: "1.25.0" + go-version: "1.25.3" - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 @@ -67,7 +67,7 @@ jobs: - uses: actions/setup-go@be3c94b385c4f180051c996d336f57a34c397495 # v3.6.1 with: - go-version: "1.25.0" + go-version: "1.25.3" - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index acb77f9b5..ee001cab2 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -78,7 +78,7 @@ jobs: - name: Set up Go uses: actions/setup-go@be3c94b385c4f180051c996d336f57a34c397495 # v3.6.1 with: - go-version: "1.25.0" + go-version: "1.25.3" # install qemu binaries for multiarch builds (needed by goreleaser/buildx) - name: Setup qemu diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 22bc96561..d8158d3fb 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -27,7 +27,7 @@ jobs: - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 - uses: actions/setup-go@be3c94b385c4f180051c996d336f57a34c397495 # v3.6.1 with: - go-version: "1.25.0" + go-version: "1.25.3" cache: true cache-dependency-path: go.sum diff --git a/CLAUDE.md b/CLAUDE.md index 88ddaa4bc..de9c1263c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -135,7 +135,7 @@ make migration_lint # Lint migration files ## Key Technologies -- **Language**: Go 1.25.0 +- **Language**: Go 1.25.3 - **API**: gRPC with HTTP/JSON gateway, Protocol Buffers with buf - **Database**: PostgreSQL with Ent ORM, Atlas for migrations - **Authentication**: OIDC, JWT tokens diff --git a/app/artifact-cas/Dockerfile b/app/artifact-cas/Dockerfile index 28b0a5076..7928f00c8 100644 --- a/app/artifact-cas/Dockerfile +++ b/app/artifact-cas/Dockerfile @@ -1,4 +1,4 @@ -FROM golang:1.25.0@sha256:9e56f0d0f043a68bb8c47c819e47dc29f6e8f5129b8885bed9d43f058f7f3ed6 AS builder +FROM golang:1.25.3@sha256:6bac879c5b77e0fc9c556a5ed8920e89dab1709bd510a854903509c828f67f96 AS builder # Not linked libraries since it will be injected into a scratch container ENV CGO_ENABLED=0 diff --git a/app/artifact-cas/Dockerfile.goreleaser b/app/artifact-cas/Dockerfile.goreleaser index 130bdc5ec..1d088d1b0 100644 --- a/app/artifact-cas/Dockerfile.goreleaser +++ b/app/artifact-cas/Dockerfile.goreleaser @@ -1,4 +1,4 @@ -FROM golang:1.25.0@sha256:9e56f0d0f043a68bb8c47c819e47dc29f6e8f5129b8885bed9d43f058f7f3ed6 AS builder +FROM golang:1.25.3@sha256:6bac879c5b77e0fc9c556a5ed8920e89dab1709bd510a854903509c828f67f96 AS builder FROM scratch diff --git a/app/cli/Dockerfile.goreleaser b/app/cli/Dockerfile.goreleaser index 1c0bc1de2..390959be7 100644 --- a/app/cli/Dockerfile.goreleaser +++ b/app/cli/Dockerfile.goreleaser @@ -1,4 +1,4 @@ -FROM golang:1.25.0@sha256:9e56f0d0f043a68bb8c47c819e47dc29f6e8f5129b8885bed9d43f058f7f3ed6 AS builder +FROM golang:1.25.3@sha256:6bac879c5b77e0fc9c556a5ed8920e89dab1709bd510a854903509c828f67f96 AS builder RUN mkdir -p /.config/chainloop FROM scratch diff --git a/app/controlplane/Dockerfile b/app/controlplane/Dockerfile index 0e7667b15..db7dd9ec5 100644 --- a/app/controlplane/Dockerfile +++ b/app/controlplane/Dockerfile @@ -1,4 +1,4 @@ -FROM golang:1.25.0@sha256:9e56f0d0f043a68bb8c47c819e47dc29f6e8f5129b8885bed9d43f058f7f3ed6 AS builder +FROM golang:1.25.3@sha256:6bac879c5b77e0fc9c556a5ed8920e89dab1709bd510a854903509c828f67f96 AS builder # Not linked libraries since it will be injected into a scratch container ENV CGO_ENABLED=0 diff --git a/app/controlplane/Dockerfile.goreleaser b/app/controlplane/Dockerfile.goreleaser index 98f6fae1f..a0fd3dbf5 100644 --- a/app/controlplane/Dockerfile.goreleaser +++ b/app/controlplane/Dockerfile.goreleaser @@ -1,4 +1,4 @@ -FROM golang:1.25.0@sha256:9e56f0d0f043a68bb8c47c819e47dc29f6e8f5129b8885bed9d43f058f7f3ed6 AS builder +FROM golang:1.25.3@sha256:6bac879c5b77e0fc9c556a5ed8920e89dab1709bd510a854903509c828f67f96 AS builder FROM scratch diff --git a/app/controlplane/Dockerfile.migrations b/app/controlplane/Dockerfile.migrations index 375a5010d..dc43a8a8c 100644 --- a/app/controlplane/Dockerfile.migrations +++ b/app/controlplane/Dockerfile.migrations @@ -1,9 +1,9 @@ # Container image built by go-releaser that's used to run migrations against the database during deployment # See https://atlasgo.io/guides/deploying/image -# from: arigaio/atlas:0.36.2 -# docker run arigaio/atlas@sha256:4d98fc2ce4a4dd3bc2124c80e3e16062e8b183836eec876dc84243dc70a120a9 version -# atlas version v0.36.2 -FROM arigaio/atlas@sha256:4d98fc2ce4a4dd3bc2124c80e3e16062e8b183836eec876dc84243dc70a120a9 as base +# from: arigaio/atlas:0.38.0 +# docker run arigaio/atlas@sha256:9883fdf5290020022ad0ac91fe20b846d32f93c19f68dfd3cf3b327c3e1b7e1a version +# atlas version v0.38.0 +FROM arigaio/atlas@sha256:9883fdf5290020022ad0ac91fe20b846d32f93c19f68dfd3cf3b327c3e1b7e1a as base FROM scratch # Update permissions to make it readable by the user diff --git a/common.mk b/common.mk index 1b26b6854..9eeba39ca 100644 --- a/common.mk +++ b/common.mk @@ -9,7 +9,7 @@ init: init-api-tools # in the community version anymore https://github.com/ariga/atlas/issues/2388#issuecomment-1864287189 # install golangci-lint with Go 1.25 support curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $$(go env GOPATH)/bin v2.4.0 - curl -sSf https://atlasgo.sh | ATLAS_VERSION=v0.36.0 sh -s -- -y + curl -sSf https://atlasgo.sh | ATLAS_VERSION=v0.38.0 sh -s -- -y # initialize API tooling .PHONY: init-api-tools diff --git a/docs/examples/ci-workflows/github.yaml b/docs/examples/ci-workflows/github.yaml index 9968b51b7..89a03dc5d 100644 --- a/docs/examples/ci-workflows/github.yaml +++ b/docs/examples/ci-workflows/github.yaml @@ -41,7 +41,7 @@ jobs: - name: Set up Go uses: actions/setup-go@v3 with: - go-version: "1.25.0" + go-version: "1.25.3" # Generate SBOM using syft in cycloneDX format - uses: anchore/sbom-action@v0 diff --git a/go.mod b/go.mod index bf6ebcd86..5cb75a342 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/chainloop-dev/chainloop -go 1.25.0 +go 1.25.3 require ( cloud.google.com/go/secretmanager v1.14.5