Skip to content

Add multi-arch install test workflow - #21

Merged
Joeavaikath merged 8 commits into
migtools:mainfrom
Joeavaikath:install-test-workflow
Jul 17, 2025
Merged

Add multi-arch install test workflow#21
Joeavaikath merged 8 commits into
migtools:mainfrom
Joeavaikath:install-test-workflow

Conversation

@Joeavaikath

@Joeavaikath Joeavaikath commented Jul 16, 2025

Copy link
Copy Markdown
Contributor

🏗️ Add Cross-Architecture Build & Test Workflow

📋 Overview

This PR introduces a comprehensive cross-architecture CI/CD pipeline that builds and tests the OADP CLI across 6 different architectures using GitHub's native runners. This ensures our kubectl plugin works correctly on all major platforms where users deploy it.

🎯 What This Adds

🔧 Multi-Architecture Build & Test Pipeline

  • Builds once, tests everywhere: Single build job creates binaries for all platforms
  • Native execution testing: Each binary runs on its target hardware architecture
  • Real kubectl plugin validation: Tests actual installation and plugin functionality
  • Comprehensive error handling: Explicit error checking with clear failure messages

🌍 Platform Coverage

Platform Architecture Runner Status
Linux amd64 ubuntu-latest ✅ Native testing
Linux arm64 ubuntu-24.04-arm ✅ Native testing
macOS amd64 macos-13 (Intel) ✅ Native testing
macOS arm64 macos-latest (Apple Silicon) ✅ Native testing
Windows amd64 windows-latest ✅ Native testing
Windows arm64 windows-11-arm ✅ Native testing

🚀 Key Features

Build Phase

  • Uses existing make release-build to build all 6 architectures
  • Validates all binaries are created successfully
  • Uploads platform-specific artifacts for parallel testing

Test Phase

  • Native execution: Each binary runs on its target platform's hardware
  • kubectl plugin testing: Validates kubectl oadp functionality
  • Help system validation: Tests all command help outputs
  • Installation testing: Verifies plugin installs correctly

Quality Assurance

  • Explicit error handling: set -e for bash, $ErrorActionPreference = "Stop" for PowerShell
  • Fail-fast behavior: Immediate exit with clear error messages on any failure
  • Comprehensive logging: Each step reports success/failure with descriptive output

🔧 Technical Improvements

Fixed Windows Testing

  • Windows .exe extension: Test binaries now properly handle .exe suffix
  • Native ARM64 testing: Uses windows-11-arm runner for true ARM64 validation
  • PowerShell error handling: Try-catch blocks with proper exception reporting

Enhanced Test Infrastructure

  • Cross-platform binary naming: Handles platform-specific executable extensions
  • Robust error detection: File existence checks before attempting operations
  • Clear failure reporting: Descriptive error messages for debugging

GitHub Actions Optimization

  • Efficient artifact strategy: Separate artifacts per platform for parallel downloads
  • Matrix optimization: Platform-specific configurations for optimal testing
  • Build status integration: Added workflow badge to README

📊 Workflow Structure

graph TD
    A[Build All Architectures] --> B[Linux Testing]
    A --> C[macOS Testing] 
    A --> D[Windows Testing]
    B --> E[Test Summary]
    C --> E
    D --> E
Loading

Jobs:

  1. build-all: Creates binaries for all 6 architectures using make
  2. test-linux: Native testing on Ubuntu x64 & ARM64 runners
  3. test-macos: Native testing on Intel & Apple Silicon runners
  4. test-windows: Native testing on Windows x64 & ARM64 runners
  5. test-summary: Aggregates results and reports overall status

🎉 Benefits

For Users

  • Confidence: All released binaries are validated on target hardware
  • Compatibility: Catches architecture-specific issues before release
  • Quality: Every platform gets the same thorough testing

For Developers

  • Early detection: CI catches cross-platform issues immediately
  • Debugging: Clear error messages pinpoint exactly what failed where
  • Automation: No manual testing needed across different architectures

For Releases

  • Release confidence: Every binary is validated before distribution
  • Platform parity: Ensures consistent experience across all platforms
  • Krew readiness: Validates kubectl plugin functionality for krew distribution

🧪 Testing

This workflow validates:

  • Cross-compilation: All 6 binaries build successfully
  • Native execution: Each binary runs correctly on target hardware
  • Help system: All command help outputs work properly
  • Plugin installation: kubectl plugin functionality across platforms
  • Test suite: Full Go test suite passes on each platform

📈 Files Changed

  • +442 lines: New comprehensive cross-architecture workflow
  • tests/common.go: Windows .exe extension support
  • .gitignore: Refined pattern for generated manifests
  • README.md: Added workflow status badge

🏁 Result

This PR establishes gold-standard cross-architecture CI that ensures the OADP CLI works flawlessly across all major platforms where Kubernetes runs, providing confidence for both developers and users.


Ready for production-quality releases! 🚀

…ution

- Fix tests/common.go to add .exe extension on Windows
- Update workflow to run on install-test-workflow branch
- Windows test only executes amd64 binary (native), verifies arm64 build
- Fixes: ResourceUnavailable error trying to execute ARM64 on amd64 runner
- Matrix testing both amd64 and arm64 fails because GitHub windows-latest runners are amd64-only
- ARM64 binaries cannot execute on amd64 runners (ResourceUnavailable error)
- Fixed approach: Execute amd64 natively, verify arm64 build only
- Use PowerShell syntax (.\) instead of bash syntax (./) for Windows
- Add install-test-workflow branch trigger for testing

This resolves: Program 'kubectl-oadp-windows-arm64.exe' failed to run: not a valid application for this OS platform
- Use windows-11-arm runner for native ARM64 Windows testing
- Matrix testing now works with native runners for both architectures:
  - windows-latest (amd64)
  - windows-11-arm (arm64)
- Both Windows binaries now get full native execution testing
- Removes build-verification-only approach for ARM64
- Complete native testing coverage: 6/6 architectures tested natively

This provides true cross-architecture validation instead of just build verification.
- Fix Linux ARM64 runner: ubuntu-latest-arm64 → ubuntu-24.04-arm
  (GitHub's ARM64 runners use -arm suffix, not -arm64)
- Simplify workflow triggers to only main branch pushes and PRs
- Remove test branch triggers (krew-init, install-test-workflow)

This should resolve the Linux ARM64 job not starting (showing blank)
and ensure workflow only runs for production branches.
cp ${{ matrix.binary }} /tmp/bin/kubectl-oadp
chmod +x /tmp/bin/kubectl-oadp
export PATH="/tmp/bin:$PATH"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While GitHub Actions runs with set -e by default (which causes the script to exit on the first non-zero exit code), it would be better to make the error handling explicit here.

Currently, if any of these kubectl-oadp commands fail, the success message will not be printed due to the default behavior, but this isn't immediately obvious from reading the code.

Consider using one of these approaches for clarity:

Option 1 - Chain with &&:

echo "Testing kubectl plugin functionality on ${{ matrix.arch }}..." && \
kubectl-oadp --help && \
kubectl-oadp version --help && \
kubectl-oadp nonadmin --help && \
echo "✅ kubectl plugin tests passed on ${{ matrix.arch }}"

Option 2 - Use if statement:

echo "Testing kubectl plugin functionality on ${{ matrix.arch }}..."
if kubectl-oadp --help && \
   kubectl-oadp version --help && \
   kubectl-oadp nonadmin --help; then
  echo "✅ kubectl plugin tests passed on ${{ matrix.arch }}"
else
  echo "❌ kubectl plugin tests failed on ${{ matrix.arch }}"
  exit 1
fi

This makes the error handling explicit and ensures the success message only appears when all commands succeed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicit error handling added, ty!

echo "Runner architecture: $(uname -m)"
echo "Go architecture: $(go env GOARCH)"

- name: Test binary execution

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same error handling concern as above. Consider making the error handling explicit with && chaining or an if statement.

Comment thread .github/workflows/cross-arch-build-test.yml Outdated
Comment thread .github/workflows/cross-arch-build-test.yml
Comment thread .github/workflows/cross-arch-build-test.yml
- Add 'set -e' to all bash scripts for immediate exit on errors
- Add '$ErrorActionPreference = "Stop"' to PowerShell scripts
- Replace implicit error handling with explicit checks and clear error messages
- Add file existence validation before attempting to use binaries
- Use try-catch blocks in PowerShell for better error reporting
- Add specific error messages with ❌ emoji for failed operations
- Simplify summary logic with overall_success tracking variable
- Ensure fail-fast behavior with descriptive failure reasons

This makes debugging much easier when builds or tests fail across different architectures.
@Joeavaikath
Joeavaikath requested a review from kaovilai July 17, 2025 16:08
@kaovilai

Copy link
Copy Markdown
Member

For future enhancement, you can https://github.com/docker/setup-qemu-action or podman equivalent and perform similar tests for ppc640le or s390x

@Joeavaikath
Joeavaikath merged commit 101cb01 into migtools:main Jul 17, 2025
10 checks passed
@Joeavaikath
Joeavaikath deleted the install-test-workflow branch February 19, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants