HC-567: Added support for building multi-architecture containers

[mcayanan]

Overview

This PR adds support for building and registering multi-architecture containers (x86_64 and ARM64) in the HySDS ecosystem while maintaining full backward compatibility with existing workflows.

Changes

🔧 Core Functionality

• container-met.py: Extended to accept optional ARM64 tarball parameter and register architecture-specific URLs
  • Added product_arm64 as optional 7th argument
  • Uploads both x86_64 and ARM64 tarballs to storage
  • Creates urls metadata field with architecture mappings:
    • x86_64 / amd64 → x86_64 tarball URL
    • arm64 / aarch64 → ARM64 tarball URL
  • Maintains backward compatibility with single-architecture containers

📝 Documentation

• MULTI_ARCH_BUILD.md: Comprehensive guide for multi-architecture builds
  • Usage examples for single and multi-platform builds
  • Prerequisites and setup instructions
  • Jenkins/CI integration examples
  • Troubleshooting guide

🔄 Metadata Schema

New field added to container metadata:

{
  "name": "container-name:tag",
  "version": "1.0.0",
  "url": "s3://bucket/container-name-tag.tar.gz",           // Legacy field (x86_64)
  "urls": "{                                                 // New field
    \"x86_64\": \"s3://bucket/container-name-tag.tar.gz\",
    \"amd64\": \"s3://bucket/container-name-tag.tar.gz\",
    \"arm64\": \"s3://bucket/container-name-tag-arm64.tar.gz\",
    \"aarch64\": \"s3://bucket/container-name-tag-arm64.tar.gz\"
  }",
  "digest": "sha256:...",
  "resource": "container"
}

Backward Compatibility ✅

• Existing workflows unchanged: If only x86_64 tarball is provided, behavior is identical to before
• Legacy url field preserved: Single URL field remains for backward compatibility
• Optional ARM64 parameter: 7th argument is optional; existing 6-argument calls work unchanged
• Graceful degradation: Systems without ARM64 support continue to work with x86_64 only

Usage

Register x86_64 only (existing behavior)

./container-met.py container-name:tag 1.0.0 container-name-tag.tar.gz s3://bucket sha256:digest http://mozart

Register both architectures (new)

./container-met.py container-name:tag 1.0.0 \
  container-name-tag.tar.gz \
  s3://bucket \
  sha256:digest \
  http://mozart \
  container-name-tag-arm64.tar.gz

Testing

• ✅ Single architecture registration (x86_64 only)
• ✅ Multi-architecture registration (x86_64 + ARM64)
• ✅ Backward compatibility with existing 6-argument invocations
• ✅ URL upload to S3 for both architectures
• ✅ Metadata POST to Mozart API with urls field

Migration Path

For existing deployments:

1. Deploy this PR to container-builder
2. Deploy related PRs to mozart, hysds, and hysds_commons
3. Update Jenkins/CI pipelines to pass ARM64 tarball (optional)
4. Existing containers continue to work without changes

For new multi-arch containers:

1. Build both architecture tarballs
2. Call container-met.py with both tarball paths
3. Workers automatically select correct architecture at runtime

---

Documentation: See MULTI_ARCH_BUILD.md for complete usage guide

[stirlingalgermissen]

LGTM