Skip to content

Improvement/build cicd#406

Merged
kellerza merged 5 commits intokellerza:mainfrom
maslyankov:improvement/build-cicd
Jan 22, 2025
Merged

Improvement/build cicd#406
kellerza merged 5 commits intokellerza:mainfrom
maslyankov:improvement/build-cicd

Conversation

@maslyankov
Copy link
Copy Markdown
Contributor

@maslyankov maslyankov commented Jan 22, 2025

Rate Limiting and Workflow Improvements

This PR addresses the "429 Too Many Requests" errors encountered during image manifest creation and improves the overall reliability of the deployment workflow.

Key Changes

1. Sequential Manifest Processing

  • Create base manifest first, then process platform-specific manifests one at a time
  • Added 30-second delays between manifest operations to respect rate limits
  • Clear logging of each step in the process

2. Enhanced Retry Mechanism

  • Implemented exponential backoff starting at 10 seconds
  • Maximum of 5 retry attempts per operation
  • Added descriptive logging for retry attempts and failures

3. Improved Git Operations

  • Added proper Git configuration for GitHub Actions bot
  • Implemented retry mechanism for Git push operations
  • Added rebase strategy for handling concurrent updates

4. Better Visibility

  • Added descriptive echo statements before sleep commands
  • Improved logging throughout the workflow
  • Clear status messages for each operation

Testing

The changes have been tested with multiple platforms (amd64, arm64, armv7, armv6) and handle rate limiting gracefully.

Impact

  • More reliable manifest creation and pushing
  • Better handling of GitHub API rate limits
  • Clearer visibility into workflow progress
  • More resilient to concurrent updates

Related issues

- Updated artifact naming to include sanitized platform name and added error handling for missing files.
- Changed the download artifact step to use a pattern for better flexibility.
- Configured Git settings for the action and implemented a rebase strategy before committing version updates.
- Added logic to conditionally update the version for the multi platform based on the environment.
- Improved push logic with retry mechanism for better reliability.

This update streamlines the deployment process and ensures version consistency across platforms.
- Updated job conditions to improve handling of workflow events, specifically for 'workflow_run' scenarios.
- Enhanced failure handling in the CI job by explicitly exiting with an error code.
- Adjusted the 'information' job to trigger on additional event types, ensuring better integration with release processes.
- Streamlined conditions for the 'publish_addon' job to ensure it only runs when all dependencies succeed.

These changes enhance the robustness and reliability of the deployment process.
- Introduced a retry mechanism with exponential backoff for the manifest creation and image inspection steps to improve reliability.
- Implemented batch processing for manifest creation to handle digests in smaller groups, reducing the risk of failures during the process.
- Added logging for batch processing to provide better visibility into the workflow execution.

These changes enhance the robustness of the deployment process, ensuring smoother multi-platform deployments.
- Added logging statements to indicate wait periods during manifest creation and image inspection steps.
- Introduced consistent wait times (30 seconds after manifest creation, 10 seconds between platform inspections, and 5 seconds before retrying pushes) to improve stability and reliability of the deployment process.

These changes aim to provide better visibility into the workflow execution and ensure smoother multi-platform deployments.
@kellerza kellerza merged commit 02f085f into kellerza:main Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants