feat: add --sleep-between-batch flag to prevent build storms (#481)#620
Open
kimjune01 wants to merge 4 commits into
Open
feat: add --sleep-between-batch flag to prevent build storms (#481)#620kimjune01 wants to merge 4 commits into
kimjune01 wants to merge 4 commits into
Conversation
Implements a sleep duration between concurrent batches to prevent overwhelming CI systems when creating multiple PRs at once. - Add --sleep-between-batch flag accepting duration (e.g., 30s, 1m) - Modify runInParallel to wait for batch completion before sleeping - Add comprehensive tests for batch sleep behavior - Default is 0 (no sleep) to preserve existing behavior Fixes lindell#481
The previous implementation had a critical bug: it reset the WaitGroup between batches while goroutines might still be running. This would cause a panic when those goroutines called Done() on the replaced WaitGroup. New approach: - Use explicit batch loops instead of modulo arithmetic - Each batch has its own WaitGroup that waits for completion - Fast path (no sleep) uses simple semaphore pattern - Slow path (with sleep) explicitly batches and waits between them This ensures no goroutine holds a stale WaitGroup reference.
Remove unnecessary whitespace changes that create diff noise against the existing struct literal style.
The previous alignment fix commit didn't actually restore the original spacing. This aligns Labels and CloneDir back to the surrounding field group width.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--sleep-between-batchflag (e.g.--sleep-between-batch 30s) that inserts a configurable delay between concurrent batches0), the existing semaphore-based concurrency behavior is unchangedrunInParallelswitches to batch mode: process--concurrentrepos, wait for them to finish, sleep, then start the next batchThis prevents CI/CD build storms when multi-gitter pushes to many repos at once.
Fixes #481
Test plan
go test ./...all greengo vetclean, race detector cleanPrinter.Printpasses0for sleep duration)Failing tests before / Passing tests after / How the tests ran
Sweep attestation
5e95dce4e609— see the receipts footer below.