Fix event loop blocking with async subprocess operations#3
Closed
Fix event loop blocking with async subprocess operations#3
Conversation
Move all custom documentation to docs/rakuten/ for clear separation from upstream docs: - Move architecture docs to docs/rakuten/architecture/ - Add CONCURRENCY_LIMITS.md with analysis of API rate limits and safe concurrency levels for Rakuten CaaS deployment - Add README.md to explain custom vs upstream documentation This organization makes it clear which docs are Rakuten-specific and prevents merge conflicts with upstream changes.
Owner
Author
|
Closing this PR to create a cleaner one with only the async subprocess changes (without the queue monitoring commits) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes pod restart issues caused by blocking subprocess operations during large PR reviews by converting git operations to non-blocking async subprocess calls.
Problem
subprocess.run()calls freeze event loop for 5-6+ secondsSolution
Converted blocking subprocess calls to
asyncio.create_subprocess_exec():_clone_inner()- git clone operation_get_diff_files_with_git_clone()- git fetch operationget_file_from_git()- git show operationChanges
Core Implementation
git_provider.py:run_subprocess_async()- non-blocking subprocess executionrun_subprocess_with_output()- non-blocking with output captureclone()andget_diff_files()async in base classPropagated Changes
get_diff_files()callsTest Results
✅ All unit tests pass: 204 tests passed in 4.03s
✅ Async functionality tests pass:
Merge Conflict Risk
Deployment Plan
Documentation
docs/rakuten/ASYNC_BLOCKING_ANALYSIS.md- Problem analysisdocs/rakuten/SUBPROCESS_FIX_PLAN.md- Implementation planTesting
Local async tests verified:
Ready for deployment testing on k8s.