⚡ Optimize GitHub scan to run in background thread#79
Conversation
Offloaded the blocking `scanner.scan_repository` call to a separate thread using `asyncio.to_thread` to prevent freezing the main event loop during repository scans. - Added `import asyncio` to `skillmeat/api/routers/marketplace_sources.py` - Wrapped `scanner.scan_repository` in `await asyncio.to_thread(...)` - Verified reduction in event loop blocking from >1s to ~0.1s using a reproduction script. Co-authored-by: miethe <6800980+miethe@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
Offloaded the blocking
scanner.scan_repositorycall to a separate thread usingasyncio.to_threadto prevent freezing the main event loop during repository scans.💡 What
Modified
skillmeat/api/routers/marketplace_sources.pyto wrap the synchronousscanner.scan_repositorycall inasyncio.to_thread.🎯 Why
Scanning a GitHub repository involves multiple synchronous API calls (e.g., fetching tree, resolving ref) which can take seconds. Running this on the main event loop freezes the server, making it unresponsive to other requests (like health checks or other API calls).
📊 Measured Improvement
Using a reproduction script that mocks a 1-second blocking scan:
Max event loop gap: 1.1170s).Max event loop gap: 0.1157s).PR created automatically by Jules for task 8730940876819846466 started by @miethe