Skip to content

fix(memory): batch semantic processing in _process_memory_directory#1304

Merged
MaojiaSheng merged 1 commit intomainfrom
fix/memory
Apr 8, 2026
Merged

fix(memory): batch semantic processing in _process_memory_directory#1304
MaojiaSheng merged 1 commit intomainfrom
fix/memory

Conversation

@chuanbao666
Copy link
Copy Markdown
Collaborator

@chuanbao666 chuanbao666 commented Apr 8, 2026

Summary

修复记忆目录分批语义处理以防止协程调度风暴
fix: batch semantic processing in _process_memory_directory to prevent CPU spikes (#1245)

Type of Change

  • New feature (feat)
  • Bug fix (fix)
  • Documentation (docs)
  • Refactoring (refactor)
  • Other

Testing

分批并发控制 : 在 semantic_processor.py 中,将无限制的 asyncio.gather 改造为按批次循环执行。

  • 动态计算批大小: min(max_concurrent_llm, 10) ,确保单次提交给事件循环的任务量可控。

  • 增加批次处理日志,提升后台任务的可观测性。

  • Unit tests pass

  • Manual testing completed

Related Issues

Checklist

  • Code follows project style guidelines
  • Tests added for new functionality
  • Documentation updated (if needed)
  • All tests pass

@chuanbao666 chuanbao666 changed the title fix(memory): batch semantic processing in _process_memory_directory t… fix(memory): batch semantic processing in _process_memory_directory Apr 8, 2026
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

1245 - Partially compliant

Compliant requirements:

  • Fix _process_memory_directory to avoid launching hundreds of concurrent coroutines at once
  • Batch asyncio.gather calls to prevent CPU spikes on low-core machines
  • Ensure steady progress through the memory file queue
  • Maintain existing functionality while adding batching

Non-compliant requirements:

Requires further human verification:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🏅 Score: 92
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

PR Code Suggestions ✨

No code suggestions found for the PR.

@MaojiaSheng MaojiaSheng merged commit f500dd2 into main Apr 8, 2026
5 of 9 checks passed
@MaojiaSheng MaojiaSheng deleted the fix/memory branch April 8, 2026 13:40
@github-project-automation github-project-automation bot moved this from Backlog to Done in OpenViking project Apr 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Bug]: _process_memory_directory launches hundreds of concurrent coroutines via asyncio.gather, causing 100% CPU on low-core machines

3 participants