Skip to content

Conversation

@colifran
Copy link
Contributor

Description of changes:

  • Refactored retrieve_customer_context to retrieve memories from multiple namespaces concurrently using ThreadPoolExecutor instead of sequentially
  • Extracted namespace retrieval logic into a helper function retrieve_for_namespace for cleaner concurrent execution
  • Added per-namespace error handling so failures in one namespace don't block retrieval from others (graceful degradation)
  • Removed unneeded braces around len(all_context)

This improves response time for support queries when multiple namespaces are configured, reducing total retrieval time from the sum of all calls to the maximum of any single call.

Testing:

  • I performed manual testing with tracing enabled this screenshot shows the current implementation is making retrieve_memories API calls sequentially:
Screenshot 2025-11-19 at 2 24 53 PM
  • This screenshot shows that retrieve_memories API calls are made in parallel with the updated code:
Screenshot 2025-11-10 at 3 59 36 PM

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@colifran colifran changed the title feat: parallelize retrieve memories API calls for multiple namespaces… feat: parallelize retrieve memories API calls for multiple namespaces to improve latency Nov 19, 2025
bergjaak
bergjaak previously approved these changes Nov 19, 2025
jona62
jona62 previously approved these changes Nov 19, 2025
Copy link
Contributor

@jona62 jona62 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@codecov-commenter
Copy link

codecov-commenter commented Nov 19, 2025

Codecov Report

❌ Patch coverage is 86.95652% with 3 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@988ca8f). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...ore/memory/integrations/strands/session_manager.py 86.95% 0 Missing and 3 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #163   +/-   ##
=======================================
  Coverage        ?   90.32%           
=======================================
  Files           ?       25           
  Lines           ?     2718           
  Branches        ?      389           
=======================================
  Hits            ?     2455           
  Misses          ?      153           
  Partials        ?      110           
Flag Coverage Δ
unittests 90.32% <86.95%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@colifran colifran dismissed stale reviews from jona62 and bergjaak via 660f0d7 November 19, 2025 21:01
bergjaak
bergjaak previously approved these changes Nov 19, 2025
@jona62 jona62 merged commit df5a2c9 into aws:main Nov 19, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants