Skip to content

TEST Fix flaky XPIA unit tests by stubbing workflow._memory#1716

Merged
hannahwestra25 merged 1 commit into
microsoft:mainfrom
hannahwestra25:hawestra/xpia_test_fix
May 11, 2026
Merged

TEST Fix flaky XPIA unit tests by stubbing workflow._memory#1716
hannahwestra25 merged 1 commit into
microsoft:mainfrom
hannahwestra25:hawestra/xpia_test_fix

Conversation

@hannahwestra25
Copy link
Copy Markdown
Contributor

Problem

Four tests in test_xpia.py were failing in CI with RuntimeError: memory offline:

  • test_perform_async_complete_workflow_with_scorer
  • test_perform_async_workflow_without_scorer
  • test_perform_async_scorer_returns_empty_list
  • test_perform_async_scorer_raises_exception

Root cause

These tests exercise XPIAWorkflow._perform_async, which calls _execute_processing_async, which in turn writes the processing response to memory via self._memory.add_message_to_memory(...). Unlike test_execute_processing_async_adds_to_memory, the failing tests did not replace workflow._memory with a mock, so they depended on a live central memory backend that isn't reliably available in CI.

Fix

Update the shared workflow pytest fixture (and the inline workflow construction in test_perform_async_workflow_without_scorer) to stub workflow._memory with a MagicMock. This keeps the tests properly unit-scoped and isolates them from the real memory backend without changing production behavior.

Tests and Documentation

Ran updated tests

@hannahwestra25 hannahwestra25 force-pushed the hawestra/xpia_test_fix branch from 1e9d757 to 7887068 Compare May 11, 2026 16:43
@hannahwestra25 hannahwestra25 added this pull request to the merge queue May 11, 2026
Merged via the queue into microsoft:main with commit b1d4f51 May 11, 2026
48 checks passed
@hannahwestra25 hannahwestra25 deleted the hawestra/xpia_test_fix branch May 11, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants