Complete Redis-backed test isolation hardening for parallel test runs

[vishal-bala]

Title

Finish hardening Redis-backed test isolation for parallel test runs

Summary

Redis-backed integration tests still have several concrete isolation gaps under pytest -n auto. The suite reuses the same Redis database for the life of a worker, and the shared Redis client fixtures do not flush state between tests. That means test isolation depends entirely on using unique Redis resource names and on consistently removing Redis state during teardown.

#543 reduced some of the highest-risk collisions, but the remaining problems are now well understood. There are still tests that use worker-scoped or fixed Redis names, tests that recreate indexes without dropping old documents, and tests whose cleanup only covers part of the Redis state they create. Those gaps are enough to leave same-worker contamination paths in the suite.

Problem

The core issue is not just that Redis-backed tests use state, but that they do so inconsistently. Some tests still use names that are only unique per xdist worker, which means a failure in one test can contaminate the next unrelated test on that same worker. Some tests still use fully fixed names, which makes repeated runs and reruns even more fragile. In other places, cleanup happens only on the happy path, so an assertion failure or unexpected exception can strand indexes, cache entries, or router metadata in Redis.

There is also a second-order problem in the way some fixtures recreate indexes. Several Redis-backed fixtures still call create(overwrite=True) without drop=True. In a worker-scoped Redis database, that means a rerun or interrupted earlier test can preserve old documents under the same prefix even if the index itself is recreated successfully. The result is a suite that can still produce order-dependent failures or confusing result-count drift even after the first stabilization pass.

Known Issues

• tests/integration/test_search_index.py and tests/integration/test_async_search_index.py still define shared fixtures using worker-scoped names like my_index_{worker_id} and stable prefixes like rvl, and those fixtures do not own teardown themselves. Both files also still contain fixed-name cases in from_existing_complex, and both have tests that hard-code my_index and leave it behind.

• tests/integration/test_hybrid.py, tests/integration/test_aggregation.py, and tests/integration/test_search_results.py still use worker-only names such as user_index_{worker_id} and v1_{worker_id} and recreate indexes with create(overwrite=True) but not drop=True, which leaves a stale-document path if an earlier run aborts after loading data.

• tests/integration/test_svs_integration.py has the same recreate pattern: worker-scoped names plus create(overwrite=True) without drop=True, which means interrupted earlier runs can leave vector documents behind for later tests on the same worker.

• tests/integration/test_llmcache.py still has multiple leak paths. Core fixtures use worker-scoped names only, cache_no_cleanup has no finalizer, and several ad hoc tests create SemanticCache instances without deleting them afterward. There is also a direct collision around float64_cache_{worker_id}, which is reused by both test_create_cache_with_different_vector_types and test_bad_dtype_connecting_to_existing_cache without cleanup.

• tests/integration/test_semantic_router.py still has structurally incomplete cleanup because router teardown clears route documents and drops the search index but does not remove the persisted route config key.

• tests/integration/test_key_separator_handling.py creates real SemanticRouter instances with fixed names like test_router_sep, router_sep_test, and router_trailing_test and does not delete them. It also creates a real index named search_test and only cleans it up at the end of the test body.

• tests/integration/test_redis_cluster_support.py and tests/integration/test_cluster_pipelining.py still use fixed index and router names such as test_cluster_index, test_cluster_router, test-real-365, test-batch-365, and test-ttl-cluster, which makes cluster-backed tests vulnerable to residual Redis state.

• tests/integration/test_embedcache_warnings.py writes under the fixed test_cache namespace and does not clear those keys afterward.

• tests/integration/test_unf_noindex_integration.py still contains a fixed-name persistence test using test_persistence.

• tests/integration/test_withsuffixtrie_integration.py uses worker-scoped index names and performs cleanup only at the end of the test body rather than in guaranteed teardown.

• tests/integration/test_no_proactive_module_checks.py uses a hard-coded shared prefix doc, and several tests create and delete indexes against that prefix. Because cleanup drops indexed documents, those tests can remove unrelated doc* records created elsewhere. Cleanup in that file is also not consistently wrapped in finally.

• tests/integration/test_message_history.py still has one cleanup inconsistency: the plain standard_history fixture calls history.clear() but does not drop the underlying search index, leaving orphaned index metadata behind for the worker lifetime.

Work To Be Done

The next pass should convert the remaining worker-scoped Redis resource names to per-test names in the highest-risk modules first, especially the sync and async search-index suites, hybrid and aggregation coverage, llmcache, router-related tests, and cluster-backed tests. Fixed Redis-backed names should also be removed from key-separator, embedcache warning, persistence, and cluster tests so those modules no longer assume a pristine Redis instance.

Teardown behavior should be normalized at the same time. Tests that currently delete Redis state only on the happy path should move that cleanup into fixture teardown or finally blocks. cache_no_cleanup should be removed or rewritten so the fixture itself is safe under xdist. Router teardown should be expanded so it removes persisted route config keys in addition to search-index state. Message-history and embedcache warning tests should also be updated so they do not leave behind empty indexes or fixed-prefix cache keys.

The recreate pattern also needs to be tightened. Anywhere a deterministic Redis resource name is reused, setup should either switch to a per-test namespace or use a drop-on-recreate path so stale documents cannot survive an interrupted earlier run.

Concrete Follow-up Areas

• Convert worker-scoped naming to per-test naming in:

  • tests/integration/test_search_index.py
  • tests/integration/test_async_search_index.py
  • tests/integration/test_hybrid.py
  • tests/integration/test_aggregation.py
  • tests/integration/test_search_results.py
  • tests/integration/test_svs_integration.py
  • tests/integration/test_llmcache.py
  • tests/integration/test_withsuffixtrie_integration.py

• Replace fixed Redis-backed names in:

  • tests/integration/test_key_separator_handling.py
  • tests/integration/test_embedcache_warnings.py
  • tests/integration/test_unf_noindex_integration.py
  • tests/integration/test_cluster_pipelining.py
  • tests/integration/test_redis_cluster_support.py

• Fix incomplete teardown in:

  • tests/integration/test_semantic_router.py
  • tests/integration/test_key_separator_handling.py
  • tests/integration/test_redis_cluster_support.py
  • tests/integration/test_message_history.py
  • tests/integration/test_llmcache.py
  • tests/integration/test_search_index.py
  • tests/integration/test_async_search_index.py
  • tests/integration/test_no_proactive_module_checks.py

• Review deterministic create(overwrite=True) setup paths that currently do not drop=True, including:

  • tests/conftest.py
  • tests/integration/test_hybrid.py
  • tests/integration/test_aggregation.py
  • tests/integration/test_search_results.py
  • tests/integration/test_svs_integration.py

Acceptance Criteria

This work is complete when Redis-backed integration tests no longer rely on worker-shared or fixed Redis resource names across unrelated tests, Redis-backed objects are cleaned up consistently even when a test fails partway through, router tests remove both search-index state and persisted route-config state, and deterministic recreate paths no longer preserve stale documents from interrupted earlier runs. Targeted repeated xdist runs should no longer show same-worker Redis residue once those changes land.