Skip to content

[Bug]: Sync cache middleware replaces and leaks caller event loops #2478

Description

@FU-max-boop

Do you need to file an issue?

  • I have searched the existing issues and pull requests and this bug is not already filed.
  • This report concerns GraphRAG's provider-independent cache middleware.
  • I believe this is a legitimate bug, not just a question.

Describe the bug

The synchronous with_cache middleware creates a new asyncio event loop and installs it as the thread's default with asyncio.set_event_loop().

On a cache miss, it closes that internal loop but leaves the now-closed loop installed as the caller's default. On a cache hit, it returns before event_loop.close(), so the internal loop remains installed and is leaked. Exceptions from cache access or the wrapped function also bypass the close.

This mutates caller-owned asyncio state even though the event loop is only an implementation detail of the cache middleware.

Steps to reproduce

Run against current main:

import asyncio

from graphrag_cache.memory_cache import MemoryCache
from graphrag_llm.middleware.with_cache import with_cache
from graphrag_llm.utils import create_completion_response

cache = MemoryCache()

def complete(**kwargs):
    return create_completion_response("hello")

async def complete_async(**kwargs):
    return complete(**kwargs)

cached_complete, _ = with_cache(
    sync_middleware=complete,
    async_middleware=complete_async,
    request_type="chat",
    cache=cache,
    cache_key_creator=lambda _: "demo",
)
original = asyncio.new_event_loop()
asyncio.set_event_loop(original)

try:
    cached_complete(model="test", messages=[])
    installed_after_miss = asyncio.get_event_loop()
    print({
        "miss_return": "hello",
        "original_preserved": installed_after_miss is original,
        "installed_closed": installed_after_miss.is_closed(),
    })

    cached_complete(model="test", messages=[])
    installed_after_hit = asyncio.get_event_loop()
    print({
        "hit_return": "hello",
        "original_preserved": installed_after_hit is original,
        "installed_closed": installed_after_hit.is_closed(),
    })
finally:
    installed = asyncio.get_event_loop()
    asyncio.set_event_loop(None)
    if installed is not original and not installed.is_closed():
        installed.close()
    original.close()

Observed output:

{'miss_return': 'hello', 'original_preserved': False, 'installed_closed': True}
{'hit_return': 'hello', 'original_preserved': False, 'installed_closed': False}

Expected Behavior

The synchronous cache middleware should preserve the caller's configured default event loop and always close its own internal event loop on cache hits, cache misses, and exceptions.

GraphRAG Config Used

Not applicable; this is reproducible directly through the LLM cache middleware.

Logs and screenshots

The observed output is included above.

Additional Information

  • GraphRAG commit: 14a00ad88fc33cf2b52f4f113f25807556f8e25e
  • Operating System: macOS
  • Python Version: 3.12.13
  • Related Issues: none found
  • Suggested direction: avoid installing the internal loop as the thread default and close it in a finally block.
  • I can submit a focused regression test and fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions