Do you need to file an issue?
Describe the bug
The synchronous with_cache middleware creates a new asyncio event loop and installs it as the thread's default with asyncio.set_event_loop().
On a cache miss, it closes that internal loop but leaves the now-closed loop installed as the caller's default. On a cache hit, it returns before event_loop.close(), so the internal loop remains installed and is leaked. Exceptions from cache access or the wrapped function also bypass the close.
This mutates caller-owned asyncio state even though the event loop is only an implementation detail of the cache middleware.
Steps to reproduce
Run against current main:
import asyncio
from graphrag_cache.memory_cache import MemoryCache
from graphrag_llm.middleware.with_cache import with_cache
from graphrag_llm.utils import create_completion_response
cache = MemoryCache()
def complete(**kwargs):
return create_completion_response("hello")
async def complete_async(**kwargs):
return complete(**kwargs)
cached_complete, _ = with_cache(
sync_middleware=complete,
async_middleware=complete_async,
request_type="chat",
cache=cache,
cache_key_creator=lambda _: "demo",
)
original = asyncio.new_event_loop()
asyncio.set_event_loop(original)
try:
cached_complete(model="test", messages=[])
installed_after_miss = asyncio.get_event_loop()
print({
"miss_return": "hello",
"original_preserved": installed_after_miss is original,
"installed_closed": installed_after_miss.is_closed(),
})
cached_complete(model="test", messages=[])
installed_after_hit = asyncio.get_event_loop()
print({
"hit_return": "hello",
"original_preserved": installed_after_hit is original,
"installed_closed": installed_after_hit.is_closed(),
})
finally:
installed = asyncio.get_event_loop()
asyncio.set_event_loop(None)
if installed is not original and not installed.is_closed():
installed.close()
original.close()
Observed output:
{'miss_return': 'hello', 'original_preserved': False, 'installed_closed': True}
{'hit_return': 'hello', 'original_preserved': False, 'installed_closed': False}
Expected Behavior
The synchronous cache middleware should preserve the caller's configured default event loop and always close its own internal event loop on cache hits, cache misses, and exceptions.
GraphRAG Config Used
Not applicable; this is reproducible directly through the LLM cache middleware.
Logs and screenshots
The observed output is included above.
Additional Information
- GraphRAG commit:
14a00ad88fc33cf2b52f4f113f25807556f8e25e
- Operating System: macOS
- Python Version: 3.12.13
- Related Issues: none found
- Suggested direction: avoid installing the internal loop as the thread default and close it in a
finally block.
- I can submit a focused regression test and fix.
Do you need to file an issue?
Describe the bug
The synchronous
with_cachemiddleware creates a new asyncio event loop and installs it as the thread's default withasyncio.set_event_loop().On a cache miss, it closes that internal loop but leaves the now-closed loop installed as the caller's default. On a cache hit, it returns before
event_loop.close(), so the internal loop remains installed and is leaked. Exceptions from cache access or the wrapped function also bypass the close.This mutates caller-owned asyncio state even though the event loop is only an implementation detail of the cache middleware.
Steps to reproduce
Run against current
main:Observed output:
Expected Behavior
The synchronous cache middleware should preserve the caller's configured default event loop and always close its own internal event loop on cache hits, cache misses, and exceptions.
GraphRAG Config Used
Not applicable; this is reproducible directly through the LLM cache middleware.
Logs and screenshots
The observed output is included above.
Additional Information
14a00ad88fc33cf2b52f4f113f25807556f8e25efinallyblock.