Skip to content

feat: lazy loading + instance-level caching for SchemaRegistry#16

Merged
nkanu17 merged 14 commits intomainfrom
feature/lazy-schema-caching
Apr 1, 2026
Merged

feat: lazy loading + instance-level caching for SchemaRegistry#16
nkanu17 merged 14 commits intomainfrom
feature/lazy-schema-caching

Conversation

@nkanu17
Copy link
Copy Markdown
Contributor

@nkanu17 nkanu17 commented Mar 28, 2026

Problem

Every SQL query executed through redisvl's SearchIndex.query(SQLQuery(...)) creates a new SchemaRegistry and calls load_all(), which issues:

  1. FT._LIST — enumerate all indexes on the server
  2. FT.INFO — for every single index on the server

With 10 indexes on the server, that's 12 Redis round-trips of pure overhead per query before the actual FT.SEARCH or FT.AGGREGATE is even sent. At 1,000 queries, that's 12,000 wasted round-trips. At 10,000 queries, it's 120,000.

On localhost this adds milliseconds. On cloud deployments with 1ms RTT, it adds seconds. Cross-region at 5ms RTT, it adds minutes.

The schema almost never changes between queries — this overhead is almost entirely waste.

Solution

Core: lazy loading + instance-level caching

get_schema() now loads on demand. Instead of requiring load_all() upfront, get_schema(index) issues a single FT.INFO call on first access for a given index and caches the result. Subsequent calls return the cached schema with zero I/O. This eliminates FT._LIST entirely — we only fetch the schema for the index being queried, not all indexes on the server.

invalidate() clears cached schemas (all or per-index), forcing a re-fetch on next access. Provides an escape hatch for stale schema scenarios without re-introducing the overhead.

Negative caching. If FT.INFO returns "no such index" or "unknown index", the result is cached as {} to avoid repeated round-trips for missing indexes. The watcher (process_pending_events()) excludes these negative-cached entries from its comparisons to prevent spurious "dropped" events.

Async: ensure_schema() with concurrency guard

AsyncSchemaRegistry.ensure_schema() provides the async lazy-load path:

  • Concurrency guard — concurrent callers for the same index share a single in-flight FT.INFO task via _loading dict
  • Cancellation safety — the shared task is wrapped in asyncio.shield() so caller cancellation (e.g. asyncio.wait_for timeout) doesn't cancel the shared load for other awaiters. invalidate()/refresh()/load_all() still cancel tasks directly via task.cancel()
  • Caller cancellation preserved — after shield(), if the shared task isn't cancelled but CancelledError is raised, it's re-raised to properly abort the caller
  • Task cleanup — a done-callback is attached at task creation to clean up _loading even if all awaiters cancel (no finally block runs). The finally block handles the common case; the callback is the backstop
  • Early unknown-index errortranslate_parsed() raises ValueError("Unknown index: ...") when get_schema() returns an empty dict, instead of deferring the error to Redis execution time

Translator/executor changes

  • Translator.parse(sql) and Translator.translate_parsed(parsed) — public methods that allow callers to parse SQL once and reuse the AST. translate(sql) delegates to these internally. No breaking changes.
  • AsyncExecutor.execute() calls ensure_schema() automatically before translation, using parse() + translate_parsed() to avoid double-parsing. Async lazy-loading works without requiring load_all() upfront, matching sync behavior.

What did NOT change

  • load_all() still works identically for callers that need it
  • No changes to parser.py or analyzer.py
  • No breaking changes to any existing public API
  • No new dependencies

Usage

Sync — lazy loading with caching (recommended)

from sql_redis.executor import Executor
from sql_redis.schema import SchemaRegistry

# Create once, reuse across all queries
registry = SchemaRegistry(redis_client)
executor = Executor(redis_client, registry)

# First query triggers a single FT.INFO for "products"
result = executor.execute("SELECT name, price FROM products WHERE category = 'electronics'")

# Subsequent queries reuse the cached schema — zero Redis overhead
result = executor.execute("SELECT name FROM products WHERE price > 100")
result = executor.execute("SELECT category, COUNT(*) FROM products GROUP BY category")

# If the index schema changes on the server, invalidate and re-fetch lazily
registry.invalidate("products")  # next query will re-fetch
registry.invalidate()            # clear all cached schemas

Async — lazy loading with caching

from sql_redis.executor import AsyncExecutor
from sql_redis.schema import AsyncSchemaRegistry

registry = AsyncSchemaRegistry(async_redis_client)
executor = AsyncExecutor(async_redis_client, registry)

# AsyncExecutor calls ensure_schema() automatically — no load_all() needed
result = await executor.execute("SELECT name, price FROM products WHERE category = 'books'")

# Subsequent queries reuse the cached schema — zero Redis overhead
result = await executor.execute("SELECT name FROM products WHERE price > 100")

# Invalidate works the same way as sync
registry.invalidate("products")

# ensure_schema() is also available for manual pre-loading if needed
await registry.ensure_schema("products")

Backward compatible — load_all() still works

# Existing code that calls load_all() continues to work unchanged
registry = SchemaRegistry(redis_client)
registry.load_all()  # loads all indexes eagerly, as before
executor = Executor(redis_client, registry)
result = executor.execute("SELECT * FROM products")

Benchmark Results

Setup

  • Redis: localhost:6379
  • 1 target index (bench_products) with 5 fields and 10 documents
  • 10 background indexes (simulating a multi-index server)
  • 5 SQL queries cycled round-robin (tag filter, numeric range, text search, aggregation, combined filter)
  • 3 runs per mode, median selected

Four modes compared

Mode What it does
current redisvl New SchemaRegistry + load_all() per query (current production behavior)
load_all (baseline) load_all() once upfront, reuse registry
lazy (no cache) Lazy get_schema() but new registry per query
lazy + cached Single registry reused with lazy get_schema() (target behavior)

Results at 100 / 1,000 / 10,000 queries (11 total indexes)

Mode 100q Cmds 1Kq Cmds 10Kq Cmds 10Kq Time
current redisvl 1,300 13,000 130,000 25,044 ms
load_all (baseline) 112 1,012 10,012 3,925 ms
lazy (no cache) 200 2,000 20,000 5,726 ms
lazy + cached 101 1,001 10,001 4,060 ms

Prediction validation (5,000 queries, 10 total indexes)

Mode Total Cmds FT._LIST FT.INFO Query Cmds Total Time
current redisvl 60,000 5,000 50,000 5,000 11,582 ms
load_all (baseline) 5,011 1 10 5,000 1,935 ms
lazy (no cache) 10,000 0 5,000 5,000 2,803 ms
lazy + cached 5,001 0 1 5,000 1,852 ms

60,000 down to 5,001 round-trips (12x reduction). Wall-clock time drops from 11.6s to 1.9s (6.3x faster on localhost).

Per-query overhead

Mode Commands per query Formula
current redisvl 1 + N_indexes + 1 = 12 FT._LIST + N × FT.INFO + query
load_all (baseline) ~1 (amortized) query only (one-time setup)
lazy (no cache) 2 1 FT.INFO + query
lazy + cached ~1 (amortized) query only (1 FT.INFO total)

Running the benchmark

# Default: 100 and 1000 queries, 10 background indexes, 3 runs per mode
python benchmarks/bench_schema_caching.py

# Custom query counts
python benchmarks/bench_schema_caching.py --queries 100 1000 5000 10000

# Point to a remote Redis instance
python benchmarks/bench_schema_caching.py --redis-url redis://myhost:6379

# More background indexes to simulate a busier server
python benchmarks/bench_schema_caching.py --bg-indexes 20

# Skip graph generation (no matplotlib dependency needed)
python benchmarks/bench_schema_caching.py --no-graphs

Limitations

  • No TTL-based expiry: Cached schemas have no time-based invalidation. If an index schema changes, invalidate() must be called explicitly.
  • Per-instance cache: Not shared across processes or connections.
  • No auto schema change detection: Acceptable because index schemas rarely change during normal operation.

@nkanu17 nkanu17 requested a review from Copilot March 28, 2026 03:23
@nkanu17 nkanu17 changed the title feature/lazy loading + instance-level caching for SchemaRegistry feat: lazy loading + instance-level caching for SchemaRegistry Mar 28, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces lazy, per-index schema loading with instance-level caching in SchemaRegistry (and async equivalents) to eliminate repeated FT._LIST/FT.INFO overhead per SQL query, plus adds tests and a benchmark script to validate/measure the impact.

Changes:

  • Make SchemaRegistry.get_schema() lazily fetch schema via FT.INFO on cache miss and reuse cached results; get_field_type() now goes through get_schema().
  • Add cache invalidation via invalidate(index|None) for sync and async registries; add AsyncSchemaRegistry.ensure_schema() for async lazy loading.
  • Add integration tests for schema caching behavior and a benchmark script to measure Redis command counts/latency across modes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 12 comments.

File Description
sql_redis/schema.py Implements lazy schema fetch + caching, adds invalidation, and adds async ensure_schema() path.
tests/test_schema_caching.py New integration tests validating lazy load + caching + invalidation behavior (sync).
benchmarks/bench_schema_caching.py New benchmark to compare command counts/latency across eager/lazy/cached schema-loading modes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/schema.py
Comment thread sql_redis/schema.py Outdated
Comment thread tests/test_schema_caching.py
Comment thread tests/test_schema_caching.py
Comment thread tests/test_schema_caching.py
Comment thread benchmarks/bench_schema_caching.py Outdated
Comment thread sql_redis/schema.py
Comment thread benchmarks/bench_schema_caching.py Outdated
Comment thread benchmarks/bench_schema_caching.py Outdated
Comment thread sql_redis/schema.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

sql_redis/schema.py:122

  • SchemaRegistry.refresh() docstring says a missing index is removed from the registry, but _load_index_schema() now negative-caches missing indexes as an empty dict. Update the docstring (or behavior) so callers don’t assume the key is removed when the index is gone.
    def refresh(self, index_name: str) -> None:
        """Refresh schema for a single index.

        If the index no longer exists, removes it from the registry.
        If the index is new, adds it to the registry.
        """

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/schema.py Outdated
Comment thread benchmarks/bench_schema_caching.py Outdated
Comment thread tests/test_schema_caching.py
Comment thread tests/test_schema_caching.py
Comment thread sql_redis/schema.py Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/schema.py Outdated
Comment thread sql_redis/executor.py Outdated
Comment thread sql_redis/schema.py Outdated
@nkanu17 nkanu17 requested a review from Copilot March 28, 2026 04:12
@nkanu17 nkanu17 marked this pull request as ready for review March 28, 2026 04:12
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3afae2b6b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sql_redis/schema.py
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3afae2b6b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sql_redis/schema.py Outdated
Comment thread sql_redis/schema.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

sql_redis/schema.py:125

  • SchemaRegistry.refresh() docstring says missing indexes are removed from the registry, but _load_index_schema() now negative-caches missing indexes by storing an empty dict. Either update the docstring to match the new behavior, or adjust refresh()/_load_index_schema() so refresh of a deleted index removes it rather than leaving a cached {} entry.
        """Refresh schema for a single index.

        If the index no longer exists, removes it from the registry.
        If the index is new, adds it to the registry.
        """

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/executor.py
Comment thread tests/test_schema_caching.py
Comment thread sql_redis/schema.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/schema.py Outdated
Comment thread benchmarks/bench_schema_caching.py
Comment thread sql_redis/translator.py
Comment thread tests/test_schema_caching.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/schema.py Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread benchmarks/bench_schema_caching.py Outdated
Comment thread sql_redis/executor.py
Comment thread sql_redis/schema.py Outdated
Comment thread sql_redis/schema.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/schema.py
Comment thread sql_redis/schema.py
Comment thread sql_redis/translator.py
Comment thread tests/test_schema_caching.py
Comment thread tests/test_schema_caching.py
Comment thread tests/test_schema_caching.py
Comment thread sql_redis/schema.py Outdated
@nkanu17 nkanu17 requested a review from Copilot March 30, 2026 13:58
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/schema.py
Comment thread sql_redis/translator.py Outdated
Comment thread benchmarks/bench_schema_caching.py Outdated
@nkanu17 nkanu17 requested a review from Copilot March 30, 2026 19:36
@nkanu17
Copy link
Copy Markdown
Contributor Author

nkanu17 commented Mar 30, 2026

@codex review

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/translator.py
Comment thread sql_redis/schema.py
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eb7ae4ca74

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sql_redis/schema.py Outdated
nkanu17 and others added 14 commits March 30, 2026 16:04
- get_schema() now fetches schema on first access via FT.INFO and caches it
- Eliminates FT._LIST calls entirely from the SQL query path
- Add invalidate() method for cache clearing (all or per-index)
- Async support via ensure_schema() in AsyncSchemaRegistry
- 14 integration tests covering caching, invalidation, and lazy loading
- Measures real Redis round-trips via CommandCounter wrapper
- 4 modes: current redisvl, load_all baseline, lazy no-cache, lazy+cached
- Configurable via CLI: --queries, --redis-url, --bg-indexes, --runs
- Generates comparison charts with matplotlib (optional)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Add negative caching for missing indexes (sync + async): cache {} on
  ResponseError to avoid repeated FT.INFO calls for non-existent indexes
- Add async concurrency guard in ensure_schema(): use _loading task map
  to deduplicate concurrent FT.INFO requests for the same index
- Cancel in-flight tasks on invalidate(), load_all(), and refresh()
- Fix benchmark Executor reuse inconsistency in run_load_all()
- Fix docstring mode number mismatches in benchmark runners
- Fix ImportError message to cover matplotlib/numpy
- Add comprehensive async integration tests for ensure_schema(),
  invalidate(), negative caching, and concurrency guard
… safe cleanup

- Wire AsyncExecutor.execute() to call ensure_schema() before translation,
  so async path works without requiring load_all() upfront (matches sync behavior)
- Narrow negative-cache in _load_index_schema (sync + async) to only cache {}
  when error contains 'no such index'; re-raise other ResponseErrors
- Limit benchmark setup_redis() cleanup to bench_*/bg_index_* prefixes
  instead of dropping all indexes on the server
- Replace bare Exception catches in async test fixtures with redis.ResponseError
- Add async_caching_data fixture and TestAsyncLazySchemaWithExecutor tests
- Handle asyncio.CancelledError in ensure_schema() so invalidate()
  during an active ensure_schema() returns post-invalidate cache state
  instead of propagating CancelledError to callers
- Only remove task from _loading if it's still the current one
- Add Translator.translate_parsed(ParsedQuery) to accept pre-parsed queries
- Wire AsyncExecutor.execute() to parse once and use translate_parsed(),
  eliminating redundant sqlglot parsing per async query
- Remove unused SQLParser import from executor.py
… parse() API

- ensure_schema(): check task.cancelled() to distinguish between internal
  task cancellation (from invalidate/refresh) and caller cancellation
  (e.g., asyncio.wait_for timeout). Only swallow internal cancellations;
  re-raise caller-driven CancelledError so callers abort properly.
- process_pending_events(): exclude negative-cached (empty) schemas from
  cached_indexes set so they don't trigger spurious 'dropped' events or
  mask newly created indexes from being detected.
- Add Translator.parse(sql) as public API for pre-parsing SQL, replacing
  private _parser access in AsyncExecutor.
- ensure_schema(): wrap shared task await with asyncio.shield() so
  caller cancellation (e.g. asyncio.wait_for timeout) doesn't cancel
  the shared FT.INFO task for other awaiters. invalidate()/refresh()
  still cancel via task.cancel() directly.
- Update PR docs to reflect translator/executor changes: Translator.parse(),
  translate_parsed(), AsyncExecutor auto lazy-load, updated test counts.
Both sync and async _load_index_schema() now match 'unknown index'
in addition to 'no such index' when deciding to negative-cache a
missing index. Defensive against Redis version differences in
FT.INFO error messages.
- Benchmark p95/p99: use statistics.quantiles() instead of manual index
  calculation that was off-by-one for small sample sizes
- process_pending_events(): decode FT._LIST bytes to str before comparing
  with _schemas keys, consistent with load_all(). Fixes incorrect
  new/deleted detection when decode_responses=False.
… cancel

- Update sync and async refresh() docstrings to reflect that missing
  indexes are now negative-cached as {} rather than removed.
- ensure_schema() finally block: only remove task from _loading if
  task.done(), so caller cancellation doesn't break the shared in-flight
  task for other awaiters when asyncio.shield() keeps it running.
…ndex, inclusive quantiles

- ensure_schema(): attach done-callback to clean up _loading when all
  awaiters cancel and no finally block runs. Prevents leaked task entries.
- translate_parsed(): raise ValueError('Unknown index: ...') early when
  get_schema() returns empty dict (negative cache), instead of deferring
  the error to Redis execution time.
- Benchmark p95/p99: use method='inclusive' so statistics.quantiles()
  works for small sample sizes (n < 100).
- Missing indexes are no longer cached as {}; each access retries FT.INFO,
  allowing automatic recovery when an index is created after first miss.
- Replaced asyncio.ensure_future() with asyncio.create_task() to match
  dict[str, asyncio.Task] annotation.
- Updated tests: TestMissingIndexRecovery verifies transient miss recovery
  including late-created indexes.
- Cleaned up docstrings, comments, and PR notes.
@nkanu17 nkanu17 force-pushed the feature/lazy-schema-caching branch from 19fb77b to 4aa968e Compare March 30, 2026 20:06
@nkanu17 nkanu17 requested a review from Copilot March 30, 2026 20:15
@nkanu17
Copy link
Copy Markdown
Contributor Author

nkanu17 commented Mar 30, 2026

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql_redis/schema.py
Comment thread sql_redis/schema.py
Comment thread tests/test_schema_caching.py
Comment thread tests/test_schema_caching.py
Comment thread tests/test_schema_caching.py
Copy link
Copy Markdown
Contributor

@rbs333 rbs333 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! Only thing I would consider adding is a per query view to the benchmark since that contextualizes well but generally awesome

@nkanu17 nkanu17 merged commit 91f30e3 into main Apr 1, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants