feat: add exists() support via FT.AGGREGATE APPLY/FILTER by nkanu17 · Pull Request #14 · redis-developer/sql-redis

nkanu17 · 2026-03-26T21:18:30Z

Summary

Translates SQL exists(field) to RediSearch exists(@field) in FT.AGGREGATE APPLY (projection) and FILTER (HAVING) clauses. Unlike ismissing(), exists() does not require INDEXMISSING on the field — it works on any indexed field. Forces FT.AGGREGATE when used. DIALECT 2 is appended automatically.

Changes

sql_redis/parser.py — Handle exp.Exists nodes in _process_select_expression_inner to route exists(column_ref) to computed_fields. Distinguish from SQL EXISTS (SELECT ...) subqueries. Add _process_having_clause to parse HAVING exists(field) into a filters list. Reject exists() in WHERE with a clear error.

sql_redis/analyzer.py — Extract field references from parsed.filters so fields inside exists() pass schema validation.

sql_redis/translator.py — Force FT.AGGREGATE when exists() is used. Generate APPLY "exists(@field)" AS alias for SELECT projections and FILTER "exists(@field)" for HAVING clauses. Include exists() fields in LOAD arguments.

Usage

SQL queries

from sql_redis.executor import Executor
from sql_redis.schema import SchemaRegistry

registry = SchemaRegistry(redis_client)
registry.refresh("myindex")
executor = Executor(redis_client, registry)

# Project field presence as 1/0
result = executor.execute(
    "SELECT name, exists(email) AS has_email FROM myindex"
)
for row in result.rows:
    print(row["name"], row["has_email"])  # "1" or "0"

# Multiple exists() projections
result = executor.execute(
    "SELECT name, exists(email) AS has_email, exists(phone) AS has_phone FROM myindex"
)

# Filter to only documents that have a field
result = executor.execute(
    "SELECT name, email FROM myindex HAVING exists(email)"
)

# Filter on NUMERIC field presence
result = executor.execute(
    "SELECT name, score FROM myindex HAVING exists(score)"
)

Translated commands

SQL	Redis command
`SELECT exists(email) AS has_email FROM idx`	`FT.AGGREGATE idx "*" LOAD 1 @email APPLY "exists(@email)" AS has_email DIALECT 2`
`SELECT name, exists(email) AS e, exists(phone) AS p FROM idx`	`FT.AGGREGATE idx "*" LOAD 3 @email @name @phone APPLY "exists(@email)" AS e APPLY "exists(@phone)" AS p DIALECT 2`
`SELECT name FROM idx HAVING exists(email)`	`FT.AGGREGATE idx "*" LOAD 2 @email @name FILTER "exists(@email)" DIALECT 2`

exists() vs ismissing()

Aspect	`exists()`	`ismissing()`
SQL syntax	`SELECT exists(field) AS ...` / `HAVING exists(field)`	`WHERE field IS NULL`
Redis function	`exists(@field)` → 1 or 0	`ismissing(@field)` → filter
Command	FT.AGGREGATE only	FT.SEARCH and FT.AGGREGATE
Context	APPLY (projection) / FILTER	Query string (filter)
INDEXMISSING needed?	No	Yes
Returns	`"1"` (present) / `"0"` (absent)	Matches/excludes documents

Error handling

exists() in WHERE: Raises ValueError with message: "exists() is a RediSearch aggregate function and cannot be used in WHERE clauses. Use HAVING exists(field) instead for post-aggregate filtering."
Unknown field: Analyzer raises ValueError if the field is not in the index schema.

Tests

8 integration tests (tests/test_exists.py) — SELECT projection (TAG, NUMERIC, multiple), HAVING filter, command structure verification, error handling
5 parser unit tests — exists in SELECT, auto-alias, multiple exists, HAVING, WHERE rejection
6 translator unit tests — APPLY generation, LOAD inclusion, multiple APPLY, FILTER generation, FILTER LOAD, DIALECT 2
350 total tests pass

Copilot

Pull request overview

Adds support for SQL exists(field) by translating it into RediSearch aggregation expressions (exists(@field)) and enabling use in both SELECT projections (via APPLY) and HAVING predicates (via FILTER) within FT.AGGREGATE.

Changes:

Extend the SQL parser to recognize exists(column) in SELECT and HAVING, and to reject exists() in WHERE.
Extend the analyzer to include fields referenced by HAVING filters in schema validation.
Update the translator to force FT.AGGREGATE, generate APPLY/FILTER for exists(), and ensure referenced fields are included in LOAD.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`sql_redis/parser.py`	Adds HAVING parsing into `filters` and SELECT handling for `exists(column)` computed fields.
`sql_redis/analyzer.py`	Attempts to include fields referenced in HAVING filters in schema validation.
`sql_redis/translator.py`	Forces aggregate path when HAVING filters exist; emits `FILTER` for `exists()` and loads fields referenced by `exists()`.
`tests/test_sql_parser.py`	Adds parser unit tests for `exists()` in SELECT/HAVING and WHERE rejection.
`tests/test_translator.py`	Adds translator unit tests for `exists()` APPLY/FILTER generation and LOAD behavior.
`tests/test_exists.py`	Adds end-to-end integration tests covering projection, HAVING filtering, and error cases.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Translate SQL IS NULL and IS NOT NULL to RediSearch ismissing(@field) and -ismissing(@field) respectively. Requires Redis 7.4+ with the INDEXMISSING attribute declared on the target field. Changes: - parser: handle sqlglot exp.Is nodes, emit IS_NULL / IS_NOT_NULL operators - query_builder: add build_missing_condition() for ismissing() syntax - translator: short-circuit IS_NULL/IS_NOT_NULL before field-type dispatch - translator: make DIALECT 2 the default for all FT.SEARCH and FT.AGGREGATE - executor: catch and re-raise ResponseError with clear version guidance when ismissing() fails (both sync and async paths) - translator: emit UserWarning on IS NULL/IS NOT NULL noting Redis 7.4+ and INDEXMISSING requirements Tests: - 18 integration tests against Redis 8 (TAG, TEXT, NUMERIC field types, combined conditions, edge cases, raw command verification) - Unit tests for parser, query_builder, and translator - Warning and error message verification tests

Translate SQL exists(field) to RediSearch exists(@field) in FT.AGGREGATE APPLY (projection) and FILTER (HAVING) clauses. - Parser: handle exp.Exists in SELECT → computed_fields and HAVING → filters, distinguishing from SQL EXISTS (SELECT ...) - Analyzer: extract field references from exists() for schema validation - Translator: force FT.AGGREGATE when exists() is used, generate APPLY/FILTER clauses, include exists() fields in LOAD - Reject exists() in WHERE with clear error (aggregate-only function) - No INDEXMISSING attribute required — works on any indexed field Tests: 8 integration, 5 parser, 7 translator (350 total pass)

…pressions like exists(a) + exists(b), ensuring the referenced fields are included in LOAD args and don't cause a Property not loaded runtime error.

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…UPBY/REDUCE - parser: unwrap exp.Paren in _process_having_clause so queries like HAVING (exists(email)) don't raise 'Unsupported HAVING expression' - translator: move exists() FILTER emission to after GROUPBY/REDUCE block so it acts as post-aggregation filtering (correct HAVING semantics) instead of pre-group document filtering

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Move import re, import asyncio, import warnings, import time, and several from-imports out of function/method bodies to module-level imports. Local imports inside TYPE_CHECKING guards and try/except compatibility blocks are intentionally kept.

Copilot

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

When FT.AGGREGATE is forced (e.g., by HAVING exists()), SELECT * was skipping the wildcard in the LOAD field collection loop, resulting in no fields being loaded and empty/partial rows. Now detects SELECT * in aggregate mode and emits LOAD * so RediSearch returns all document attributes. Adds test for SELECT * with HAVING exists() verifying LOAD * is emitted.

When FT.AGGREGATE is forced (e.g., by HAVING exists()), SELECT * was skipping the wildcard in the LOAD field collection loop, resulting in no fields being loaded and empty/partial rows. Now detects SELECT * in aggregate mode and emits LOAD * so RediSearch returns all document attributes.

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

nkanu17 requested review from Copilot and rbs333 March 26, 2026 21:23

Copilot started reviewing on behalf of nkanu17 March 26, 2026 21:24 View session

Copilot AI reviewed Mar 26, 2026

View reviewed changes

Comment thread sql_redis/parser.py

Comment thread sql_redis/analyzer.py

nkanu17 changed the base branch from feat/index-ismissing to main March 27, 2026 20:04

nkanu17 marked this pull request as ready for review March 27, 2026 20:04

nkanu17 added 3 commits March 27, 2026 16:53

chore: linter

59c7d9d

nkanu17 force-pushed the feat/exists branch from 66692e1 to c70bd97 Compare March 27, 2026 22:43

fix: handle sqlglot uppercasing exists() to EXISTS() in arithmetic ex…

1c24fab

…pressions like exists(a) + exists(b), ensuring the referenced fields are included in LOAD args and don't cause a Property not loaded runtime error.

nkanu17 requested a review from Copilot March 27, 2026 22:45

Copilot started reviewing on behalf of nkanu17 March 27, 2026 22:46 View session

nkanu17 force-pushed the feat/exists branch from c70bd97 to 1c24fab Compare March 27, 2026 22:47

Copilot AI reviewed Mar 27, 2026

View reviewed changes

Comment thread sql_redis/parser.py

Comment thread sql_redis/translator.py Outdated

nkanu17 requested a review from Copilot March 27, 2026 22:58

Copilot started reviewing on behalf of nkanu17 March 27, 2026 22:59 View session

Copilot AI reviewed Mar 27, 2026

View reviewed changes

Comment thread sql_redis/translator.py

nkanu17 requested a review from Copilot March 28, 2026 02:56

Copilot started reviewing on behalf of nkanu17 March 28, 2026 02:57 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

nkanu17 requested a review from Copilot March 28, 2026 03:04

Copilot started reviewing on behalf of nkanu17 March 28, 2026 03:05 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

fix: remove benchmark file accidentally committed to feat/exists

02c9dff

nkanu17 requested a review from Copilot March 28, 2026 03:23

Copilot started reviewing on behalf of nkanu17 March 28, 2026 03:23 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

Comment thread sql_redis/translator.py Outdated

Comment thread tests/test_exists.py

nkanu17 added 2 commits March 28, 2026 00:04

nkanu17 requested a review from Copilot March 28, 2026 04:06

Copilot started reviewing on behalf of nkanu17 March 28, 2026 04:07 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

nkanu17 merged commit b0e37cb into main Mar 30, 2026
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add exists() support via FT.AGGREGATE APPLY/FILTER#14

feat: add exists() support via FT.AGGREGATE APPLY/FILTER#14
nkanu17 merged 9 commits intomainfrom
feat/exists

nkanu17 commented Mar 26, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nkanu17 commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Usage

SQL queries

Translated commands

exists() vs ismissing()

Error handling

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nkanu17 commented Mar 26, 2026 •

edited

Loading