Skip to content

Add raw SQL expression support for virtual fields#14

Merged
gblikas merged 8 commits intomainfrom
copilot/add-virtual-fields-improvement
Feb 24, 2026
Merged

Add raw SQL expression support for virtual fields#14
gblikas merged 8 commits intomainfrom
copilot/add-virtual-fields-improvement

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 24, 2026

Virtual fields previously only supported standard comparison expressions, preventing database-specific operations like JSONB array membership checks and computed fields based on date calculations.

Changes

Type System

  • Added IRawSqlExpression interface to parser/types.ts - encapsulates SQL generation via toSql(context) => SQL
  • Added IRawSqlContext interface to virtual-fields/types.ts - provides adapter, tableName, schema to SQL generators
  • Extended QueryExpression union to include raw expressions

Resolution & Translation

  • Updated virtual-fields/resolver.ts - pass through raw expressions unchanged
  • Updated translators/drizzle/index.ts - invoke toSql() method and return Drizzle SQL object
  • Updated security/validator.ts - handle raw expressions in all validation methods (skip field/value checks, count as one clause)

Helper Utilities (new file: virtual-fields/helpers.ts)

  • jsonbContains(field, value) - PostgreSQL JSONB @> operator for array membership
  • dateWithinDays(field, days) - date range check using NOW() - INTERVAL
  • Field name validation to prevent SQL injection (alphanumeric, dots, underscores; max 64 chars)

Usage

import { jsonbContains, dateWithinDays } from '@gblikas/querykit/virtual-fields';

const qk = createQueryKit({
  adapter: drizzleAdapter({ db, schema }),
  schema,
  virtualFields: {
    // JSONB array membership
    my: {
      allowedValues: ['assigned'] as const,
      resolve: (input, ctx) => jsonbContains('assigned_to', ctx.currentUserId)
    },
    // Computed field from timestamp
    priority: {
      allowedValues: ['high', 'medium', 'low'] as const,
      resolve: (input) => {
        const days = { high: 1, medium: 7, low: 30 }[input.value];
        return dateWithinDays('created_at', days);
      }
    }
  },
  createContext: async () => ({ currentUserId: await getCurrentUserId() })
});

// Now works:
await qk.query('tasks').where('my:assigned AND priority:high').execute();

Testing

Added virtual-fields/raw-sql.test.ts (38 tests) and user-example-integration.test.ts (4 tests) covering unit, integration, and end-to-end scenarios including field validation.

Original prompt

Problem

Virtual fields currently only support resolving to standard comparison expressions on schema fields. This limitation prevents users from implementing advanced query patterns such as:

  1. JSONB array contains checks - e.g., my:assigned where assignedTo is a JSONB array and we need to check if the current user's ID is in that array
  2. Computed/derived fields - e.g., priority:high based on createdAt timestamps, where "high" priority means created within the last day

User Example

A user has a Drizzle schema like:

export const my_table = createTable("my_table_type", {
  id: integer("id").primaryKey().generatedByDefaultAsIdentity(),
  title: varchar("title", { length: 256 }),
  description: varchar("description", { length: 1024 }),
  createdAt: timestamp("created_at").defaultNow(),
  // JSONB array of user IDs - "opaque" to standard QueryKit field resolution
  assignedTo: jsonb("assigned_to")
    .$type<string[]>()
    .default(sql`'[]'`),
});

They want to support queries like my:assigned which should check if ctx.currentUserId is contained in the assignedTo JSONB array.

Proposed Solution

Extend the virtual fields system to support raw SQL expressions that resolvers can return for database-specific operations.

1. Add new types in src/virtual-fields/types.ts

/**
 * Context provided to raw SQL generators for adapter-specific SQL generation.
 */
export interface IRawSqlContext {
  /**
   * The database adapter identifier (e.g., 'drizzle')
   */
  adapter: string;
  /**
   * The table name being queried
   */
  tableName: string;
  /**
   * Access to the schema for field references
   */
  schema: Record<string, unknown>;
}

/**
 * A raw SQL expression for database-specific operations.
 * Used when virtual fields need to generate complex SQL that can't be expressed
 * as simple field comparisons (e.g., JSONB operations, date ranges).
 */
export interface IRawSqlExpression {
  type: 'raw';
  /**
   * Function that generates the raw SQL for the adapter.
   * For Drizzle, this should return a SQL template result.
   */
  toSql: (context: IRawSqlContext) => unknown;
}

// Update ITypedQueryExpression to include raw expressions:
export type ITypedQueryExpression<TFields extends string = string> =
  | ITypedComparisonExpression<TFields>
  | IRawSqlExpression
  | QueryExpression;

2. Update resolver in src/virtual-fields/resolver.ts

Pass through raw expressions without modification:

export function resolveVirtualFields<...>(...): QueryExpression {
  if (expr.type === 'comparison') {
    return resolveComparisonExpression(expr, virtualFields, context);
  }

  if (expr.type === 'logical') {
    return resolveLogicalExpression(expr, virtualFields, context);
  }

  // Pass through raw expressions
  if (expr.type === 'raw') {
    return expr;
  }

  return expr;
}

3. Update Drizzle translator in src/translators/drizzle/index.ts

Handle raw expressions by calling their toSql method:

import { sql, SQL } from 'drizzle-orm';

// In translate method:
if (expression.type === 'raw') {
  const rawExpr = expression as IRawSqlExpression;
  return rawExpr.toSql({
    adapter: 'drizzle',
    tableName: this.currentTableName,
    schema: this.options.schema
  }) as SQL;
}

4. Add helper utilities (optional but recommended)

Provide common helpers in src/virtual-fields/helpers.ts:

import { sql } from 'drizzle-orm';
import { IRawSqlExpression } from './types';

/**
 * Create a JSONB array contains expression (PostgreSQL).
 * Checks if the JSONB array field contains the given value.
 */
export function jsonbContains(field: string, value: unknown): IRawSqlExpression {
  return {
    type: 'raw',
    toSql: () => sql`${sql.identifier(field)} @> ${JSON.stringify(Array.isArray(value) ? value : [value])}::jsonb`
  };
}

/**
 * Create a date range expression.
 * Checks if a timestamp field is within the specified number of days from now.
 */
export function dateWithinDays(field: string, days: number): IRawSqlExpression {
  return {
    type: 'raw',
    toSql: () => sql`${sql.identifier(field)} >= NOW() - INTERVAL '${sql.raw(days.toString())} days'`
  };
}

Example Usage After Implementation

import { createQueryKit } from '@gblikas/querykit';
import { drizzleAdapter } from '@gblikas/querykit/adapters/drizzle';
import { jsonbContains, dateWithinDays } from '@gblikas/querykit/virtual-fields';
import { sql } from 'drizzle-orm';

interface MyContext {
  currentUserId: string;
}

const qk = createQueryKit<typeof schema, MyContext>({
  adapter: drizzleAdapter({ db, schema }),
  schema,

  virtualFields: {
    // JSONB array contains example
    my: {
      allowedValues: ['assigned'] as const,
      description: 'Filter by your relationship to items',
      resolve: (input, ctx) => {
        if (input.value === 'assigned') {
          return jsonbContains('assigned_to...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented Feb 24, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
querykit Ready Ready Preview, Comment Feb 24, 2026 7:08pm

…al fields

Co-authored-by: gblikas <13577108+gblikas@users.noreply.github.com>
…tests

Co-authored-by: gblikas <13577108+gblikas@users.noreply.github.com>
Co-authored-by: gblikas <13577108+gblikas@users.noreply.github.com>
Co-authored-by: gblikas <13577108+gblikas@users.noreply.github.com>
Co-authored-by: gblikas <13577108+gblikas@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for advanced virtual fields in queries Add raw SQL expression support for virtual fields Feb 24, 2026
Copilot AI requested a review from gblikas February 24, 2026 05:52
@gblikas gblikas marked this pull request as ready for review February 24, 2026 18:59
Comment thread src/virtual-fields/helpers.ts
Comment thread src/virtual-fields/helpers.ts
vercel Bot and others added 2 commits February 24, 2026 19:07
…ys` parameter, allowing invalid values like Infinity, NaN, and negative numbers to generate invalid SQL.

This commit fixes the issue reported at src/virtual-fields/helpers.ts:64

## Bug Explanation

The `dateWithinDays` function in `src/virtual-fields/helpers.ts` accepts a `days` parameter without validating it. While the original issue description claimed this was a SQL injection vulnerability, the actual problem is more subtle:

1. **The SQL injection claim is technically incorrect**: JavaScript numbers converted to strings can only contain digits, decimals, and scientific notation characters - none of which can break out of the SQL string context to inject code.

2. **The real issue is input validation**: JavaScript allows special numeric values like `Infinity`, `-Infinity`, and `NaN`. When these are passed to the function, they generate invalid SQL:
   - `INTERVAL 'Infinity days'` - Invalid PostgreSQL syntax
   - `INTERVAL 'NaN days'` - Invalid PostgreSQL syntax
   - Negative numbers also logically don't make sense for a date range filter

3. **Impact**: This could cause query execution failures and suggests the function is not robust against unexpected inputs.

## Fix Explanation

I added a `validateDaysParameter()` function that:
- Checks that `days` is finite using `Number.isFinite()`
- Checks that `days` is positive (greater than 0)
- Throws descriptive error messages for invalid values

This follows the existing validation pattern used in the codebase for the `field` parameter via `validateFieldName()`. The fix:
- Prevents invalid SQL generation
- Makes the function more robust
- Maintains backward compatibility with all valid existing usage
- Adds comprehensive test coverage for edge cases

All existing tests pass, and new tests validate that invalid numeric values are properly rejected.

Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: gblikas <gblikas@gmail.com>
…reSQL SQL by inserting raw JSON strings without proper quoting, resulting in `field @> [...]::jsonb` instead of `field @> '[...]'::jsonb`

This commit fixes the issue reported at src/virtual-fields/helpers.ts:42

## Bug Analysis

**Why this is a bug:**
The jsonbContains function was generating SQL like:
```sql
assigned_to @> ["user123"]::jsonb
```

PostgreSQL interprets `[...]` as a SQL array literal syntax, not JSON. When PostgreSQL tries to cast a SQL array literal to JSONB, it fails because:
- `["user123"]` is not valid JSON (JSON uses string syntax, not array literals)
- The `@>` operator expects a JSONB value on the right side
- JSONB values must be constructed from string literals containing valid JSON

The correct SQL should be:
```sql
assigned_to @> '["user123"]'::jsonb
```

The string literal quotes are essential - without them, PostgreSQL cannot parse the JSON syntax.

**When it manifests:**
- Any time `jsonbContains()` is used to generate SQL, the query will fail with a PostgreSQL syntax error
- This affects all uses of virtual field features that rely on JSONB contains checks
- The error would occur at query execution time, not at build/parse time

**Impact:**
- Complete failure of JSONB-based filtering in virtual fields
- Users cannot use `jsonbContains()` helper in their queries
- This is a critical runtime bug that prevents core functionality

## Fix Analysis

**What changed:**
Modified the SQL generation in `jsonbContains()` to properly quote the JSON string:

```typescript
// Before:
sql`${sql.identifier(field)} @> ${JSON.stringify(...)}::jsonb`

// After:  
sql`${sql.identifier(field)} @> ${sql.raw("'" + JSON.stringify(...).replace(/'/g, "''") + "'::jsonb")}`
```

**Why this solves it:**
1. `JSON.stringify()` produces a valid JSON string (e.g., `["user123"]`)
2. Wrapping in single quotes creates a PostgreSQL string literal: `'["user123"]'`
3. Escaping single quotes by doubling them prevents SQL injection: `.replace(/'/g, "''")`
4. Appending `::jsonb` casts the string to the correct type
5. Using `sql.raw()` ensures the entire quoted string is inserted as-is into the template

**Why it's safe:**
- Field names are still protected by `sql.identifier()` 
- Values are already escaped by `JSON.stringify()` (which handles JSON encoding)
- Additional single quote escaping prevents any possible injection from values containing single quotes
- All tests (526) pass, confirming backward compatibility and correctness

Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: gblikas <gblikas@gmail.com>
@gblikas gblikas merged commit 990a29f into main Feb 24, 2026
6 checks passed
@gblikas gblikas deleted the copilot/add-virtual-fields-improvement branch February 24, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants