diff --git a/.changeset/between-endpoint-string-comparand.md b/.changeset/between-endpoint-string-comparand.md new file mode 100644 index 0000000000..efdd6b10ac --- /dev/null +++ b/.changeset/between-endpoint-string-comparand.md @@ -0,0 +1,58 @@ +--- +"@objectstack/spec": minor +--- + +fix(spec): `$between` accepts the ISO/clock strings the platform itself produces (#6571) + +The sibling half of #5685. Both of `$between`'s endpoints declared +`number | Date | FieldReference` — and the platform's own producers put a +**string** in them. As with the four ordering slots, the declaration did not +merely under-describe reality, it contradicted it, and in the one slot where a +date window is the natural spelling: + +- **The date-macro resolver descends into arrays.** `resolveFilterTokens` + (`@objectstack/core`, `filter-tokens.ts`) has an explicit array arm in its + `walk`, so a tuple comparand is resolved member by member, and every branch of + the resolver returns a string. `{ close_date: { $between: + ['{current_year_start}', '{current_year_end}'] } }` becomes + `{ close_date: { $between: ['2026-01-01', '2026-12-31'] } }` — two endpoints of + exactly the type this schema declared it refused. +- **This package's own conformance corpus already spells it.** + `temporal-conformance.ts`, the shared cross-driver expectation table, states + three `$between` cases with string endpoints: a `datetime` range with its + `{90_days_ago}`/`{today}` token twin, the degenerate single-day range, and + `{ at: { $between: ['08:00:00', '18:00:00'] } }` on a `Field.time` column. +- **The driver already normalises both ends per column type.** + `SqlDriver.coerceFilterValue` recurses through arrays member-wise, and + `calendarDayBetweenRewrite` coerces the min and rewrites a bare-calendar-day + max into the half-open `< next-day(max)` bound (#3777). + +**This is additive and declaration-side only.** No producer, caller or driver +changed, and no compile surface needed to: the endpoints were already being +normalised driver-side by column type, so every filter that validated before +still validates. + +Widened in all three places this contract is spelled — `RangeOperatorSchema` +(documentation), `FieldOperatorsSchema` (the copy `NormalizedFilterSchema` +validates against and `FieldOperators` is inferred from), and the `Filter` +TypeScript helper. #5685 moved the documentation copy first and had to come back +for the reachable one; both spellings move together here. + +In `Filter` the guard stays type-precise because `T` is known, mirroring the +ordering guard slot for slot: a `Date` field also takes the resolver's ISO +strings, a `string` field (a `Field.time` `'08:00:00'`, an autonumber code) is +rangeable instead of collapsing to `never`, and a `number` field stays +numbers-only. Each endpoint is widened independently, so a partially-resolved +range (`[Date, '2026-12-31']`) type-checks. + +**The endpoint form the contract guarantees** is the ISO/clock one — an ISO +calendar day (`YYYY-MM-DD`), a UTC ISO-8601 instant, or a wall-clock time of day +(`HH:MM[:SS[.fff]]`). Those are ASCII and fixed-width, so lexicographic order IS +chronological order and every backend agrees. The union is a bare `string` +rather than an ISO refinement for the reasons #5685 measured and this change +re-measured for the tuple: the schema is field-agnostic, an ISO refinement would +reject `Field.time`'s declared `HH:MM` form, and date-only vs full-timestamp is +already reconciled by the driver. Ranging over **non-temporal** text is +therefore permitted but not promised — the order is the backend collation's — +and nothing here promises the endpoints are ordered relative to each other: an +inverted `[max, min]` range is well-formed and matches nothing, at every backend. diff --git a/packages/spec/src/data/filter.test.ts b/packages/spec/src/data/filter.test.ts index df72150189..aee78f7c99 100644 --- a/packages/spec/src/data/filter.test.ts +++ b/packages/spec/src/data/filter.test.ts @@ -178,11 +178,123 @@ describe('RangeOperatorSchema', () => { }); it('should accept $between with date range', () => { - const filter = { + const filter = { $between: [new Date('2024-01-01'), new Date('2024-12-31')] as [Date, Date] }; expect(() => RangeOperatorSchema.parse(filter)).not.toThrow(); }); + + // ========================================================================== + // #6571 — BOTH endpoints accept the STRING the platform itself produces. + // + // Before this was pinned each endpoint union was `number | Date | + // FieldReference`, so every accepted shape below threw — including the shape + // this package's own `temporal-conformance.ts` corpus states as the expected + // cross-driver behaviour. `$between` is the sibling of the four ordering + // slots #5685 fixed; a range IS its two ordering bounds. See + // `RangeOperatorSchema`'s docblock for why the union is a bare `string` + // rather than an ISO refinement. + // + // These assert through `safeParse` rather than a bare `toThrow()` so a + // rejection names WHICH endpoint was refused: a tuple carries two independent + // unions and `toThrow()` cannot tell a min-side refusal from a max-side one. + // (The ADR-0112 `code`/`status` envelope does not apply here — these are Zod + // parse verdicts on a declaration surface, not runtime refusals.) + // ========================================================================== + + describe('string endpoints (#6571)', () => { + /** + * `resolveFilterTokens`' `walk` descends into arrays, and every branch of + * the resolver returns a string, so a token range resolves to two strings: + * `['{current_year_start}', '{current_year_end}']` -> `['2026-01-01', '2026-12-31']`. + */ + it('accepts the two calendar-day strings a token range resolves to', () => { + const resolved = RangeOperatorSchema.safeParse({ $between: ['2026-01-01', '2026-12-31'] }); + expect(resolved.success).toBe(true); + }); + + /** The sub-day tokens (`{now}`, `{N_hours_ago}`) emit a full `.toISOString()`. */ + it('accepts a range of full ISO instants', () => { + const result = RangeOperatorSchema.safeParse({ + $between: ['2026-08-08T04:32:56.000Z', '2026-08-09T04:32:56.000Z'], + }); + expect(result.success).toBe(true); + }); + + /** + * `temporal-conformance.ts`: "time: $between spans the working day + * inclusively" — `{ at: { $between: ['08:00:00', '18:00:00'] } }` on a + * `Field.time` column. `CLOCK_TIME_TYPES` declares this form NOT + * `Date.parse`-able, which is why an ISO refinement was rejected. + */ + it('accepts the wall-clock range the temporal conformance corpus pins', () => { + const result = RangeOperatorSchema.safeParse({ $between: ['08:00:00', '18:00:00'] }); + expect(result.success).toBe(true); + }); + + /** + * Endpoints are widened independently, which is what a partially-resolved + * range actually looks like — one literal `Date` and one resolved macro. + */ + it('accepts mixed endpoints, each union resolved on its own', () => { + expect(RangeOperatorSchema.safeParse({ + $between: [new Date('2026-01-01'), '2026-12-31'], + }).success).toBe(true); + expect(RangeOperatorSchema.safeParse({ + $between: ['2026-01-01', { $field: 'contract.end_date' }], + }).success).toBe(true); + }); + + it('still accepts numbers, Dates and field references — widening is additive', () => { + expect(RangeOperatorSchema.safeParse({ $between: [18, 65] }).success).toBe(true); + expect(RangeOperatorSchema.safeParse({ + $between: [new Date('2024-01-01'), new Date('2024-12-31')], + }).success).toBe(true); + expect(RangeOperatorSchema.safeParse({ + $between: [{ $field: 'a.min' }, { $field: 'a.max' }], + }).success).toBe(true); + }); + + /** + * The shapes that were rejected before stay rejected, and the failure is + * attributed to the offending endpoint rather than to the tuple as a whole. + */ + it('still rejects an endpoint that is rangeable at no backend', () => { + const boolMax = RangeOperatorSchema.safeParse({ $between: ['2026-01-01', true] }); + expect(boolMax.success).toBe(false); + expect(boolMax.error?.issues[0]?.path).toEqual(['$between', 1]); + + const objectMin = RangeOperatorSchema.safeParse({ $between: [{ nope: 1 }, '2026-12-31'] }); + expect(objectMin.success).toBe(false); + expect(objectMin.error?.issues[0]?.path).toEqual(['$between', 0]); + }); + + /** Arity is the tuple's own contract and is untouched by the widening. */ + it('still rejects a range that is not a two-element [min, max]', () => { + expect(RangeOperatorSchema.safeParse({ $between: ['2026-01-01'] }).success).toBe(false); + expect(RangeOperatorSchema.safeParse({ + $between: ['2026-01-01', '2026-06-30', '2026-12-31'], + }).success).toBe(false); + expect(RangeOperatorSchema.safeParse({ $between: '2026-01-01' }).success).toBe(false); + }); + + /** + * The documentation copy above and the ENFORCED copy must not drift: it is + * `FieldOperatorsSchema` that `NormalizedFilterSchema` validates against and + * that the exported `FieldOperators` type is inferred from. #5685 shipped + * the sibling ordering slots and had to come back for this same second + * spelling, so both are pinned here. + */ + it('is matched by the enforced copy — FieldOperatorsSchema and the normalized AST', () => { + expect(FieldOperatorsSchema.safeParse({ $between: ['2026-01-01', '2026-12-31'] }).success) + .toBe(true); + expect(FieldOperatorsSchema.safeParse({ $between: ['08:00:00', '18:00:00'] }).success) + .toBe(true); + expect(NormalizedFilterSchema.safeParse({ + $and: [{ close_date: { $between: ['2026-01-01', '2026-12-31'] } }], + }).success).toBe(true); + }); + }); }); // ============================================================================ @@ -556,6 +668,42 @@ describe('TypeScript Type System', () => { expect([resolvedMacro, stillTakesDate, clockTime, numeric]).toHaveLength(4); }); + /** + * #6571 — the TYPED half of the range contract, the exact mirror of the + * ordering block above. Checked by `pnpm typecheck`, NOT by the runtime + * expectation below: vitest never typechecks, so reverting `filter.zod.ts` + * leaves this test GREEN under vitest and RED under `tsc`. Measured on the + * reverted guard, the errors land in this block: + * `$between: ['2026-01-01', '2026-12-31']` on a Date field + * -> TS2322 Type 'string' is not assignable to type 'Date' + * `$between: ['08:00:00', '18:00:00']` on a string field + * -> TS2322 ... not assignable to type 'undefined' + * (the second reads `undefined`, not `never`, for the same reason #5685 + * recorded: the old guard's `never` meets the slot's own `?`, and an optional + * `never` IS `undefined`.) + */ + it('accepts a resolved macro range on a Date field and ranges string fields (#6571)', () => { + interface Deal { + close_date: Date; // a resolved token range arrives as two 'YYYY-MM-DD' + shift_start: string; // Field.time — 'HH:MM[:SS[.fff]]' + amount: number; + } + + const resolvedMacroRange: Filter = { close_date: { $between: ['2026-01-01', '2026-12-31'] } }; + const stillTakesDates: Filter = { + close_date: { $between: [new Date('2026-01-01'), new Date('2026-12-31')] }, + }; + // A partially-resolved range: endpoints are widened independently. + const halfResolved: Filter = { close_date: { $between: [new Date('2026-01-01'), '2026-12-31'] } }; + // The shape `temporal-conformance.ts` pins for a Field.time column. + const workingDay: Filter = { shift_start: { $between: ['08:00:00', '18:00:00'] } }; + // A number field stays numbers-only. + const numericRange: Filter = { amount: { $between: [1000, 5000] } }; + + expect([resolvedMacroRange, stillTakesDates, halfResolved, workingDay, numericRange]) + .toHaveLength(5); + }); + it('should support logical operators', () => { interface Task { title: string; diff --git a/packages/spec/src/data/filter.zod.ts b/packages/spec/src/data/filter.zod.ts index 0740bcfec4..6a62ba9356 100644 --- a/packages/spec/src/data/filter.zod.ts +++ b/packages/spec/src/data/filter.zod.ts @@ -225,16 +225,101 @@ export const SetOperatorSchema = lazySchema(() => z.object({ $nin: z.array(z.any()).optional(), })); +/** + * The endpoint contract shared by both of `$between`'s bounds (#6571). + * + * Module-private on purpose, exactly like {@link ORDERING_COMPARAND_DESCRIPTION}: + * it is documentation attached to a slot, not an authorable surface of its own, + * so it stays out of the exported API surface. The reasoning behind every + * sentence is in {@link RangeOperatorSchema}'s docblock. + */ +const RANGE_ENDPOINT_DESCRIPTION = + 'Closed interval [min, max]. Each endpoint is a number, a Date, a string, or ' + + 'a { $field } reference — the SAME union the ordering comparisons take, ' + + 'because a range IS its two ordering bounds. STRING is the form the ' + + 'platform itself produces: the date-macro resolver walks INTO arrays, so ' + + '{ $between: ["{current_year_start}", "{current_year_end}"] } resolves to ' + + 'two strings. The guaranteed spellings are an ISO calendar day ' + + '(YYYY-MM-DD), a UTC ISO-8601 instant, or a wall-clock time of day ' + + '(HH:MM[:SS[.fff]]) for a Field.time column; those are ASCII and ' + + 'fixed-width, so lexicographic order IS chronological order and every ' + + 'backend agrees. The driver reconciles each endpoint with the column ' + + 'independently (a bare calendar day used as the MAX becomes the half-open ' + + 'next-day boundary). Ranging over NON-temporal text is permitted but NOT ' + + 'promised: the order is the backend collation\'s, and those coincide only ' + + 'for ASCII.'; + /** * Range operator for interval checks (closed interval). * SQL: BETWEEN ? AND ? | MongoDB: $gte AND $lte + * + * Supported endpoint types: **Number, Date, ISO/clock STRING, FieldReference**. + * + * ## Why `string` is in BOTH endpoint unions (#6571) + * + * This is the same contradiction {@link ComparisonOperatorSchema} carried until + * #5685, in the one slot where it bites hardest. Until this was written down + * both endpoints read `number | Date | FieldReference` — and the platform's own + * producers put a STRING in them: + * + * - **The date-macro resolver descends into arrays.** `resolveFilterTokens` + * (`@objectstack/core`, `filter-tokens.ts`) evaluates the `{token}` grammar, + * and its `walk` has an explicit array arm (`if (Array.isArray(node)) return + * node.map(walk)`), so a tuple comparand is resolved member by member. Every + * branch of that resolver returns a string — `asYmd(…)` for a calendar day, + * `.toISOString()` for the sub-day tokens. So + * `{ close_date: { $between: ['{current_year_start}', '{current_year_end}'] } }` + * becomes `{ close_date: { $between: ['2026-01-01', '2026-12-31'] } }`, whose + * two endpoints were **exactly the type this schema declared it refused**. + * - **This package's own conformance corpus spells it.** + * `temporal-conformance.ts` — the shared cross-driver expectation table, in + * `packages/spec` itself — states three `$between` cases with string + * endpoints: `{ at: { $between: ['2026-04-29', '2026-07-28'] } }` with its + * `{90_days_ago}`/`{today}` token twin, the degenerate single-day range, and + * `{ at: { $between: ['08:00:00', '18:00:00'] } }` on a `Field.time` column. + * A declaration contradicted by the conformance table one directory over is + * not under-describing reality; it is disagreeing with it. + * - **The driver already normalises both ends per column type.** + * `SqlDriver.coerceFilterValue` recurses through arrays member-wise + * (`value.map(v => this.coerceFilterValue(table, field, v))`), and + * `calendarDayBetweenRewrite` coerces the min and rewrites a bare-calendar-day + * max into the half-open `< next-day(max)` bound — knex's `whereBetween` being + * inclusive on both ends, it inherits the same rule `$lte` has (#3777). + * + * A closed interval is the natural spelling of a **date window**, which makes + * this the slot an author — an AI author in particular — is most likely to + * reach for with the resolver's own output in hand, and the old declaration + * told them that output was invalid. + * + * ## Why a BARE string, and not an ISO-shaped refinement (#6571 rider ①) + * + * Identical to {@link ComparisonOperatorSchema}'s finding, and re-measured for + * the tuple: this schema is field-**agnostic** (it never sees which column the + * range applies to), an ISO refinement would reject the `HH:MM[:SS[.fff]]` form + * `field-value.zod.ts`'s `CLOCK_TIME_TYPES` declares and the conformance case + * above exercises, and date-only vs full-timestamp is already reconciled + * downstream by `calendarDayBetweenRewrite`. Endpoint-vs-column correctness is + * a field-TYPED judgement that already has an owner; re-guessing it here would + * refuse working ranges. + * + * ## What widening ADMITS, stated plainly + * + * `string` also admits ranges over NON-temporal text (`{ code: { $between: + * ['A', 'M'] } }`). That is real SQL and every backend answers it — but **the + * ORDER is the backend's, not this contract's**: `driver-sql` emits + * `whereBetween`, decided by the dialect's collation, while the JS matchers use + * UTF-16 code-unit order. Those coincide for ASCII and diverge outside it. + * Nothing here promises the two endpoints are ordered relative to each other + * either — an inverted `[max, min]` range is a well-formed filter that matches + * nothing, at every backend. */ export const RangeOperatorSchema = lazySchema(() => z.object({ /** Between (inclusive) - takes [min, max] array */ $between: z.tuple([ - z.union([z.number(), z.date(), FieldReferenceSchema]), - z.union([z.number(), z.date(), FieldReferenceSchema]) - ]).optional(), + z.union([z.number(), z.date(), z.string(), FieldReferenceSchema]), + z.union([z.number(), z.date(), z.string(), FieldReferenceSchema]) + ]).optional() + .describe(`Between (inclusive). ${RANGE_ENDPOINT_DESCRIPTION}`), })); // ============================================================================ @@ -419,11 +504,20 @@ export const FieldOperatorsSchema = lazySchema(() => z.object({ // Set & Range $in: z.array(z.any()).optional(), $nin: z.array(z.any()).optional(), + // Range. `string` is in BOTH endpoint unions for the reason + // {@link RangeOperatorSchema} gives at length (#6571): the date-macro resolver + // walks into arrays, so a token range resolves to two ISO/clock STRINGS, and + // this package's own `temporal-conformance.ts` corpus spells that shape. This + // copy is the ENFORCED one — `NormalizedFilterSchema` validates against it and + // the exported `FieldOperators` is inferred from it — so it must not drift + // from the documentation copy above. #5685 landed the sibling ordering slots + // in the documentation copy first and left the reachable surface still + // rejecting the platform's own output; both spellings move together. $between: z.tuple([ - z.union([z.number(), z.date(), FieldReferenceSchema]), - z.union([z.number(), z.date(), FieldReferenceSchema]) + z.union([z.number(), z.date(), z.string(), FieldReferenceSchema]), + z.union([z.number(), z.date(), z.string(), FieldReferenceSchema]) ]).optional(), - + // String-specific. Case-SENSITIVE, except `$icontains` which folds ASCII case // only — see {@link StringOperatorSchema} for the contract and its boundary. $contains: z.string().optional(), @@ -627,7 +721,25 @@ export type Filter = { $lte?: T[K] extends number ? number : T[K] extends Date | string ? T[K] | string : never; $in?: T[K][]; $nin?: T[K][]; - $between?: T[K] extends number | Date ? [T[K], T[K]] : never; + // Range (#6571). The TYPED half of what {@link RangeOperatorSchema} + // declares, and the exact mirror of the ordering guard above — a range + // IS its two ordering bounds, so the two must agree slot for slot: + // - a `Date` field also takes the ISO STRINGS the date-macro resolver + // produces for a token range (it walks into arrays), which the old + // `T[K] extends number | Date ? [T[K], T[K]]` guard rejected outright; + // - a `string` field (a `Field.time` `'08:00:00'`, an autonumber code) + // is rangeable at every backend, where the old guard collapsed it to + // `never` and made the operator unwritable — the very shape + // `temporal-conformance.ts` pins for `Field.time`; + // - a `number` field stays numbers-only — nothing here wants `['5','9']`. + // Each endpoint is widened independently, so a half-resolved range + // (`[new Date(...), '2026-12-31']`) type-checks, which is what a partial + // macro resolution actually hands the author. + $between?: T[K] extends number + ? [number, number] + : T[K] extends Date | string + ? [T[K] | string, T[K] | string] + : never; $contains?: T[K] extends string ? string : never; $notContains?: T[K] extends string ? string : never; $startsWith?: T[K] extends string ? string : never;