Ensure schemas can apply defaults when inserting

[DawidWraga]

see #207 (comment) for details

[changeset-bot]

🦋 Changeset detected

Latest commit: 64deab6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
@tanstack/db-collections	Patch
@tanstack/db	Patch
@tanstack/db-example-react-todo	Patch
@tanstack/react-db	Patch
@tanstack/vue-db	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

@tanstack/db-example-react-todo

npm i https://pkg.pr.new/@tanstack/db@209

npm i https://pkg.pr.new/@tanstack/db-collections@209

npm i https://pkg.pr.new/@tanstack/react-db@209

npm i https://pkg.pr.new/@tanstack/vue-db@209

commit: 64deab6

[KyleAMathews]

Hey it's looking good! A few failing tests to fix still...

[DawidWraga]

Hey I've fixed the previously failing tests but after merging in the new changes there's a bunch of different test failures. I'm moving houses tomorrow so I won't be able to address this for some time. If anyone has the moment to look into these conflicts I'd appreciate your help! Thanks

[KyleAMathews]

👍 I'll pick it up after lunch

[KyleAMathews]

@samwillis could you review my type changes? Particularly to the query builder? Don't want to mess with any guarantees you wanted. Our types are getting complicated... :-\

[DawidWraga]

Thanks for picking this up. I have an idea that could simplify some of the collection types:

export class CollectionImpl<
  T extends object = Record<string, unknown>,
  TKey extends string | number = string | number,
  // Replace `TExplicit` `TSchema` `TFallback` with TInsertInput:
  TInsertInput extends object = T,
> {
  // ...
}


export class Collection<
  T extends object = Record<string, unknown>,
  TKey extends string | number = string | number,
  TUtils extends UtilsRecord = {},
  // Replace `TExplicit` `TSchema` `TFallback` with TInsertInput:
  TInsertInput extends object = T,
> {
  // ...
}

export function createCollection<
  TExplicit = unknown,
  TKey extends string | number = string | number,
  TUtils extends UtilsRecord = {},
  TSchema extends StandardSchemaV1 = StandardSchemaV1,
  TFallback extends object = Record<string, unknown>,
>(options): Collection<
  ResolveType<TExplicit, TSchema, TFallback>,
  TKey,
  TUtils,
  // Pass the resolved type:
  CollectionInsertInput<TExplicit, TSchema, TFallback>
> {
  // ...
const collection = new CollectionImpl<
    ResolveType<TExplicit, TSchema, TFallback>,
    TKey
    // Pass the resolved type:
    CollectionInsertInput<TExplicit, TSchema, TFallback>
  >(options)

  return collection as Collection<
    ResolveType<TExplicit, TSchema, TFallback>,
    TKey,
    TUtils
    // Pass the resolved type:
    CollectionInsertInput<TExplicit, TSchema, TFallback>
  >
}

This is similar to how ResolveType already works, only passing the resolved type instead of the 3 generics.

[DawidWraga]

ah also I forgot to update the collection jsdoc string @templates :

// packages/db/src/collection.ts

/**
 * Enhanced Collection interface that includes both data type T and utilities TUtils
 * @template T - The type of items in the collection
 * @template TKey - The type of the key for the collection
 * @template TUtils - The utilities record type
 * @template TSchema - The schema type for validation and type inference
 */
export interface Collection<

[DawidWraga]

Also, I think both insert / update should be using the input type instead of output.

Then, since input would then not be specific to insert, it's probably worth updating the name:
CollectionInsertInput -> CollectionInput
TInsertInput                -> TInput

This might also be clearer:
ResolveType -> CollectionOutput
T                   -> TOutput

Here's a usecase of why it matters from gemini:

A key use case is handling Calculated/Derived Fields, where the system calculates a value that should not be settable directly by the user. Here's an example:

The Input takes a price and quantity, but the Output automatically calculates and includes the finalPrice.

• Zod Schema:

  import { z } from 'zod';
  
  const LineItemSchema = z.object({
    price: z.number().positive(),
    quantity: z.number().int().positive(),
    discountPercent: z.number().min(0).max(100).default(0),
  }).transform(data => {
    const finalPrice = data.price * data.quantity * (1 - data.discountPercent / 100);
    return {
      ...data,
      finalPrice: Math.round(finalPrice), // Use rounded integer for currency
    };
  });

• Substantial Difference for update:

  • ❌ Current Way (Partial<Output>): The draft includes finalPrice. A developer might mistakenly try to set it (draft.finalPrice = 170), which is incorrect logic.

  • ✅ Proposed Way (Partial<Input>): The draft would not have a finalPrice field. It would be impossible to set it directly, correctly forcing the developer to update a source field like draft.quantity = 3 and letting the transform logic do its job. This prevents a whole class of bugs.

  ---

Although, would it be better to handle derived fields in a live query collection anyway? I guess if you want it shared across many queries, and it's not an aggregate

[KyleAMathews]

Hmm DB doesn't have the concept of derived fields atm that the dev can't modify directly. Our general assumption is you'd calculate those in select in queries & share query fragments as needed. There is benefit to putting the derived field directly in the collection for sure to be easily reusable so we might want to do it in the future but it does complicate the collection implementation enough that my preference is to not do it unless there's a lot of interest.

[KyleAMathews]

I'll try out your type idea in a bit!

[samwillis]

@samwillis could you review my type changes? Particularly to the query builder? Don't want to mess with any guarantees you wanted. Our types are getting complicated... :-\

All the code in this file is completely re-written in the new query builder, so there is little point in making this perfect if its TTL is a better of days...

We will need to make similar changes in the new query builder - I think they looks fine, although I haven't dug into what the new generic args are for.

[KyleAMathews]

@samwillis ok let's wait until the query builder lands