Supporte Metadata like description

[xcfox]

Valibot is a super cool Schema Builder! I am attracted by Valibot's modular design.

I plan to use Valibot as an Entity Schema Builder for MikroORM.

So far, I am using TypeScript's class and Decorators to define MikroORM's Entity like this:

@Entity()
export class BaseEntity {
  @PrimaryKey()
  id!: number

  @Property()
  createdAt = new Date()

  @Property({ onUpdate: () => new Date() })
  updatedAt = new Date()
}

interface WithName {
  name: string
}

@Entity()
export class User extends BaseEntity implements WithName {
  @Property({ unique: true, comment: "user's name" })
  name!: string

  @index()
  @Property({ nullable: true })
  email: string

  @Property({ type: 'text' })
  password!: string
}

Because TypeScript's class doesn't support multiple inheritance, I have to define the name field repeatedly.

Maybe I can use Valibot and custom pipelines to solve this problem:

import { intersect, Output, nullable, date, string, object } from 'valibot'
import { primaryKey, property, index, unique, description } from './custom'

const baseEntity = object({
  id: string([primaryKey()]),
  createdAt: date([property()]),
  updatedAt: date([property()]),
})

const withName = object({
  name: string([property(), index(), description("user's name")]),
})

export const userEntity = intersect([
  object({
    email: nullable(string([property(), unique()])),
    password: string([property()]),
  }),
  baseEntity,
  withName,
])

export interface IUser extends Output<typeof userEntity> {}

Here I use many custom pipelines: primaryKey, property, index, unique, description. However, the current version of Valibot's pipeline only supports Validation and Transformation. I don't think these custom pipes belong to either Validation or Transformation.

Once Metadata is supported, Valibot has the ability to shine in all scenarios where a Schema needs to be defined, such as graphql, typegoose, typeorm and not just for data validation like zod. Valibot's modularity, combined with Metadata, makes for a very powerful Schema Builder! All projects using TypeScript can use Valibot to define Schemas, and the development experience will be far superior to that of class and Decorators.

[fabian-hiller]

Yes, I am generally open to expand that. What exactly do property, index or unique return?

[fabian-hiller]

And we should also think about if the pipeline is the right location to define such metadata. Here is another idea for a method that does not change the schema but adds metadata to it:

import * as v from 'valibot';

const ProfileSchema = v.object({
  id: v.metadata(v.number(), [v.unique(), v.index()]),
  name: v.string([v.maxLength(30)]),
  age: v.number([v.minValue(18), v.maxValue(99)]),
  avatar: v.string([v.url()]),
  bio: v.string([v.maxLength(1000)]),
});

[xcfox]

property, index and unique are Metadata specially prepared for MikroORM, they should accept the same parameters as the decorators @Property(), @Index() and @Unique().

In addition, all MikroORM decorators for Entity definitions should have a corresponding Metadata method.

[xcfox]

And we should also think about if the pipeline is the right location to define such metadata. Here is another idea for a method that does not change the schema but adds metadata to it:

import * as v from 'valibot'

const ProfileSchema = v.object({
  id: v.metadata(v.number(), [v.unique(), v.index()]),
  name: v.string([v.maxLength(30)]),
  age: v.number([v.minValue(18), v.maxValue(99)]),
  avatar: v.string([v.url()]),
  bio: v.string([v.maxLength(1000)]),
})

I think the pipeline is the right place for Metadata.

As with decorators, Metadata, Validation, and Transformation are often placed in the same place.
For example, in TypeGraphQL:

Use decorators:

import { Length, MaxLength } from 'class-validator'
import { Field, InputType } from 'type-graphql'
import { type Recipe } from './recipe.type'

@InputType()
export class RecipeInput implements Partial<Recipe> {
  @Field()
  @MaxLength(30)
  title!: string

  @Field({ nullable: true })
  @Length(30, 255)
  description?: string
}

Use valibot:

import * as v from 'valibot'
import { field, inputType } from './valibot-type-graphql'
export const RecipeInput = v.object(
  {
    title: v.string([field(), v.maxLength(30)]),
    description: v.string([field(), v.length(30, 255)]),
  },
  [inputType()]
)

In fact, Validation and Transformation are special Metadata in the view of the decorator, and there are some use cases where Validation or Transformation are not needed,, such as when TypeGraphQL is used together with TypeORM:

Use decorators:

import { Field, ID, ObjectType } from 'type-graphql'
import { Column, Entity, OneToMany, PrimaryGeneratedColumn } from 'typeorm'
import { Recipe } from './recipe'

@ObjectType()
@Entity()
export class User {
  @Field((_type) => ID)
  @PrimaryGeneratedColumn()
  readonly id!: number

  @Field()
  @Column()
  email!: string

  @Field({ nullable: true })
  @Column({ nullable: true })
  nickname?: string

  @Column()
  password!: string

  @OneToMany((_type) => Recipe, (recipe) => recipe.author, { lazy: true })
  @Field((_type) => [Recipe])
  recipes!: Recipe[] | Promise<Recipe[]>
}

Use valibot:

import * as v from 'valibot'
import { ID } from 'type-graphql'
import { field, objectType } from './valibot-type-graphql'
import {
  column,
  entity,
  oneToMany,
  primaryGeneratedColumn,
} from './valibot-typeorm'
import { Recipe } from './recipe'

export const User = v.object(
  {
    id: v.number([primaryGeneratedColumn(), field(() => ID)]),
    email: v.string([column(), field()]),
    nickname: v.nullable(v.string([column(), field()])),
    password: v.string([column()]),
    recipes: v.array(Recipe, [
      oneToMany(() => Recipe, { lazy: true }),
      field(),
    ]),
  },
  [entity(), objectType()]
)

[fabian-hiller]

I understand, and from a DX perspective it makes perfect sense, but logically a pipeline sends the input through to validate and transform it, and metadata does not fit that idea. It's just something we should discuss.

[xcfox]

@fabian-hiller
Perhaps we should change the name for the current pipeline, which often implies manipulation of data.

But the usage scenario becomes much broader with the addition of metadata, which is additional information beyond the type.

In go, this is called struct tags:

type User struct {
    FirstName      string     `json:"fname"`
    LastName       string     `json:"lname"`
    Age            uint8      `validate:"gte=0,lte=130"`
    Email          string     `json:"e-mail" validate:"required,email"`
    FavouriteColor string     `validate:"hexcolor|rgb|rgba"`
    Addresses      []*Address `validate:"required,dive,required"` // a person can have a home and cottage...
    Gender         Gender     `json:"gender" validate:"required,gender_custom_validation"`
}

In Prisma, C#, Rust, this is called attributes:

model User {
  id        Int     @id @default(autoincrement())
  firstName String
  lastName  String
  email     String  @unique
  isAdmin   Boolean @default(false)

  @@unique([firstName, lastName])
}

In Kotlin, Java, this is called annotations:

@Fancy class Foo {
    @Fancy fun baz(@Fancy foo: Int): Int {
        return (@Fancy 1)
    }
}

In Python, TypeScript, this is called decorators.

It seems that decorators is already well known in TypeScript, so we need to avoid using that name. annotations in Java | Kotlin are usually used with class and method , which doesn't really fit the scenario where valibot is used as a schema builder.

attributes and tags seem like good names. What do you think?

[fabian-hiller]

Thank you for this detailed answer. I will think about it.

How does the ORM recognize that index() or unique() have been added to a schema?

[xcfox]

@fabian-hiller
In traditional TypeScript projects, the ORM uses Metadata Reflection to collect schema metadata.

There are three pre-built Metadata Providers in MikroORM. In order to make schema and meta-information defined using valibot(e.g. index() and unique()) available to MikroORM, we have two options:

1. implement Metadata Providers based on the valibot schema. this seems challenging as it requires a deep understanding of the MikroORM source code.

2. use MikroORM's built-in EntitySchema. This approach is relatively simple, requiring only the conversion of a valibot schema into an EntitySchema. We can implement a function called toEntitySchema to do this conversion, which looks like this:

import { object, intersect, string, number } from 'valibot'
import { property, primaryKey, toEntitySchema, unique, index } from 'valibot-mikro'
import { BaseEntitySchema } from './base-schema'

const User = intersect([
  object({
    id: number([primaryKey()]),
    fullName: string([property(), index()]),
    email: string([property(), unique()]),
    password: string([property()]),
  }),
  BaseEntitySchema,
])

// UserEntity is a real MikroORM Entity
const UserEntity = toEntitySchema(User)

const orm = await MikroORM.init({
  entities: [..., UserEntity],
  ...
});

toEntitySchema would do two things, first extract the metadata of the entity from the valibot schema and then convert the metadata to an EntitySchema.

Perhaps validot could have a getMetadata method built in, which would facilitate the development of third-party libraries.

I think common metadata can be provided directly by valibot, such as description(). description() is needed by most community libraries, e.g. valibot-json-schema

[fabian-hiller]

Thank you for your detailed answers. They help me a lot to make a faster decision. I will investigate the possibilities today or in the next few days and get back to you here. I will try to turn this discussion into an issue.