SDK Schema Generator

[br41nslug]

Description

fixes #18121

This PR introduces a TypeScript type generator to automatically build a schema based on your existing instance to be used with the SDK.

This includes both a new endpoint and a CLI tool that use the same logic for generating the types.

Note: both methods require a token with admin permissions

Endpoint

Implemented as endpoint for easy access to generate the SDK Schema for your instance.

GET /schema/sdk

Options

This endpoint supports the following query parameters:

• download will download the file instead of rendering in the browser
  Example: &download=my-schema.ts
• naming will transform the collection names
  Options: database,camelcase,pascalcase (default: database)
• root_name will set the name of the root schema type
  Example: &root_name=MyBetterName (default: MySchema)

CLI

Implemented as CLI for easy integration into projects and workflows.

# when running from the repo
pnpm run cli generate -h http://localhost:8055/ -t admin-token

# when running as package
npx directus-sdk-schema generate -h http://localhost:8055/ -t admin-token

Options

This endpoint supports the following query parameters:

• file will write to file instead of rendering in the cli
  Example: --file src/my-schema.ts
• naming will transform the collection names
  Options: database,camelcase,pascalcase (default: database)
  Example: --naming pascalcase
• root_name will set the name of the root schema type
  Example: --root-name=MyBetterName (default: MySchema)

Example Output

import { DirectusUsers } from '@directus/sdk';

interface CustomName {
    directus_users: directus_users[];
    languages: languages[];
    posts: posts[];
    posts_translations: posts_translations[];
    related: related[];
    related_m2a: related_m2a[];
    related_test_1: related_test_1[];
    test_1: test_1[];
    test_2: test_2[];
}

interface directus_users {
    custom: string | null;
}

interface languages {
    code: string;
    direction: string | null;
    name: string | null;
}

interface posts {
    id: number;
    csv: 'csv' | null;
    date_created: 'datetime' | null;
    date_updated: 'datetime' | null;
    m2o: posts | number | null;
    sort: number | null;
    status: string;
    user_created: DirectusUsers<CustomName> | string | null;
    user_updated: DirectusUsers<CustomName> | string | null;
    wy: string | null;
}

interface posts_translations {
    id: number;
    category: languages | string | null;
    languages_id: languages | string | null;
    posts_id: posts | number | null;
}

interface related {
    id: number;
    date_created: 'datetime' | null;
    date_updated: 'datetime' | null;
    m2a: related_m2a[] | number[];
    m2m: related_test_1[] | number[];
    o2m: test_2[] | number[];
    sort: number | null;
    status: string;
    user_created: DirectusUsers<CustomName> | string | null;
    user_updated: DirectusUsers<CustomName> | string | null;
}

interface related_m2a {
    id: number;
    collection: 'test_1' | 'test_2' | null;
    item: test_2 | test_1 | string | null;
    related_id: related | number | null;
}

interface related_test_1 {
    id: number;
    related_id: related | number | null;
    test_1_id: test_1 | number | null;
}

interface test_1 {
    id: number;
    alias: test_1[] | number[];
    bool: boolean | null;
    field1: string | null;
    moneyz: number | null;
    recursive: test_1 | number | null;
}

interface test_2 {
    id: string;
    date_created: 'datetime' | null;
    m2o: related | number | null;
    sort: number | null;
    status: string;
    tew: string | null;
    user_created: DirectusUsers<CustomName> | string | null;
}

[changeset-bot]

🦋 Changeset detected

Latest commit: abba3ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages

Name	Type
@directus/api	Patch
@directus/sdk-schema-generator	Major
@directus/types	Patch
directus	Patch
@directus/extensions-sdk	Patch
create-directus-extension	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[br41nslug]

@rijkvanzanten Should this package be licensed as MIT or BSL? 🤔

[rijkvanzanten]

Instead of having a sdk-schema-generator package, lets take this opportunity to standardize the schema generation for all formats. I'd much rather have a single schema-generator package that uses registrable plugins for typescript / graphql / openapi 🤩

[rijkvanzanten]

I have my concerns around exporting the type with a dependency that may or may not exist:

import { DirectusUsers } from '@directus/sdk';

Especially seeing that type is locked to the sdk (instead of something more generic) and might not match the permissions of the user that does the type generation

[br41nslug]

Especially seeing that type is locked to the sdk (instead of something more generic)

There are already multiple "generic typescript types" generators in our ecosystem which don't work 1 on 1 in the SDK. I'd rather have a generator for which i can confidently claim that the output will actually work with the SDK without manual intervention/correction.

and might not match the permissions of the user that does the type generation

Permissions are indeed problematic considering the required endpoints and services to build this Schema have hardcoded admin permission guards preventing users from getting non-admin schemas

I could however probably narrow the schema down to specific permissions after the complete schema has been generated.

[br41nslug]

I'd much rather have a single schema-generator package that uses registrable plugins for typescript / graphql / openapi

Can you elaborate on how you envision this? Not directly sure how "plugins" figure in, do want them all moved into a single package of "plugins" or want to abstract each generator into a separate package that plug into something else 🤔

[rijkvanzanten]

Can you elaborate on how you envision this? Not directly sure how "plugins" figure in, do want them all moved into a single package of "plugins" or want to abstract each generator into a separate package that plug into something else 🤔

I'm hoping we can do a similar thing to the storage location adapters or data abstraction drivers where we have a singular entry point that you pass the schema object and the language, and it'll use the plugin for that language to get the output 🤔

[noblica]

@br41nslug Hey thanks for replying.
I'm also curious about why you used the json literal as a type in some cases where the type was unknown, instead of just using unknown?

[br41nslug]

@br41nslug Hey thanks for replying. I'm also curious about why you used the json literal as a type in some cases where the type was unknown, instead of just using unknown?

Please do elaborate? I am not explicitly mapping unknown types at all i think

directus/packages/sdk-schema-generator/src/constants.ts

Line 18 in d9955df

export const fieldTypeMap: Record<string, TypeScriptTypes | TypeLiterals> = {

[noblica]

Specifically I mean the mapping here, on line 11: https://github.com/directus/directus/blob/d9955dfb6559987ca6372d130f086d8eb285df1c/packages/sdk-schema-generator/src/constants.ts#L11C6-L11C6

Wouldn't a unknown, or a Record<string, unknown> type be a better choice for something like this, instead of having it be a string literal "JSON"?

BTW please let me know if this discussion belongs somewhere else. Don't wanna crowd your PR, with unneeded discussion, but wasn't sure where else to ask these questions 🙂

[br41nslug]

Specifically I mean the mapping here, on line 11: https://github.com/directus/directus/blob/d9955dfb6559987ca6372d130f086d8eb285df1c/packages/sdk-schema-generator/src/constants.ts#L11C6-L11C6

Wouldn't unknown be a better choice for something like this, instead of having it be a string literal "JSON"?

BTW please let me know if this discussion belongs somewhere else. Don't wanna crowd your PR, with unneeded discussion, but wasn't sure where else to ask these questions 🙂

Discord is generally the place for more casual chat. Using unknown as type there would lose all context of the original field type thus leaving us without the ability to determine what functions are accepted on this field in type hinting (count() only currently but also json querying in the future). The literals are replaced with the appropriate JSON types for the return type (which are stricter than using unknown or any)

directus/sdk/src/types/output.ts

Line 110 in d9955df

export type FieldOutputMap = {

[nickrum]

Just curious, how would you use such a generated schema.ts file? Shouldn't the interfaces be exported or something? 🤔

[muratgozel]

the code here does its job but needs organizing i guess. having a specification called TypescriptSpecsService in https://github.com/directus/directus/blob/main/api/src/services/specifications.ts and creating a relevant endpoint in https://github.com/directus/directus/blob/main/api/src/controllers/server.ts would just do?

[br41nslug]

having a specification called TypescriptSpecsService and creating a relevant endpoint would just do?

@muratgozel Ignoring the naming, and specific locations, yes thats an approach. Not sure how that question relates to this PR tho as it's not deadlocked on implementation details but rather a conceptional/vision conflict.

[muratgozel]

okay, thanks for the clarification