Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC: Generate a Prisma schema using migration metadata obtained from the sync service #872

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

alco
Copy link
Member

@alco alco commented Jan 23, 2024

Fixes VAX-1339, #863.

This is a proof-of-concept pull request that demonstrates the ability to obtain a Prisma schema which is later used in the type-safe client generation process without having Prisma connect to the Migrations proxy.

Overview of the included changes:

  • The SatOpMigrate.Column message gets a new is_nullable field. This information is needed to correctly indicate optional fields in the generated Prisma schema.
  • The SatOpMigrate message is extended to allow for inclusion of an enum type definition: a CREATE TYPE ... AS ENUM ... SQL statement and associated metadata (relevant commit range. @icehaunter @magnetised, I would love to get your feedback on this.
  • The introspectDB() function is reimplemented to generate a Prisma schema from the migrations metadata and write it to the temporary schema.prisma file.
  • The complete implementation of the new approach to generating a Prisma schema can be found in the clients/typescript/src/cli/migrations/migration-to-prisma-models.ts file.

How to proceed from here:

  • I'd like to have both the backend and the frontend teams look at this POC and point out any gotchas or missing functionality
  • Once we agree that this is a workable solution, I can open a new pull request that will only include changes to the protocol and the server-side part of the implementation
  • Someone on the frontend team can then open a pull request (based on the server one) that will take my client-side changes, clean them up and address all the bits that I've missed or implemented poorly due to my lack of TS expertise.

TODO:

  • deduplicate enum definitions in ddl_commands.

@alco alco force-pushed the alco/generate-prisma-schema branch 8 times, most recently from c9d3d76 to 774a5a9 Compare January 26, 2024 11:26
@alco alco changed the title wip: generate prisma schema POC: Generate a Prisma schema using migration metadata obtained from the sync service Jan 26, 2024
@electric-sql electric-sql deleted a comment from linear bot Jan 26, 2024
@alco alco mentioned this pull request Feb 7, 2024
@alco alco force-pushed the alco/generate-prisma-schema branch from 774a5a9 to 1a008de Compare February 14, 2024 14:26
@alco alco changed the base branch from main to alco/capture-enum-ddl February 14, 2024 14:26
Base automatically changed from alco/capture-enum-ddl to main February 19, 2024 13:07
@icehaunter icehaunter closed this Mar 13, 2024
@icehaunter icehaunter reopened this Mar 13, 2024
Use this app to rerun client generation as you're reworking the
generator implementation.

.electric_migrations_reference/prisma/schema.prisma includes a Prisma
schema obtained via `prisma db pull`. This is used as a reference for
the new implementation.
This removes the need to run `prisma db pull` and side-steps all
problems associated with it, caused by missing or incompatible
functionality in Electric Migrations proxy's "prisma introspection mode".
…ting Postgres

This commit shows the diff between the original Prisma schema
that was obtained via `prisma db pull` and the updated implementation
that only uses data obtained from the sync service's /api/migrations endpoint.
@alco alco force-pushed the alco/generate-prisma-schema branch from 1a008de to 002ccfa Compare April 17, 2024 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants