diff --git a/README.md b/README.md index d1d8a7f03..f4ae3a8fa 100644 --- a/README.md +++ b/README.md @@ -10,110 +10,261 @@ ## What is ZenStack? -ZenStack is a toolkit for simplifying full-stack development with Node.js web frameworks like [Next.js](https://nextjs.org/), [Nuxt.js](https://nuxtjs.org/) and [SvelteKit](https://kit.svelte.dev/) , using Typescript language. +ZenStack is a toolkit for simplifying full-stack development with Node.js web frameworks like [Next.js](https://nextjs.org/), [Nuxt.js](https://nuxtjs.org/) , and [SvelteKit](https://kit.svelte.dev/) , using Typescript language. -Thanks to the increasing power of frameworks, it's becoming more and more practical to build a complex web app all within one unified framework. However, you'll still need to spend significantly amout of engergy to design and build up the backend part of your app. +Thanks to the increasing power of frameworks, building a complex web app within one unified framework is becoming more practical than ever. However, you'll still need to spend a significant amount of energy designing and building up your app's server-side part. -Things that make you stressful include: +Things that make you stressed include: - What kind of API to use? RESTful or GraphQL? - How to model your data and map the model to both source code and database (ORM)? -- How to implement common CRUD operations? Manually construct it or use a generator? +- How to implement CRUD operations? Manually construct it or use a generator? - How to evolve your data model? - How to authenticate users and authorize their requests? -ZenStack aims to simplify these tasks by providing an intuitive data modeling language to define data types and access policies, integrated with user authentication. It maps your data model to a relational database schema, generates RESTful services as well as a client-side library, allowing flexible and painless CRUD operations. +ZenStack aims to simplify these tasks by providing: -Typescript types are generated for data models, CRUD input, filters, etc., so that you get great coding experiences in IDE and have much fewer chances to make mistakes. +- An intuitive data modeling language for defining data types, relations, and access policies -![Diagram here] +```prisma +model User { + id String @id @default(cuid()) + email String @unique -Since CRUD APIs are automatically generated with access policies injected, you can implement most of your business logic in your front-end code safely. Read operations never return data that's not supposed to be visible for the current user, and writes will be rejected if unauthorized. The data-access client library also supports nested writes, which allows you to make a batch of creates/updates atomically, eliminating the needs for explicitly using a transaction. + // one-to-many relation to Post + posts Post[] +} -ZenStack is both heavily inspired and built above [Prisma ORM](https://www.prisma.io/), which is in our oppinion the best ORM toolkit in the market. Familarity with Prisma should make it very easy to pick up ZenStack, but it's not prerequisite since the modeling language is intuitive and the development workflow is straightforward. +model Post { + id String @id @default(cuid()) + title String + content String + published Boolean @default(false) + + // one-to-many relation from User + author User? @relation(fields: [authorId], references: [id]) + authorId String? + + // must signin to CRUD any post + @@deny('all', auth() == null) + + // allow CRUD by author + @@allow('all', author == auth()) +} +``` + +- Auto-generated CRUD services and strongly typed front-end library + +```jsx +// React example + +const { find } = usePost(); +const posts = get({ where: { public: true } }); +// only posts owned by current login user are returned +return ( + <> + {posts?.map((post) => ( + + ))} + +); +``` + +Since CRUD APIs are automatically generated with access policies injected, you can safely implement most of your business logic in your front-end code. Read operations never return data that's not supposed to be visible to the current user, and writes will be rejected if unauthorized. The generated front-end library also supports nested writes, allowing you to make a batch of creates/updates atomically, eliminating the need for explicitly using a transaction. + +ZenStack is heavily inspired and built over [Prisma](https://www.prisma.io) ORM, which is, in our opinion, the best ORM toolkit in the market. Familiarity with Prisma should make it easy to pick up ZenStack, but it's not a prerequisite since the modeling language is intuitive and the development workflow is straightforward. ## Getting started -### With Next.js +### [For Next.js](docs/get-started/next-js.md) + +### For Nuxt.js + +### For SvelteKit + +## How does it work? + +ZenStack has four essential responsibilities: -The easiest way to start using ZenStack is by creating a new Next.js project from a preconfigured starter template. +1. Modeling data and mapping the model to DB schema and program types +1. Integrating with authentication +1. Generating CRUD APIs and enforcing data access policies +1. Providing type-safe client CRUD library -Here we demonstrate the process with a simple Blog starter using [Next-Auth](https://next-auth.js.org/) for user authentication. +Let's briefly go through each of them in this section. -1. Make sure you have Node.js 16 or above and NPM 8 or above installed +### Data modeling -2. Create a new Next.js project from ZenStack starter +ZenStack uses a schema language called `ZModel` to define data types and their relations. The `zenstack` CLI takes a schema file as input and generates database client code. Such client code allows you to program against database in server-side code in a fully typed way without writing any SQL. It also provides commands for synchronizing data models with DB schema, and generating "migration records" when your data model evolves. -```bash -npx create-next-app [project name] --use-npm -e https://github.com/zenstackhq/nextjs-auth-starter +Internally, ZenStack entirely relies on Prisma for ORM tasks. The ZModel language is a superset of Prisma's schema language. When `zenstack generate` is run, a Prisma schema named 'schema.prisma' is generated beside your ZModel file. You don't need to commit schema.prisma to source control. The recommended practice is to run `zenstack generate` during deployment, so Prisma schema is regenerated on the fly. -cd [project name] -``` +### Authentication -3. Run ZenStack generator to generate data services, auth adapter, and the client library +ZenStack is not an authentication library, but it gets involved in two ways. -```bash -npm run generate -``` +Firstly, if you use any authentication method that involves persisting users' identity, you'll model the user's shape in ZModel. Some auth libraries, like [NextAuth](https://next-auth.js.org/), require user entity to include specific fields, and your model should fulfill such requirements. In addition, credential-based authentication requires validating user-provided credentials, and you should implement this using the database client generated by ZenStack. + +To simplify the task, ZenStack automatically generates an adapter for NextAuth when it detects that the `next-auth` npm package is installed. Please refer to [the starter code](https://github.com/zenstackhq/nextjs-auth-starter/blob/main/pages/api/auth/%5B...nextauth%5D.ts) for how to use it. We'll keep adding integrations/samples for other auth libraries in the future. + +Secondly, authentication is almost always connected to authorization. ZModel allows you to reference the current login user via `auth()` function in access policy expressions. Like, -4. Initialize your local db and creates the first migration - The starter is preconfigured with a local sqlite database. Run the following command to populate its schema and generates a migration history: +```prisma +model Post { + author User @relation(fields: [authorId], references: [id]) + ... -```bash -npm run db:migrate -- -n init + @@deny('all', auth() == null) + @@allow('all', auth() == author) ``` -5. Start the app +The value returned by `auth()` is provided by your auth solution via the `getServerUser` hook function you provide when mounting ZenStack APIs. Check [this code](https://github.com/zenstackhq/nextjs-auth-starter/blob/main/pages/api/zenstack/%5B...path%5D.ts) for an example. + +### Data access policy + +The primary value that ZenStack adds over a traditional ORM is the built-in data access policy engine. This allows most business logic to be safely implemented in front-end code. Since ZenStack delegates database access to Prisma, it enforces access policies by analyzing queries sent to Prisma and injecting guarding conditions. For example, suppose we have a policy saying "a post can only be accessed by its author if it's not published", expressed in ZModel as: + +```prisma +@@deny('all', auth() != author && !published) +``` -```bash -npm run dev +When client code sends a query to list all `Post`s, ZenStack's generated code intercepts it and injects the `where` clause before passing it through to Prisma (conceptually): + +```js +{ + where: { + AND: [ + { ...userProvidedFilter }, + { + // injected by ZenStack, "user" object is fetched from context + NOT: { + AND: [ + { author: { not: { id: user.id } } }, + { published: { not: true } }, + ], + }, + }, + ]; + } +} ``` -If everything worked correctly, you should have a blog site where you can signup, author drafts and publish them. +Similar procedures are applied to write operations and more complex queries involving nested reads and writes. To ensure good performance, ZenStack generates conditions statically, so it doesn't need to introspect ZModel at runtime. The engine also makes the best effort to push down policy constraints to the database to avoid fetching data unnecessarily and discarding afterward. -You can also try signing up multiple accounts and verify that drafts created by different users are isolated. +Please **beware** that policy checking is only applied when data access is done using the generated client-side hooks or, equivalently, the RESTful API. If you use `service.db` to access the database directly from server-side code, policies are bypassed, and you have to do all necessary checking by yourself. We've planned to add helper functions for "injecting" the policy checking on the server side in the future. -Checkout [the starter's documentation](https://github.com/zenstackhq/nextjs-auth-starter#readme) for more details. +### Type-safe client library -### With Nuxt.js +Thanks to Prisma's power, ZenStack generates accurate Typescript types for your data models: -![](https://img.shields.io/badge/-Coming%20Soon-lightgray) +- The model itself +- Argument types for listing models, including filtering, sorting, pagination, and nested reads for related models +- Argument types for creating and updating models, including nested writes for related models -### With SvelteKit +The cool thing is that the generated types are shared between client-side and server-side code, so no matter which side of code you're writing, you can always enjoy the pleasant IDE intellisense and typescript compiler's error checking. -![](https://img.shields.io/badge/-Coming%20Soon-lightgray) +## Programming with the generated code -## How does it work? +### Client-side -ZenStack has four essential responsibilities: +#### For Next.js -1. Mapping data model to db schema and program types (ORM) -1. Integrating with authentication -1. Generating CRUD APIs and enforcing data access policy checks -1. Providing type-safe client CRUD library +The generated CRUD services should be mounted at `/api/zenstack` route. React hooks are generated for calling these services without explicitly writing Http requests. -We'll briefly go through each of them in this section. +The following hooks methods are generated: -### ORM +- find: listing entities with filtering, ordering, pagination, and nested relations -### Authentication +```ts +const { find } = usePost(); +// lists unpublished posts with their author's data +const posts = find({ + where: { published: false }, + include: { author: true }, + orderBy: { updatedAt: 'desc' }, +}); +``` -### Data access policy checking +- get: fetching a single entity by id, with nested relations -### Type-safe client library +```ts +const { get } = usePost(); +// fetches a post with its author's data +const post = get(id, { + include: { author: true }, +}); +``` + +- create: creating a new entity, with the support for nested creation of related models + +```ts +const { create } = usePost(); +// creating a new post for current user with a nested comment +const post = await create({ + data: { + title: 'My New Post', + author: { + connect: { id: session.user.id }, + }, + comments: { + create: [{ content: 'First comment' }], + }, + }, +}); +``` -## Developing with the generated code +- update: updating an entity, with the support for nested creation/update of related models + +```ts +const { update } = usePost(); +// updating a post's content and create a new comment +const post = await update(id, { + data: { + const: 'My post content', + comments: { + create: [{ content: 'A new comment' }], + }, + }, +}); +``` -### Client-side +- del: deleting an entity + +```js +const { del } = usePost(); +const post = await del(id); +``` + +Internally ZenStack generated code uses [SWR](https://swr.vercel.app/) to do data fetching so that you can enjoy its caching, polling, and automatic revalidation features. -### Server-side usage +### Server-side -## Development workflow +If you need to do server-side coding, either through implementing an API endpoint or by using `getServerSideProps` for SSR, you can directly access the database client generated by Prisma: -## Database considerations +```ts +import service from '@zenstackhq/runtime'; + +export const getServerSideProps: GetServerSideProps = async () => { + const posts = await service.db.post.findMany({ + where: { published: true }, + include: { author: true }, + }); + return { + props: { posts }, + }; +}; +``` + +**Please note** that server-side database access is not protected by access policies. This is by-design so as to provide a way of bypassing the policies. Please make sure you implement authorization properly. ## What's next? +### [Learning the ZModel language](/docs/get-started/learning-the-zmodel-language.md) + +### [Evolving data model with migration](/docs/ref/evolving-data-model-with-migration.md) + +### [Database hosting considerations](/docs/ref/database-hosting-considerations.md) + ## Reach out to us for issues, feedback and ideas! [Discussions](../discussions) [Issues](../issues) [Discord]() [Twitter]() diff --git a/docs/get-started/learning-the-zmodel-language.md b/docs/get-started/learning-the-zmodel-language.md new file mode 100644 index 000000000..25b92c80f --- /dev/null +++ b/docs/get-started/learning-the-zmodel-language.md @@ -0,0 +1,347 @@ +# Learning the ZModel Language + +ZModel is a declarative language for defining data models, relations and access policies. +ZModel is a superset of [Prisma's Schema Language](https://www.prisma.io/docs/concepts/components/prisma-schema), so if you're already familiar with Prisma, feel free to jump directly to the [Access Policies](#access-policies) section. Otherwise, don't worry. The syntax is intuitive and easy to learn. + +## Data source + +Every model needs to include exactly one `datasource` declaration, providing information on how to connect to the underlying databases. + +The recommended way is to load the connection string from an environment variable, like: + +```prisma +datasource db { + provider = "postgresql" + url = env("DATABASE_URL") +} +``` + +Do not commit the `DATABASE_URL` value in source code; instead configure it in your deployment as an environment variable. + +## Data models + +Data models define the shapes of entities in your application domain. They include fields and attributes for attaching additional metadata. Data models are mapped to your database schema and used to generate CRUD services and front-end library code. + +Here's an example of a blog post model: + +```prisma +model Post { + // the mandatory primary key of this model with a default UUID value + id String @id @default(uuid()) + + // fields can be DateTime + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + // or string + title String + + // or integer + viewCount Int @default(0) + + // and optional + content String? + + // and a list too + tags String[] +} +``` + +As you can see, fields are typed and can be a list or optional. Default values can be attached with the `@default` attribute. You can find all built-in types [here](https://www.prisma.io/docs/reference/api-reference/prisma-schema-reference#model-field-scalar-types). + +Model fields can also be typed as other Models. We'll cover this in the [Relations](#relations) section. + +## Attributes + +Attributes attach additional metadata to models and fields. Some attributes take input parameters, and you can provide values by using literal expressions or calling attribute functions. + +Attributes attached to fields are prefixed with '@', and those to models are prefixed with '@@'. + +Here're some examples of commonly used attributes: + +```prisma +model Post { + // @id is a field attribute, marking the field as a primary key + // @default is another field attribute for specifying a default value for the field if it's not given at creation time + // uuid() is a function for generating a UUID + id String @id @default(uuid()) + + // now() is a function that returns current time + createdAt DateTime @default(now()) + + // @updatedAt is a field attribute indicating its value should be updated to the current time whenever the model entity is updated + updatedAt DateTime @updatedAt + + // @unique adds uniqueness constraint to field + slug String @unique + + // @map can be used to give a different column name in database + title String @map("my_title") + + // @@map can be used to give a different table name in database + @@map("posts") + + // uniqueness constraint can also be expressed via @@unique model attribute, and you can combine multiple fields here + @@unique([slug]) + + // use @@index to specify fields to create database index for + @@index([slug]) +} +``` + +Please refer to [Prisma's documentation](https://www.prisma.io/docs/reference/api-reference/prisma-schema-reference#attributes) for an exhaustive list of attributes and functions. + +## Relations + +The special `@relation` attribute expresses relations between data models. Here're some examples. + +- One-to-one + +```prisma +model User { + id String @id + profile Profile? +} + +model Profile { + id String @id + user @relation(fields: [userId], references: [id]) + userId String @unique +} +``` + +- One-to-many + +```prisma +model User { + id String @id + posts Post[] +} + +model Post { + id String @id + author User? @relation(fields: [authorId], references: [id]) + authorId String? +} +``` + +- Many-to-many + +```prisma +model Space { + id String @id + members Membership[] +} + +// Membership is the "join-model" between User and Space +model Membership { + id String @id() + + // one-to-many from Space + space Space @relation(fields: [spaceId], references: [id]) + spaceId String + + // one-to-many from User + user User @relation(fields: [userId], references: [id]) + userId String + + // a user can be member of a space for only once + @@unique([userId, spaceId]) +} + +model User { + id String @id + membership Membership[] +} + +``` + +## Access policies + +Access policies use `@@allow` and `@@deny` rules to specify the eligibility of an operation over a model entity. The signatures of the attributes are: + +```prisma +@@allow(operation, condition) +@@deny(operation, condition) +``` + +, where `operation` can be of: "all", "read", "create", "update" and "delete" (or a comma-separated string of multiple values, like "create,update"), and `condition` must be a boolean expression. + +The logic of permitting/rejecting an operation is as follows: + +- By default, all operations are rejected if there isn't any @@allow rule in a model +- The operation is rejected if any of the conditions in @@deny rules evaluate to `true` +- Otherwise, the operation is permitted if any of the conditions in @@allow rules evaluate to `true` +- Otherwise, the operation is rejected + +### Using authentication in policy rules + +It's very common to use the current login user to verdict if an operation should be permitted. Therefore, ZenStack provides a built-in `auth()` attribute function that evaluates to the `User` entity corresponding to the current user. To use the function, your ZModel file must define a `User` data model. + +You can use `auth()` to: + +- Check if a user is logged in + +```prisma +deny('all', auth() == null) +``` + +- Access user's fields + +```prisma +allow('update', auth().role == 'ADMIN') +``` + +- Compare user identity + +```prisma +// owner is a relation field to User model +allow('update', auth() == owner) +``` + +### A simple example with Post model + +```prisma +model Post { + // reject all operations if user's not logged in + @@deny('all', auth() == null) + + // allow all operations if the entity's owner matches the current user + @@allow('all', auth() == owner) + + // posts are readable to anyone + @allow('read', true) +} +``` + +### A more complex example with multi-user spaces + +```prisma +model Space { + id String @id + members Membership[] + owner User @relation(fields: [ownerId], references: [id]) + ownerId String + + // require login + @@deny('all', auth() == null) + + // everyone can create a space + @@allow('create', true) + + // owner can do everything + @@allow('all', auth() == owner) + + // any user in the space can read the space + // + // Here the ?[condition] syntax is called + // "Collection Predicate", used to check if any element + // in the "collection" matches the "condition" + @@allow('read', members?[user == auth()]) +} + +// Membership is the "join-model" between User and Space +model Membership { + id String @id() + + // one-to-many from Space + space Space @relation(fields: [spaceId], references: [id]) + spaceId String + + // one-to-many from User + user User @relation(fields: [userId], references: [id]) + userId String + + // a user can be member of a space for only once + @@unique([userId, spaceId]) + + // require login + @@deny('all', auth() == null) + + // space owner can create/update/delete + @@allow('create,update,delete', space.owner == auth()) + + // user can read entries for spaces which he's a member of + @@allow('read', space.members?[user == auth()]) +} + +model User { + id String @id + email String @unique + membership Membership[] + ownedSpaces Space[] + + // allow signup + @@allow('create', true) + + // user can do everything to herself; note that "this" represents + // the current entity + @@allow('all', auth() == this) + + // can be read by users sharing a space + @@allow('read', membership?[space.members?[user == auth()]]) +} + +``` + +### Accessing relation fields in policy + +As you've seen in the examples above, you can access fields from relations in policy expressions. For example, to express "a user can be read by any user sharing a space" in the `User` model, you can directly read into its `membership` field. + +```prisma + @@allow('read', membership?[space.members?[user == auth()]]) +``` + +In most cases, when you use a "to-many" relation in a policy rule, you'll use "Collection Predicate" to express a condition. See [next section](#collection-predicate-expressions) for details. + +### Collection predicate expressions + +Collection predicate expressions are boolean expressions used to express conditions over a list. It's mainly designed for building policy rules for "to-many" relations. It has three forms of syntaxes: + +1. Any + + ``` + ?[condition] + ``` + + Any element in `collection` matches `condition` + +2. All + + ``` + ![condition] + ``` + + All elements in `collection` match `condition` + +3. None + + ``` + ^[condition] + ``` + + None element in `collection` matches `condition` + +The `condition` expression has direct access to fields defined in the model of `collection`. E.g.: + +```prisma + @@allow('read', members?[user == auth()]) +``` + +, in condition `user == auth()`, `user` refers to the `user` field in model `Membership`, because the collection `members` is resolved to `Membership` model. + +Also, collection predicates can be nested to express complex conditions involving multi-level relation lookup. E.g.: + +```prisma + @@allow('read', membership?[space.members?[user == auth()]]) +``` + +In this example, `user` refers to `user` field of `Membership` model because `space.members` is resolved to `Membership` model. + +### A complete example + +Please check out the [Collaborative Todo](../../samples/todo) for a complete example on using access policies. + +## Summary + +This document serves as a quick overview for starting with the ZModel language. For more thorough explanations about data modeling, please check out [Prisma's schema references](https://www.prisma.io/docs/reference/api-reference/prisma-schema-reference). diff --git a/docs/get-started/next-js.md b/docs/get-started/next-js.md new file mode 100644 index 000000000..0471c6a88 --- /dev/null +++ b/docs/get-started/next-js.md @@ -0,0 +1,128 @@ +# Getting started for Next.js + +## Create a new project from a starter + +The easiest way to start using ZenStack is by creating a new Next.js project from a preconfigured starter template. + +Here we demonstrate the process with a simple Blog starter using [Next-Auth](https://next-auth.js.org/) for user authentication. + +1. Make sure you have Node.js 16 or above and NPM 8 or above installed + +2. Create a new Next.js project from the ZenStack starter + +```bash +npx create-next-app [project name] --use-npm -e https://github.com/zenstackhq/nextjs-auth-starter + +cd [project name] +``` + +3. Run ZenStack generator to generate data services, auth adapter, and the client library + +```bash +npm run generate +``` + +4. Initialize your local DB and create the first migration + The starter is preconfigured with a local sqlite database. Run the following command to populate its schema and generates a migration history: + +```bash +npm run db:migrate -- -n init +``` + +5. Start the app + +```bash +npm run dev +``` + +If everything worked correctly, you should have a blog site where you can signup, author drafts, and publish them. Congratulations! In case anything broke, [reach out to us](#reach-out-to-us-for-issues-feedback-and-ideas), and we'll help. + +You can also try signing up multiple accounts and verify that drafts created by different users are isolated. + +Checkout [the starter's documentation](https://github.com/zenstackhq/nextjs-auth-starter#readme) for more details. + +## Add to an existing project + +1. Install the CLI and runtime + +```bash +npm i -D zenstack + +npm i @zenstackhq/runtime @zenstackhq/internal +``` + +2. Install [VSCode extension](https://marketplace.visualstudio.com/items?itemName=zenstack.zenstack) for authoring the model file + +3. Create a "zenstack" folder under your project root, and add a "schema.zmodel" file in it + +4. Configure database connection in "schema.model" + + Here's an example of using a Postgres database with connection string specified in `DATABASE_URL` environment variable: + +```prisma +datasource db { + provider = 'postgresql' + url = env('DATABASE_URL') +} +``` + +5. Add models and define access policies as needed + +You can use the [sample model](https://github.com/zenstackhq/nextjs-auth-starter/blob/main/zenstack/schema.zmodel) in the start project as a reference. + +6. Generate server and client code from your model + +```bash +npx zenstack generate +``` + +7. Mount data-access API to endpoint + + Create file `pages/api/zenstack/[...path].ts`, with content: + +```ts +import { NextApiRequest, NextApiResponse } from 'next'; +import { + type RequestHandlerOptions, + requestHandler, +} from '@zenstackhq/runtime/server'; +import service from '@zenstackhq/runtime'; + +const options: RequestHandlerOptions = { + async getServerUser(req: NextApiRequest, res: NextApiResponse) { + // If you're using next-auth, current user can be fetched like: + // import { authOptions } from '@api/auth/[...nextauth]'; + // const session = await unstable_getServerSession(req, res, authOptions); + // return session?.user; + // + // Otherwise implement logic to return the user object for the current + // session. Database can be accessed with: "service.db". + }, +}; +export default requestHandler(service, options); +``` + +7. Initialize your local db and creates the first migration + +```bash +npm run db:migrate -- -n init +``` + +8. Start building your app by using the generated React hooks + E.g.: + +```jsx +import { usePost } from '@zenstackhq/runtime/hooks'; + +... + +const { get } = usePost(); +const posts = get({ where: { public: true } }); +return ( + <> + {posts?.map((post) => ( + + ))} + +); +``` diff --git a/docs/ref/database-hosting-considerations.md b/docs/ref/database-hosting-considerations.md new file mode 100644 index 000000000..438f8ed69 --- /dev/null +++ b/docs/ref/database-hosting-considerations.md @@ -0,0 +1,11 @@ +# Database Hosting Considerations + +ZenStack is agnostic about where and how you deploy your web app, but hosting on serverless platforms like [Vercel](https://vercel.com/) is definitely a popular choice. + +Serverless architecture has some implications on how you should care about your database hosting. Different from traditional architecture where you have a fixed number of long-running Node.js servers, in a serverless environment, a new Node.js context can potentially be created for each user request, and if traffic volume is high, this can quickly exhaust your database's connection limit, if you connect to the database directly without a proxy. + +You'll likely be OK if your app has a low number of concurrent users, otherwise you should consider using a proxy in front of your database server. Here's a number of (incomplete) solutions you can consider: + +- [Prisma Data Proxy](https://www.prisma.io/data-platform/proxy) +- [Supabase](https://supabase.com/)'s [connection pool](https://supabase.com/docs/guides/database/connecting-to-postgres#connection-pool) +- [Deploy pgbouncer with Postgres on Heroku](https://devcenter.heroku.com/articles/postgres-connection-pooling) diff --git a/docs/ref/evolving-data-model-with-migration.md b/docs/ref/evolving-data-model-with-migration.md new file mode 100644 index 000000000..140648c5e --- /dev/null +++ b/docs/ref/evolving-data-model-with-migration.md @@ -0,0 +1,63 @@ +# Evolving Data Model with Migration + +When using ZenStack, your schema.zmodel file represents the current status of your app's data model and your database's schema. When you make changes to schema.zmodel, however, your data model drifts away from database schema. At your app's deployment time, such drift needs to be "fixed", and so that your database schema stays synchronized with your data model. This processing of "fixing" is called migration. + +Here we summarize a few common scenarios and show how you should work on migration. + +## For a newly created schema.zmodel + +When you're just starting out a ZenStack project, you have an empty migration history. After creating schema.zmodel, adding a development datasource and adding some models, run the following command to bootstrap your migration history and synchronize your development database schema: + +```bash +npx zenstack migrate dev -n init +``` + +After it's run, you should find a folder named `migrations` created under `zenstack` folder, inside of which you can find a .sql file containing script that initializes your database. Please note that when you run "migration dev", the generated migration script is automatically run agains your datasource specified in schema.zmodel. + +Make sure you commit the `migrations` folder into source control. + +## After updating an existing schema.zmodel + +After making update to schema.zmodel, run the "migrate dev" command to generate an incremental migration record: + +```bash +npx zenstack migrate dev -n [short-name-for-the-change] +``` + +If any database schema change is needed based on the previous version of data model, a new .sql file will be generated under `zenstack/migrations` folder. Your development database's schema is automatically synchronized after running the command. + +Make sure you review that the generated .sql script reflects your intention before committing it to source control. + +## Pushing model changes to database without creating migration + +This is helpful when you're prototyping locally and don't want to create migration records. Simply run: + +```bash +npx zenstack db push +``` + +, and your database schema will be synced with schema.zmodel. After prototyping, reset your local database and generate migration records: + +```bash +npx zenstack migrate reset + +npx zenstack migrate dev -n [name] +``` + +### During deployment + +When deploying your app to an official environment (a shared dev environment, staging, or production), **DO NOT** run `migrate dev` command in CI scripts. Instead, run `migrate deploy`. + +```bash +npx zenstack migrate deploy +``` + +The `migrate deploy` command does not generate new migration records. It simply detects records that are created after the previous deployment and execute them in order. As a result, your database schema is synchronized with data model. + +If you've always been taking the "migrate dev" and "migrate deploy" loop during development, your migration should run smoothly. However manually changing db schema, manually changing/deleting migration records can result in failure during migration. Please refer to this documentation for [troubleshooting migration issues in production](https://www.prisma.io/docs/guides/database/production-troubleshooting). + +## Summary + +ZenStack is built over [Prisma](https://www.prisma.io) and it internally delegates all ORM tasks to Prisma. The migration workflow is exactly the same as Prisma's workflow, with the only exception that the source of input is schema.zmodel, and a Prisma schema is generated on the fly. The set of migration commands that ZModel CLI offers, like "migrate dev" and "migrate deploy", are simple wrappers around Prisma commands. + +Prisma has [excellent documentation](https://www.prisma.io/docs/concepts/components/prisma-migrate) about migration. Make sure you look into those for a more thorough understanding.