feat(schema-cli): support multi file schema #4874

Weakky · 2024-05-17T09:36:53Z

Overview

Closes https://github.com/prisma/team-orm/issues/1039 - Follow up PR to #4809

Refactors most of the CLI inputs to take in a list of schema path + schema content (introspection will be done in a separate PR). Have a look at the .toml files to see the API changes
Handles multi-file schema from most of the commands (except introspection)

codspeed-hq · 2024-05-17T09:40:43Z

CodSpeed Performance Report

Merging #4874 will not alter performance

_{Comparing feat/multi-file-migrate (2fbc51b) with main (0af42cb)}

Summary

✅ 11 untouched benchmarks

github-actions · 2024-05-17T09:41:41Z

WASM Query Engine file Size

Engine	This PR	Base branch	Diff
Postgres	2.153MiB	2.153MiB	0.000B
Postgres (gzip)	846.057KiB	846.061KiB	-4.000B
Mysql	2.119MiB	2.119MiB	0.000B
Mysql (gzip)	831.746KiB	831.749KiB	-3.000B
Sqlite	2.015MiB	2.015MiB	0.000B
Sqlite (gzip)	793.078KiB	793.081KiB	-3.000B

github-actions · 2024-05-17T09:51:40Z

✅ WASM query-engine performance won't change substantially (1.012x)

Full benchmark report

DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.20.2 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - ~50K)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     372 ms/iter       (367 ms … 376 ms)    375 ms    376 ms    376 ms
Web Assembly: Latest       455 ms/iter       (450 ms … 463 ms)    458 ms    463 ms    463 ms
Web Assembly: Current      456 ms/iter       (453 ms … 459 ms)    459 ms    459 ms    459 ms
Node API: Current          199 ms/iter       (194 ms … 206 ms)    205 ms    206 ms    206 ms

summary for movies.findMany() (all - ~50K)
  Web Assembly: Current
   2.3x slower than Node API: Current
   1.23x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  15'349 µs/iter (14'545 µs … 21'131 µs) 15'137 µs 21'131 µs 21'131 µs
Web Assembly: Latest    18'454 µs/iter (17'899 µs … 20'897 µs) 18'543 µs 20'897 µs 20'897 µs
Web Assembly: Current   18'589 µs/iter (18'277 µs … 19'554 µs) 18'628 µs 19'554 µs 19'554 µs
Node API: Current        8'078 µs/iter   (7'869 µs … 8'477 µs)  8'159 µs  8'477 µs  8'477 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   2.3x slower than Node API: Current
   1.21x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   2'321 µs/iter   (2'214 µs … 3'574 µs)  2'315 µs  3'196 µs  3'574 µs
Web Assembly: Latest     2'885 µs/iter   (2'779 µs … 3'655 µs)  2'870 µs  3'522 µs  3'655 µs
Web Assembly: Current    2'913 µs/iter   (2'821 µs … 3'619 µs)  2'897 µs  3'309 µs  3'619 µs
Node API: Current        1'380 µs/iter   (1'282 µs … 1'800 µs)  1'401 µs  1'709 µs  1'800 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   2.11x slower than Node API: Current
   1.25x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     575 ms/iter       (565 ms … 594 ms)    582 ms    594 ms    594 ms
Web Assembly: Latest       778 ms/iter       (772 ms … 795 ms)    785 ms    795 ms    795 ms
Web Assembly: Current      786 ms/iter       (779 ms … 799 ms)    794 ms    799 ms    799 ms
Node API: Current          470 ms/iter       (463 ms … 478 ms)    476 ms    478 ms    478 ms

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.67x slower than Node API: Current
   1.37x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  80'484 µs/iter (78'741 µs … 83'895 µs) 82'309 µs 83'895 µs 83'895 µs
Web Assembly: Latest       111 ms/iter       (110 ms … 112 ms)    112 ms    112 ms    112 ms
Web Assembly: Current      111 ms/iter       (110 ms … 112 ms)    112 ms    112 ms    112 ms
Node API: Current       62'356 µs/iter (61'414 µs … 62'990 µs) 62'988 µs 62'990 µs 62'990 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.78x slower than Node API: Current
   1.38x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'020 ms/iter   (1'013 ms … 1'029 ms)  1'026 ms  1'029 ms  1'029 ms
Web Assembly: Latest     1'310 ms/iter   (1'285 ms … 1'328 ms)  1'328 ms  1'328 ms  1'328 ms
Web Assembly: Current    1'307 ms/iter   (1'295 ms … 1'321 ms)  1'320 ms  1'321 ms  1'321 ms
Node API: Current          885 ms/iter       (858 ms … 911 ms)    907 ms    911 ms    911 ms

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.48x slower than Node API: Current
   1.28x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     143 ms/iter       (142 ms … 145 ms)    145 ms    145 ms    145 ms
Web Assembly: Latest       183 ms/iter       (182 ms … 185 ms)    185 ms    185 ms    185 ms
Web Assembly: Current      185 ms/iter       (184 ms … 187 ms)    186 ms    187 ms    187 ms
Node API: Current          109 ms/iter       (107 ms … 111 ms)    111 ms    111 ms    111 ms

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.69x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'028 µs/iter     (977 µs … 1'859 µs)  1'030 µs  1'555 µs  1'859 µs
Web Assembly: Latest     1'376 µs/iter   (1'318 µs … 2'221 µs)  1'376 µs  1'814 µs  2'221 µs
Web Assembly: Current    1'460 µs/iter   (1'323 µs … 2'627 µs)  1'400 µs  2'535 µs  2'627 µs
Node API: Current          793 µs/iter     (741 µs … 1'297 µs)    806 µs    879 µs  1'297 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.84x slower than Node API: Current
   1.42x slower than Web Assembly: Baseline
   1.06x slower than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'027 µs/iter     (983 µs … 1'755 µs)  1'027 µs  1'412 µs  1'755 µs
Web Assembly: Latest     1'365 µs/iter   (1'298 µs … 2'379 µs)  1'362 µs  1'740 µs  2'379 µs
Web Assembly: Current    1'380 µs/iter   (1'312 µs … 2'052 µs)  1'380 µs  1'823 µs  2'052 µs
Node API: Current          781 µs/iter     (702 µs … 1'258 µs)    802 µs    913 µs  1'258 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.77x slower than Node API: Current
   1.34x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

After changes in 2fbc51b

remove dead code fix tests

schema-engine/core/src/lib.rs

schema-engine/core/src/state.rs

schema-engine/json-rpc-api-build/methods/common.toml

schema-engine/core/src/commands/diff.rs

libs/test-cli/src/main.rs

SevInf

LGTM, but let me finish CLI integration first so we won't break the client

Companion to prisma/prisma-engines#4874 Close prisma/team-orm#1040 /engine-branch feat/multi-file-migrate

* feat(migrate): Support split schema files in migrate Companion to prisma/prisma-engines#4874 Close prisma/team-orm#1040 /engine-branch feat/multi-file-migrate * Integrate engine update

Weakky force-pushed the feat/multi-file-migrate branch from bc8bb6e to 0ffe739 Compare May 21, 2024 10:00

feat(schema-cli): support multi file schema

0ab9913

remove dead code fix tests

Weakky force-pushed the feat/multi-file-migrate branch from 0ffe739 to 0ab9913 Compare May 21, 2024 10:01

Weakky added this to the 5.15.0 milestone May 21, 2024