Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

impr(qe): PostgreSQL DISTINCT ON w/ ORDER BY #4525

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

Druue
Copy link
Contributor

@Druue Druue commented Dec 6, 2023

Added support for connector based results for tests where it makes sense.
OrderBy tests will always return data in specific orders and can therefore keep the snapshots.

Test Name should in DB is in DB note
empty_database 🟢 🟢
no_panic::pg 🟢 🟢
no_panic::_ 🔴 🔴
shorthand_works 🟢 🟢
with_duplicates::pg 🟢 🟢
with_duplicates::_ 🔴 🔴
with_orderby_basic 🟢 🟢 New!
with_orderby_non_matching 🔴 🔴
with_orderby_leftmost_non_matching 🔴 🔴
with_orderby_similar 🟢 🟢 New!
with_skip_basic 🔴 🔴 Adds implicit orderby on id
with_skip_orderby 🟢 🟢
nested_distinct_id::pg 🟠 🟠 Partial distinct in-db (nested query has invalid orderby)
nested_distinct_id::_ 🔴 🔴
nested_distinct_title::pg 🟢 🟢 New!
nested_distinct_title::_ 🔴 🔴
nested_distinct_reversed 🔴 🔴 id orderBy

Note! ⚠️

Per internal discussion, this PR will be scoped so as to only provide support for one-to-many relations. Support for many-to-many relations will be moved to a different PR

contributes prisma/prisma#14765
closes https://github.com/prisma/team-orm/issues/775
follow-up for #4223

@Druue Druue added this to the 5.8.0 milestone Dec 6, 2023
Copy link

codspeed-hq bot commented Dec 6, 2023

CodSpeed Performance Report

Merging #4525 will not alter performance

Comparing impr/distinct-orderby (1884b4c) with main (fecfd2f)

Summary

✅ 11 untouched benchmarks

@Druue Druue self-assigned this Dec 7, 2023
@Druue Druue marked this pull request as ready for review December 20, 2023 06:39
@Druue Druue requested a review from a team as a code owner December 20, 2023 06:39
@Druue Druue requested review from laplab, jkomyno and Weakky and removed request for a team December 20, 2023 06:39
Comment on lines 112 to 133
fn can_orderby_with_in_db_distinct(&self) -> bool {
match &self.distinct {
Some(distinct) => {
let dist_len = distinct.as_fields().len();

if self.order_by.len() <= dist_len {
Self::ord_matches_distinct(&self.order_by, distinct)
} else {
let initial_order = &self.order_by[0..dist_len];

Self::ord_matches_distinct(initial_order, distinct)
}
}
None => unreachable!(),
}
}

fn ord_matches_distinct(order_by: &[OrderBy], distinct: &FieldSelection) -> bool {
order_by
.iter()
.all(|ord| distinct.contains(ord.field().unwrap().name()))
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

orderBy support added here

Copy link
Contributor

github-actions bot commented Dec 20, 2023

WASM Size

Engine This PR Base branch Diff
Postgres 2.043MiB
Postgres (gzip) 809.457KiB
Mysql 2.024MiB
Mysql (gzip) 801.007KiB
Sqlite 1.985MiB
Sqlite (gzip) 787.742KiB

Copy link
Contributor

github-actions bot commented Dec 20, 2023

✅ WASM query-engine performance won't change substantially (1.002x)

Full benchmark report
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.19.0 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - 25000)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  315.55 ms/iter (313.18 ms … 320.27 ms) 318.07 ms 320.27 ms 320.27 ms
Web Assembly: Latest    384.53 ms/iter  (383.6 ms … 385.78 ms) 385.55 ms 385.78 ms 385.78 ms
Web Assembly: Current   385.59 ms/iter    (384 ms … 387.17 ms) 386.85 ms 387.17 ms 387.17 ms
Node API: Current       236.31 ms/iter (225.97 ms … 246.28 ms)  238.4 ms 246.28 ms 246.28 ms

summary for movies.findMany() (all - 25000)
  Web Assembly: Current
   1.63x slower than Node API: Current
   1.22x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.67 ms/iter   (12.36 ms … 13.46 ms)  12.75 ms  13.46 ms  13.46 ms
Web Assembly: Latest     16.17 ms/iter   (15.93 ms … 17.34 ms)  16.15 ms  17.34 ms  17.34 ms
Web Assembly: Current     16.5 ms/iter   (16.03 ms … 18.21 ms)  16.44 ms  18.21 ms  18.21 ms
Node API: Current        9,275 µs/iter   (8,997 µs … 9,599 µs)  9,370 µs  9,599 µs  9,599 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   1.78x slower than Node API: Current
   1.3x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,984 µs/iter   (1,860 µs … 3,098 µs)  1,968 µs  3,029 µs  3,098 µs
Web Assembly: Latest     2,517 µs/iter   (2,396 µs … 3,976 µs)  2,498 µs  3,523 µs  3,976 µs
Web Assembly: Current    2,498 µs/iter   (2,395 µs … 3,716 µs)  2,499 µs  2,857 µs  3,716 µs
Node API: Current        1,590 µs/iter   (1,428 µs … 2,458 µs)  1,587 µs  2,350 µs  2,458 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   1.57x slower than Node API: Current
   1.26x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.58 ms/iter   (12.29 ms … 12.86 ms)  12.66 ms  12.86 ms  12.86 ms
Web Assembly: Latest     16.83 ms/iter   (16.01 ms … 27.83 ms)  16.23 ms  27.83 ms  27.83 ms
Web Assembly: Current     16.1 ms/iter    (15.95 ms … 16.3 ms)  16.17 ms   16.3 ms   16.3 ms
Node API: Current        9,219 µs/iter   (8,924 µs … 9,714 µs)  9,280 µs  9,714 µs  9,714 µs

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.75x slower than Node API: Current
   1.28x slower than Web Assembly: Baseline
   1.05x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,983 µs/iter   (1,834 µs … 3,215 µs)  1,965 µs  3,145 µs  3,215 µs
Web Assembly: Latest     2,519 µs/iter   (2,405 µs … 3,375 µs)  2,522 µs  3,030 µs  3,375 µs
Web Assembly: Current    2,563 µs/iter   (2,422 µs … 3,914 µs)  2,558 µs  3,352 µs  3,914 µs
Node API: Current        1,550 µs/iter   (1,465 µs … 1,997 µs)  1,555 µs  1,914 µs  1,997 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.65x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline      13 ms/iter   (12.46 ms … 18.02 ms)  12.86 ms  18.02 ms  18.02 ms
Web Assembly: Latest     16.05 ms/iter    (15.9 ms … 16.18 ms)  16.11 ms  16.18 ms  16.18 ms
Web Assembly: Current    16.97 ms/iter   (15.97 ms … 25.48 ms)  16.26 ms  25.48 ms  25.48 ms
Node API: Current        9,397 µs/iter   (8,975 µs … 9,753 µs)  9,533 µs  9,753 µs  9,753 µs

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.81x slower than Node API: Current
   1.31x slower than Web Assembly: Baseline
   1.06x slower than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,919 µs/iter   (1,824 µs … 2,930 µs)  1,918 µs  2,323 µs  2,930 µs
Web Assembly: Latest     2,482 µs/iter   (2,379 µs … 3,032 µs)  2,487 µs  2,927 µs  3,032 µs
Web Assembly: Current    2,502 µs/iter   (2,392 µs … 3,999 µs)  2,487 µs  3,444 µs  3,999 µs
Node API: Current        1,559 µs/iter   (1,437 µs … 2,183 µs)  1,573 µs  2,040 µs  2,183 µs

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.61x slower than Node API: Current
   1.3x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  915.78 µs/iter  (856.77 µs … 1,451 µs) 917.23 µs  1,381 µs  1,451 µs
Web Assembly: Latest     1,221 µs/iter   (1,144 µs … 1,840 µs)  1,216 µs  1,806 µs  1,840 µs
Web Assembly: Current    1,204 µs/iter   (1,147 µs … 1,708 µs)  1,214 µs  1,460 µs  1,708 µs
Node API: Current       845.62 µs/iter  (794.67 µs … 1,053 µs)  858.4 µs 949.73 µs  1,053 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.42x slower than Node API: Current
   1.32x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  907.61 µs/iter  (855.03 µs … 1,426 µs) 907.79 µs  1,343 µs  1,426 µs
Web Assembly: Latest     1,228 µs/iter   (1,150 µs … 2,054 µs)  1,224 µs  1,987 µs  2,054 µs
Web Assembly: Current    1,217 µs/iter   (1,152 µs … 1,921 µs)  1,223 µs  1,525 µs  1,921 µs
Node API: Current        816.9 µs/iter  (751.15 µs … 1,165 µs) 849.86 µs 936.93 µs  1,165 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.49x slower than Node API: Current
   1.34x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

After changes in fe44013

@Druue Druue changed the title impr(qe): PostgreSQL DISTINCT ON w/ ORDER BY impr(qe): PostgreSQL DISTINCT ON w/ ORDER BY Dec 20, 2023
- Pass down parent from find_related
- Pass down then to finalize_arguments

Updated finalize_arguments
- Implictly add relation_ids to find_related distinct queries
@Jolg42 Jolg42 removed this from the 5.8.0 milestone Jan 10, 2024
@Jolg42 Jolg42 added this to the 5.9.0 milestone Jan 10, 2024
- append relation_id to end of distinct set
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants