Performance regression with napi for high concurrency

[vjpr]

Bug description

See: #7382

No napi
{
  min: '25ms',
  max: '617ms',
  sum: '3m 17.5s',
  variance: '13.7s',
  mean: '198ms',
  stddev: '117ms',
  count: 1000,
  median: '160ms',
  p75: '203ms',
  p95: '518ms',
  p99: '601ms',
  p999: '617ms'
}
Napi
{
  min: '94ms',
  max: '436ms',
  sum: '5m 30.1s',
  variance: '1.5s',
  mean: '330ms',
  stddev: '40ms',
  count: 1000,
  median: '338ms',
  p75: '343ms',
  p95: '364ms',
  p99: '379ms',
  p999: '436ms'
}

How to reproduce

https://github.com/vjpr/benchmark-prisma

Expected behavior

No response

Prisma information

#7382

Environment & setup

• OS: macOS
• Database: PostgreSQL
• Node.js version: 14

Prisma Version

prisma               : 2.25.0-dev.3
@prisma/client       : 2.25.0-dev.3
Current platform     : darwin
Query Engine         : query-engine f3e341280d96d0abc068f97e959ddf01f321a858 (at ../../../nvm/versions/node/v14.2.0/pnpm-global/5/node_modules/.pnpm/@prisma+engines@2.24.0-30.f3e341280d96d0abc068f97e959ddf01f321a858/node_modules/@prisma/engines/query-engine-darwin)
Migration Engine     : migration-engine-cli f3e341280d96d0abc068f97e959ddf01f321a858 (at ../../../nvm/versions/node/v14.2.0/pnpm-global/5/node_modules/.pnpm/@prisma+engines@2.24.0-30.f3e341280d96d0abc068f97e959ddf01f321a858/node_modules/@prisma/engines/migration-engine-darwin)
Introspection Engine : introspection-core f3e341280d96d0abc068f97e959ddf01f321a858 (at ../../../nvm/versions/node/v14.2.0/pnpm-global/5/node_modules/.pnpm/@prisma+engines@2.24.0-30.f3e341280d96d0abc068f97e959ddf01f321a858/node_modules/@prisma/engines/introspection-engine-darwin)
Format Binary        : prisma-fmt f3e341280d96d0abc068f97e959ddf01f321a858 (at ../../../nvm/versions/node/v14.2.0/pnpm-global/5/node_modules/.pnpm/@prisma+engines@2.24.0-30.f3e341280d96d0abc068f97e959ddf01f321a858/node_modules/@prisma/engines/prisma-fmt-darwin)
Default Engines Hash : f3e341280d96d0abc068f97e959ddf01f321a858
Studio               : 0.397.0
Preview Features     : nApi, filterJson

[pantharshit00]

Hey @vjpr

Thanks for the reproduction, I can reproduce these results now.

{
  min: '10ms',
  max: '287ms',
  sum: '1m 12.1s',
  variance: '3.2s',
  mean: '72ms',
  stddev: '57ms',
  count: 1000,
  median: '55ms',
  p75: '58ms',
  p95: '254ms',
  p99: '281ms',
  p999: '287ms'
}
Napi
{
  min: '39ms',
  max: '178ms',
  sum: '2m 21.1s',
  variance: '292ms',
  mean: '141ms',
  stddev: '17ms',
  count: 1000,
  median: '145ms',
  p75: '146ms',
  p95: '147ms',
  p99: '165ms',
  p999: '178ms'
}

Looks like min and p75 has dropped while p99 and p999 has improve so this might be a slow startup issue.

We will have a look.

[vjpr]

[pimeys]

See perf fix stats in: napi-rs/napi-rs#596 (comment)

[pantharshit00]

Results after the fix with engines hash 78b0f2b031373a23869ef1ceace79ab171aa9273

No napi
{
  min: '10ms',
  max: '283ms',
  sum: '1m 10.7s',
  variance: '3.1s',
  mean: '71ms',
  stddev: '56ms',
  count: 1000,
  median: '55ms',
  p75: '59ms',
  p95: '232ms',
  p99: '278ms',
  p999: '283ms'
}
Napi
{
  min: '2ms',
  max: '192ms',
  sum: '48.9s',
  variance: '954ms',
  mean: '49ms',
  stddev: '31ms',
  count: 1000,
  median: '46ms',
  p75: '64ms',
  p95: '110ms',
  p99: '169ms',
  p999: '192ms'
}

It will be available in dev channel after #7422 is merged(its waiting on CI).

[pimeys]

Released in https://www.npmjs.com/package/prisma/v/2.25.0-dev.9

[pimeys]

@vjpr new graphs plz when you have time.

[vjpr]

[pantharshit00]

This made into stable in 2.24.1