Prisma `node_modules` files are very large #14141

mmahalwy · 2022-07-05T06:39:29Z

Bug description

Prisma in node_modules is occupying quite a bit of storage on heroku. See:

39M	./node_modules/prisma/node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node
39M	./node_modules/prisma/libquery_engine-debian-openssl-1.1.x.so.node
39M	./node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node
36M	./node_modules/@prisma/sdk/dist/libquery_engine-debian-openssl-1.1.x.so.node
36M	./node_modules/@prisma/sdk/dist/get-generators/libquery_engine-debian-openssl-1.1.x.so.node
36M	./node_modules/@prisma/sdk/dist/get-generators/engines/461d6a05159055555eb7dfb337c9fb271cbd4d7e/libquery_engine-debian-openssl-1.1.x.so.node
36M	./node_modules/@prisma/sdk/dist/get-generators/engines/461d6a05159055555eb7dfb337c9fb271cbd4d7e
36M	./node_modules/@prisma/sdk/dist/get-generators/engines
36M	./node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node
35M	./node_modules/prisma/node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x
32M	./node_modules/prisma/node_modules/@prisma/engines/introspection-engine-debian-openssl-1.1.x
32M	./node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x
29M	./node_modules/@prisma/engines/introspection-engine-debian-openssl-1.1.x

How to reproduce

Prisma 4.0

Expected behavior

No response

Prisma information

Prisma 4.0

Environment & setup

OS: Heroku Dyno
Database: PostgreSQL
Node.js version: 16

Prisma Version

prisma                  : 4.0.0
@prisma/client          : 4.0.0
Current platform        : debian-openssl-1.1.x
Query Engine (Node-API) : libquery-engine da41d2bb3406da22087b849f0e911199ba4fbf11 (at node_modules/prisma/node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node)
Migration Engine        : migration-engine-cli da41d2bb3406da22087b849f0e911199ba4fbf11 (at node_modules/prisma/node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x)
Introspection Engine    : introspection-core da41d2bb3406da22087b849f0e911199ba4fbf11 (at node_modules/prisma/node_modules/@prisma/engines/introspection-engine-debian-openssl-1.1.x)
Format Binary           : prisma-fmt da41d2bb3406da22087b849f0e911199ba4fbf11 (at node_modules/prisma/node_modules/@prisma/engines/prisma-fmt-debian-openssl-1.1.x)
Default Engines Hash    : da41d2bb3406da22087b849f0e911199ba4fbf11
Studio                  : 0.465.0
Preview Features        : interactiveTransactions

The text was updated successfully, but these errors were encountered:

janpio · 2022-07-06T20:32:50Z

Do you have @prisma/sdk installed explicitly in your project?

mmahalwy · 2022-07-06T22:22:13Z

@janpio nope

janpio · 2022-07-07T20:35:07Z

What entries are in your package.json concerning Prisma?
What tool are you using to install your dependencies?

mmahalwy · 2022-07-07T20:36:43Z

@janpio using yarn 1 to install. What you mean entries?

janpio · 2022-07-07T20:50:00Z

Entries in dependencies or devDependencies.
Can you reproduce this problem in a new project? If so, could you upload it to GitHub so we can reproduce this ourselves?

mmahalwy · 2022-07-08T18:31:28Z

@janpio
This is a new directory, with only prisma in it:

{
  "dependencies": {
    "@prisma/client": "^4.0.0",
    "prisma": "^4.0.0",
    "prisma-docs-generator": "^0.5.0"
  }
}

Running du on all node_modules

> du -sh node_modules/* | sort -sh

 60K	node_modules/ignore
 60K	node_modules/is-ci
 60K	node_modules/minipass
 60K	node_modules/safer-buffer
 60K	node_modules/tmp
 64K	node_modules/emoji-regex
 64K	node_modules/function-bind
 64K	node_modules/glob
 64K	node_modules/micromatch
 64K	node_modules/send
 72K	node_modules/archiver
 72K	node_modules/compress-commons
 72K	node_modules/mime
 72K	node_modules/whatwg-url
 76K	node_modules/braces
 76K	node_modules/commander
 76K	node_modules/human-signals
 76K	node_modules/shell-quote
 80K	node_modules/execa
 80K	node_modules/fastq
 84K	node_modules/body-parser
 88K	node_modules/semver
 92K	node_modules/bl
 96K	node_modules/buffer
108K	node_modules/type-fest
112K	node_modules/http-proxy-agent
112K	node_modules/picomatch
112K	node_modules/readdir-glob
120K	node_modules/debug
120K	node_modules/temp-write
132K	node_modules/https-proxy-agent
136K	node_modules/agent-base
148K	node_modules/yargs-parser
164K	node_modules/node-fetch
180K	node_modules/readable-stream
188K	node_modules/object-inspect
196K	node_modules/tempy
216K	node_modules/mime-db
224K	node_modules/meow
228K	node_modules/fs-jetpack
232K	node_modules/tar
240K	node_modules/@timsuchanek
244K	node_modules/express
248K	node_modules/archiver-utils
248K	node_modules/fast-glob
256K	node_modules/@babel
256K	node_modules/qs
260K	node_modules/@nodelib
264K	node_modules/lazystream
272K	node_modules/tr46
276K	node_modules/ansi-escapes
292K	node_modules/prisma-docs-generator
308K	node_modules/ts-pattern
312K	node_modules/normalize-package-data
312K	node_modules/uuid
396K	node_modules/iconv-lite
400K	node_modules/prompts
420K	node_modules/resolve
692K	node_modules/read-pkg
860K	node_modules/checkpoint-client
1.0M	node_modules/undici
1.1M	node_modules/async
1.8M	node_modules/@types
3.6M	node_modules/prismjs
5.6M	node_modules/fp-ts
 98M	node_modules/prisma
197M	node_modules/@prisma

Running du on node_modules

> du -sh node_modules
324M	node_modules

nephix · 2022-09-03T09:15:51Z

Same here when installing with npm

{
  "dependencies": {
    "@prisma/client": "~4.3.0",
    "prisma": "~4.3.0",
  }
}

$ du -sh node_modules/* | sort -sh
...
 99M	node_modules/prisma
221M	node_modules/@prisma

$ du -sh node_modules/prisma/* | sort -sh
4.0K	prisma/package.json
8.0K	prisma/README.md
8.0K	prisma/scripts
 12K	prisma/LICENSE
 20K	prisma/install
 88K	prisma/preinstall
2.4M	prisma/prisma-client
 12M	prisma/build
 85M	prisma/node_modules

$ du -sh node_modules/@prisma/* | sort -sh
2.5M	@prisma/client
3.1M	@prisma/prisma-fmt-wasm
216M	@prisma/internals

hke-ghtest2 · 2022-09-21T13:57:19Z

Same here, would like to know why the size is so large?

kristinlindquist · 2022-11-03T15:55:27Z

Amongst other things, this large binary is in multiple places (not symlinks)

$ find node_modules/@prisma/ -name 'libquery_engine-debian-openssl-1.1.x.so.node'
node_modules/@prisma//internals/dist/libquery_engine-debian-openssl-1.1.x.so.node
node_modules/@prisma//internals/dist/get-generators/libquery_engine-debian-openssl-1.1.x.so.node
node_modules/@prisma//internals/dist/get-generators/engines/0362da9eebca54d94c8ef5edd3b2e90af99ba452/libquery_engine-debian-openssl-1.1.x.so.node

$ du -sch  node_modules/@prisma/internals/dist/get-generators/libquery_engine-debian-openssl-1.1.x.so.node
 40M    node_modules/@prisma/internals/dist/get-generators/libquery_engine-debian-openssl-1.1.x.so.node
 40M    total

Closes #15244 Related #14141 Before https://www.npmjs.com/package/@prisma/internals Unpacked Size 124 MB After, from `pnpm publish --dry-run --no-git-checks` npm notice package size: 77.7 kB npm notice unpacked size: 397.7 kB

Jolg42 · 2022-12-07T09:38:55Z

I created and merged a PR about @prisma/internals which included unneeded artifacts.
This is usually not an issue for Prisma users, as @prisma/client and prisma CLI packages bundle dependencies and do not include these artifacts.
But this is an issue when you have a direct dependency on @prisma/internals (previously @prisma/sdk) in your dependency graph. So either in your package.json or in the package.json of one of your dependencies, for example, prisma-docs-generator has a direct dependency on it and is affected.

@prisma/internals unpacked package went from 124.4 MB to 395.4 kB as a result of the PR (will be in next 4.8.0 release planned for December 20th)

janpio · 2022-12-20T18:41:02Z

Today's release 4.8.0 has significantly (average 50%, on some platforms even more) decreased the size of our engine files:

Improved serverless experience — smaller engines size

In this release, we have decreased the size of our engine files by an average of 50%. The size of the Query Engine used on Debian, with OpenSSL 3.0.x, for example, went from 39MB to 14MB. We will also remove some smaller engines to decrease the total size in future versions.

Additionally, we have started optimizing how the Prisma schema is loaded in Prisma Client. You should notice a considerable improvement when executing the first query if you're working with a bigger schema with many models and relations.

We will continue investing in this direction in the next releases and further improve the experience with Prisma and serverless environments.

https://github.com/prisma/prisma/releases/tag/4.8.0

We know we still have duplicated engine files in multiple locations, and that we are including too many of our engine files - but this was the obvious place to start and we are happy how significantly we could improve the situation already. Expect more updates like this in the coming releases from us.

janpio · 2023-05-26T00:28:46Z

As written before, we have reduced the size of our engines a lot and as far as we know, this solved this problem for most our user and their use cases. Hence I am closing this issue for now.

If you and your project are still having problems because of the remaining size of our engines, please open a new issue and describe your situation - we are happy to take a look and hopefully fix that. Thanks!

mmahalwy added the kind/bug A reported bug. label Jul 5, 2022

pimeys added bug/1-unconfirmed Bug should have enough information for reproduction, but confirmation has not happened yet. process/candidate team/client Issue for team Client. labels Jul 6, 2022

janpio added topic: dependencies topic: engine labels Jul 7, 2022

dpetrick added kind/improvement An improvement to existing feature and code. and removed bug/1-unconfirmed Bug should have enough information for reproduction, but confirmation has not happened yet. kind/bug A reported bug. process/candidate labels Jul 8, 2022

Jolg42 mentioned this issue Dec 6, 2022

fix(internals): remove libquery_engine from npm package #16649

Merged

janpio added the topic: performance/engine size formerly "binary size" label Dec 20, 2022

janpio changed the title ~~Prisma node_modules files are very large~~ Prisma node_modules files are very large Dec 20, 2022

janpio closed this as completed May 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prisma `node_modules` files are very large #14141

Prisma `node_modules` files are very large #14141

mmahalwy commented Jul 5, 2022

janpio commented Jul 6, 2022

mmahalwy commented Jul 6, 2022

janpio commented Jul 7, 2022

mmahalwy commented Jul 7, 2022

janpio commented Jul 7, 2022

mmahalwy commented Jul 8, 2022 •

edited

nephix commented Sep 3, 2022 •

edited

hke-ghtest2 commented Sep 21, 2022

kristinlindquist commented Nov 3, 2022 •

edited

Jolg42 commented Dec 7, 2022

janpio commented Dec 20, 2022 •

edited

janpio commented May 26, 2023

Prisma node_modules files are very large #14141

Prisma node_modules files are very large #14141

Comments

mmahalwy commented Jul 5, 2022

Bug description

How to reproduce

Expected behavior

Prisma information

Environment & setup

Prisma Version

janpio commented Jul 6, 2022

mmahalwy commented Jul 6, 2022

janpio commented Jul 7, 2022

mmahalwy commented Jul 7, 2022

janpio commented Jul 7, 2022

mmahalwy commented Jul 8, 2022 • edited

nephix commented Sep 3, 2022 • edited

hke-ghtest2 commented Sep 21, 2022

kristinlindquist commented Nov 3, 2022 • edited

Jolg42 commented Dec 7, 2022

janpio commented Dec 20, 2022 • edited

janpio commented May 26, 2023

Prisma `node_modules` files are very large #14141

Prisma `node_modules` files are very large #14141

mmahalwy commented Jul 8, 2022 •

edited

nephix commented Sep 3, 2022 •

edited

kristinlindquist commented Nov 3, 2022 •

edited

janpio commented Dec 20, 2022 •

edited