Prisma generate - Error: write EPIPE on WSL <-> Windows

[divyenduz]

Bug description

In the internal introspection CI: https://github.com/prisma/introspection-ci/issues/309#issuecomment-674075918

We ran into Error: write EPIPE with MacOS. Here is the internal commit: https://github.com/prisma/introspection-ci/pull/316/commits/392528c430679bfd42ac8d09e98e8d9403c1502c#diff-99e681890513b0f406b692a64ece0ddb

This has been seen before but was closed due to no reproduction: #1816

Environment:

• Multiple prisma generate commands were running in parallel
• One of the commands has Error: write EPIPE
• Some generate commands failed with no output (may or may not be related)

How to reproduce

• Probably try to run the same command on MacOS as the CI did. Introspection CI, branch fix_309

yarn -s generator-ci:prepare-generate
yarn -s generator-ci:run-generate

Prisma information

@prisma/cli          : 2.5.0-dev.52
Current platform     : darwin
Query Engine         : query-engine 8f08c6fa89afd00710e41d99621786150b7eebbe (at node_modules/@prisma/cli/query-engine-darwin)
Migration Engine     : migration-engine-cli 8f08c6fa89afd00710e41d99621786150b7eebbe (at node_modules/@prisma/cli/migration-engine-darwin)
Introspection Engine : introspection-core 8f08c6fa89afd00710e41d99621786150b7eebbe (at node_modules/@prisma/cli/introspection-engine-darwin)
Format Binary        : prisma-fmt 8f08c6fa89afd00710e41d99621786150b7eebbe (at node_modules/@prisma/cli/prisma-fmt-darwin)
Studio               : 0.254.0

[pantharshit00]

I am unable to run on my tiny macbook as it doesn't have enough memory. My laptop almost crashed when I ran this.

I need a more powerful machine or need to do this in a server. Maybe someone else with a more powerful machine from the team can confirm this instead.

[Jolg42]

Could not reproduce 🤷‍♂️

This is intense in RAM and takes a few minutes.

[ForbesLindesay]

I'm also getting this error, with what looks like the generate command running multiple times in quick succession:

[janpio]

Can you help us reproduce this @ForbesLindesay? An example project on Github would be gold here.

[ForbesLindesay]

It's in a pretty big project so not super easy to extract. It seemed to be watch mode only, and the setup I was using was a ubuntu docker image. We've disabled watch mode, which seems to have fixed the issue for now.

One suggestion: I think the files may end up appearing to be edited multiple times in quick succession. If you added a short (say 200ms) debounce to the file system watcher you are using, I suspect that would resolve things. Alternatively you could throttle the generate function to only run once at a time using something like https://www.npmjs.com/package/throat

[Jolg42]

@ForbesLindesay Indeed something like that is happening probably, thanks for the feedback!

[AlaaZorkane]

Any updates on this? I am using Prisma on WSL2 (Ubuntu2020) and each time I update my schema the watcher crashes with EPIPE

[pantharshit00]

Hey @AlaaZorkane

Does this happen on first save or after leaving it for some time and then trying to save the file?

[AlaaZorkane]

Hey @pantharshit00

Both, leaving it or not.
I tried saving with another editor (vim) and the watcher didn't crash, my theory is that it has something to do with both the Prisma VS code extension trying to lint/format and the watcher trying to read what's being written/updated, wyt?

Also another thing, the watcher crashes even if I make no changes to the file and save (causing it to still be linted/formatted by the prisma extension) so I guess that supports what I think it's happening?

EDIT: nvm, seems like the extension doesn't format/lint with no changes done to the file.

[pantharshit00]

I can kind of reproduce this and I think this is related to #4308

I will wait here for #4308

[pantharshit00]

Still blocked by #4308

I bumped that one.

[aqrln]

Haven't reproduced with the watch command yet, however the previous comments mention generating clients in parallel. When running multiple instances of prisma generate in parallel, I can reliably get another error: ENOENT: no such file or directory, unlink '/Users/aqrln/prisma/transient-repros/parallel-gen/node_modules/.prisma/client/index.js' and sometimes Unexpected end of JSON input.

[aqrln]

When using prisma generate --watch, I can't reproduce the EPIPE error either with the latest Prisma but I can confirm that we still don't properly throttle the filesystem events and start regenerating the client in parallel multiple times when the schema file is being saved often. This is something that we should improve regardless of whether this leads to errors because it is suboptimal (and prone to races: I haven't gone as far as to check if it actually happens, but it's quite plausible that one of the generators spawned earlier might sometimes take longer than the ones spawned later and will overwrite the client with one generated for an outdated schema).

[aqrln]

The error happens in GeneratorProcess#sendMessage when we try to call getManifest or generate but the generator process has exited because of an error or because it was killed.

Reproduced in tests in #19385. I'll fix it in that PR as well.