Feasability of not having prisma pull query engine from mirror server

[puchm]

Hi,

I am having some big issues pulling the engine from the mirror server (https://binaries.prisma.sh). I have read that many people are having similar issues because they are behind corporate proxies or have issues with certificates. I get that there are ways around this as documented in some issues and documentation (#12162, docs).

For me the current status is that (since I can't possibly put my personal password into our CI/CD pipeline) I need to do some requests for them to put up an internal mirror (as documented here: https://www.prisma.io/docs/reference/api-reference/environment-variables-reference#prisma_engines_mirror).

This is slowing me down a lot - it even goes to the point where I need to consider switching to a different ORM.

I wanted to ask if there is any possibility we can build the query engines by ourself. I don't know anything about the way they are built but would be interested in hearing some thoughts.

[puchm]

I just saw that this is actually possible as documented in https://github.com/prisma/prisma-engines but it requires Rust which is too bad.

Why can't we have an npm package that just contains the binaries without needing to fetch them?

[austincrim]

Hey @puchm 👋 ,

Maintaining an NPM module with all the engines embedded would introduce a fairly large operational overhead due to the number of binaries and supported platforms. We do have the engines wrapper repo, but it just downloads binaries from binaries.prisma.sh under-the-hood.

You could manually download the necessary engines and then use them in CI with the PRISMA_QUERY_ENGINE_BINARY environment variable, but the burden of keeping the binaries up to date would be on you.

[austincrim]

How exactly can I find out which version I need? Choosing the right platform is simple but there's a lot of versions for every single one of those.

If you run npx prisma -v in your project, it will show the compatible binaries for your current Prisma Client version.

Also, what operational overhead are you referring to?

I am referencing the maintenance burden on us, the Prisma team, to have to create and maintain the extra NPM package in perpetuity.

[puchm]

Ok I get it. When I get some more free time I would like to look into creating a pipeline that automatically maintains such a repository, maybe we can get back to this some time in the future. Right now is not the time though.

It currently looks like we will not be using Prisma at all for this reason since we do not want to work around our IT security. Thankfully we don't have a lot of queries yet so it won't be a lot of work to switch to a different ORM.

I still find it interesting though that Prisma is the only package in our tech stack that fetches resources from the internet. It doesn't seem to be common practice. I get that building Prisma on the user's computer would introduce a dependency on Rust and probably some other stuff so distributing the binaries over the internet is still the way to go, but it would be nice to just have them on npm.

[janpio]

@puchm Am I understanding correctly that you can npm install any package on Npm, but connection to other servers to download files are prohibited?

[puchm]

Yes, you are. I don't quite understand why the restriction is necessary but with these things you often just have to accept it the way it is.

[janpio]

I understand.

That is an interesting situation, that indeed could be prevented by having an Npm package with the Engine file(s).

As we have 4+ Engines, and many platforms, and each commit of the engine repository creates new versions of these that would mean a lot of published Npm packages. So we never considered it.

But of course the creation process could be automated.

Then the manual work would be on your side: You would need to install the correct Npm packages in your project and configure the environment variables manually to point to the files in these Npm packages in your node_modules. Advanced features like binaryTargets would probably not work any more, but there would me manual workaround ("just" always install the correct Npm packages and set the correct env var values).

[leungandrew]

FWIW, I'm also experiencing the same exact problem.

In our enterprise Jenkins CI environment, it seems it isn't capable of reaching the binaries domain.

The problem is during a yarn install, the install scripts will go and attempt to download binaries from https://binaries.prisma.sh but hangs and eventaully gets ETIMEDOUT.

I'm trying to workaround this with setting the engine env variables, but even having set them, it seems the post install script will still run? Perhaps it's because I only set the ENV for the Query Engine and not the rest, and the post install script will continue to try and download the rest (and since it will always attempt to use the default https://binaries.prisma.sh, it'll continue to stall.

Perhaps the solution is to include ALL the engines, even the ones my app doesn't need for the host env?

[austincrim]

Are you also setting PRISMA_CLI_QUERY_ENGINE_TYPE=binary and PRISMA_CLIENT_ENGINE_TYPE=binary?

see here

[janpio]

Engine Type is unrelated to this @austincrim. The binaries hostname is just a legacy artifact and does not imply you have to use the "binary" engine type internally.

[janpio]

@leungandrew Does it work if you set only the env var PRISMA_ENGINES_MIRROR=http://prisma-builds.s3-eu-west-1.amazonaws.com/?

If you are going the other route of downloading all engine files and setting the env vars, please share where exactly and with which error message it is then still failing.

[SirMatGuy]

Hi everyone!!
I know i'm late to this,
But although i have the engines files downloaded and set them in an .env file, when running npx prisma init it will still try to fetch from binaries.prisma.sh.
We have our own artifact server to which i can put npm modules. so I can do npm install prisma etc.

[ludralph]

Hi there,

To keep our discussions organized and focused on the most relevant topics, we’re reviewing and tidying up our backlog. As part of this process, we’re closing discussions that haven’t had any recent activity and appear to be outdated.

If this discussion is still important to you or unresolved, we’d love to hear from you! Feel free to reopen it or start a new one with updated details.

For more details about our priorities and vision for the future of Prisma ORM, check out our latest blog post: https://www.prisma.io/blog/prisma-orm-manifesto.

Thank you for your understanding and being part of the community!