binaryTargets not respected when PRISMA_QUERY_ENGINE_LIBRARY set

[jpentland]

Bug description

Since I am running on nixos, I must set PRISMA_QUERY_ENGINE_LIBRARY to point to the query engine library inside of the nix store.

However, I would also like to have other query engines downloaded, in order to package the tool for offline use on other platforms.

How to reproduce

1. Add prisma to project
2. Add someline like the following to schema.prisma:

generator client {
    provider      = "prisma-client-js"
    binaryTargets = ["debian-openssl-1.1.x", "debian-openssl-3.0.x", "darwin", "linux-musl", "windows"]
}

4. Run "prisma generate"
5. No other binaries downloaded to node_modules/prisma etc

Expected behavior

Would expect that the binaryTargets are downloaded: "debian-openssl-1.1.x", "debian-openssl-3.0.x", "darwin", "linux-musl", "windows"

Prisma information

generator client {
    provider      = "prisma-client-js"
    binaryTargets = ["debian-openssl-1.1.x", "debian-openssl-3.0.x", "darwin", "linux-musl", "windows"]
}

Environment & setup

• OS: nixos
• Database: sqlite
• Node.js version: v18.17.1

Prisma Version

prisma                  : 4.16.1
@prisma/client          : 4.16.1
Current platform        : linux-nixos
Query Engine (Node-API) : libquery-engine  (at ../../../../../../../../../nix/store/hmxi33qpcc3w2lk3z8n5v69pb3bcqd3i-prisma-engines-4.13.0/lib/libquery_engine.node, resolved by PRISMA_QUERY_ENGINE_LIBRARY)
Migration Engine        : migration-engine-cli  (at ../../../../../../../../../nix/store/hmxi33qpcc3w2lk3z8n5v69pb3bcqd3i-prisma-engines-4.13.0/bin/migration-engine, resolved by PRISMA_MIGRATION_ENGINE_BINARY)
Format Wasm             : @prisma/prisma-fmt-wasm 4.16.0-66.b20ead4d3ab9e78ac112966e242ded703f4a052c
Default Engines Hash    : b20ead4d3ab9e78ac112966e242ded703f4a052c
Studio                  : 0.484.0

[janpio]

This is most probably happening as we currently treat PRISMA_QUERY_ENGINE_LIBRARY as an alternative to downloading any engines. If you provide a local query engine file, you do not need to download anything - so our original thinking I would guess.

As a possible workaround:
You can probably run any other Prisma command without the env var set to trigger the download logic, and then run prisma generate with the env var set so that it works on NixOS. Does that work?

[jpentland]

It seems like every command either wants to download the nixos binary, and fails because it could not be found on the mirror, or doesn't download anything at all.

I'm not seeing anything that would only download other platforms.

[janpio]

Is there a simple way to run a NixOS setup to reproduce this? (I am on Windows, but have access to Linux machines in the Cloud of course)

[jpentland]

I created a repository with a simple ts/prisma example, and a shell.nix file which will allow you to reproduce the build environement, and a README.md:

git clone git@github.com:jpentland/nix-ts-prisma-example.git -b engines-example

Note that by entering the nix-shell, the prisma environment variables get set.

When I tested this on Archlinux, I noticed that removing the environment variables results in the platform being detected as debian-openssl.3.0.x and so downloading the engines succeeds, whereas on nixos proper it would fail.

If you want to replicate it exactly, you will need access to a nixos system. Any basic x86_64 setup would do, and should work without root access, since the nix shell will pull in all dependencies, and isolate the example from the system configuration.

[janpio]

$ npm run migrate

> prisma-example@1.0.0 migrate
> prisma migrate dev --preview-feature && prisma generate

Error: Env var PRISMA_SCHEMA_ENGINE_BINARY is provided but provided path /nix/store/990y1nyq6xxdix2pqyn7dp9w1cqzhvny-prisma-engines-5.0.0/bin/migration-engine can't be resolved.

I assume instructions are missing a step how to compile or provide this file somehow?

When I tested this on Archlinux, I noticed that removing the environment variables results in the platform being detected as debian-openssl.3.0.x and so downloading the engines succeeds, whereas on nixos proper it would fail.

That is also what I am seeing on Gitpod.io, which has Nix already installed.
Any good instructions to make this a "real nixos system" for the test?

[janpio]

I still have an update: I think you are getting the error message without the env var set, because you include native in the binaryTargets array. That resolves to nixos in your case and hence fails. If you remove that, you should be able to successfully run any CLI command that downloads the files, and then run the other command with the env var set for it to actually use the correct Query Engine for nixos that you provide.

[jpentland]

There should be an easy way to run it in qemu. I will look into that later when I have more time. For now I can show the output when I remove "native" on my nixos system:

Generate prisma schema definitions...
~/prisma-example
Warning Precompiled engine files are not available for nixos.
Prisma schema loaded from prisma/schema.prisma
Warning Precompiled engine files are not available for nixos.
Error: Failed to fetch sha256 checksum at https://binaries.prisma.sh/all_commits/b20ead4d3ab9e78ac112966e242ded703f4a052c/linux-nixos/libquery_engine.so.node.gz.sha256. 404 Not Found

If you need to ignore this error (e.g. in an offline environment), set the PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING environment variable to a truthy value.
Example: PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1

[janpio]

Unexpected to me. Definitely something to figure out.
What happens if you do set PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1? Another download error for the actual engine file?

[jpentland]

Yes