-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nix: When generating the client, prisma:get-platform
step infers incorrect libssl version in github action under ubuntu-latest
#20905
Comments
prisma:get-platform
step infers incorect libssl version in github action under ubuntu-latestprisma:get-platform
step infers incorrect libssl version in github action under ubuntu-latest
Anything special about your GitHub Action setup, or should we be able to reproduce this by creating a plain workflow that installs Prisma and tries to use it? If so, I think that is what we are already doing at https://github.com/prisma/ecosystem-tests, concretely this run with Node 18: https://github.com/prisma/ecosystem-tests/actions/runs/6041638512/job/16395076442. I created prisma/ecosystem-tests#3881 that adds your |
@janpio there must be something unique about our setup or I'd expect there to be a deluge of reports for this! Our project uses Nix to manage most of its dependencies (in our action, this is installed via https://github.com/cachix/install-nix-action); we have the |
That sounds very possible. I fear without a public workflow script we won't be able to really understand what is going on there. |
Let me see if I can create a public repo that reproduces the issue! It's in my best interest if my favorite libraries play well with my favorite operating system/package managers :) |
@janpio this public repo's action is demonstrating the "incorrect" openSSL resolution: https://github.com/haaksmash/prisma-20905/actions/runs/6043463892/job/16400508990#step:7:118 but... it's not crashing? somehow it's finding |
@janpio Reproduced the failure by adding an (essentially empty) seed script: https://github.com/haaksmash/prisma-20905/actions/runs/6043616043/job/16400924179 |
Oh, now I also see in your original post that the error is not when running the CLI in any way, but when Prisma Client tries to run some queries - which is also what happens in the seed script. Curious. I will take a look at the reproduction. |
@janpio hm, i'm not sure what you mean: from my perspective as a user, it's from running the CLI! The error is happening while running Plus, the seed script in the reproduction repository does not run any queries; it only constructs the prisma client and then disconnects from it. |
Yes, but if you add Still investigating why that crashes. |
Ok, so the problem is somehow related to the usage of Nix. When I remove the usage of Nix, the seed step works fine: https://github.com/haaksmash/prisma-20905/pull/1/files + https://github.com/janpio/prisma-20905/actions/runs/6116184645/job/16601005039. I know very little about Nix, does this make sense to you in any way? |
Can you shed some light on what mechanism prisma is using to infer the version of openSSL to reach for? i suspect the root of the problem is going to be found there. nix-related issues i've encountered with other tools and libraries are:
however, perhaps the simplest "fix" would be to have some way to override the inference of 1.1.x to 3.x. "Trust me bro, use this version of openssl". |
@janpio nudge here; I'm happy to keep looking but I'm not familiar with the inner workings of Prisma, especially once we cross the Rust boundary! |
I can tell you that this is not on the Rust side. On the TS side, I would start in this file and understand who calls what: https://github.com/prisma/prisma/blob/a04e3499742d7246d505cb04c342d188579404d1/packages/get-platform/src/getPlatform.ts |
Thank you! This is not relevant to debugging, but ironic in an Alanis Morisette way:
|
@janpio I just noticed that in your build-passing fork, the SSL version seems to be detected as 1.1.x; Does prisma choose the oldest-possible libssl...? https://github.com/janpio/prisma-20905/actions/runs/6116184645/job/16601005039#step:6:83 Are you able to confirm that, in a non-nix setup, prisma will find 3.0.x when that's what has been demanded? On my side, I've confirmed that the various SSL
so... curious that it's not finding what it needs when the files appear to be there. I will keep tracing execution to see if I can better understand why the |
Prisma chooses a libssl that is available and works, that might be an older one sometimes. Changing this behavior would be a breaking change, it usually does not cause trouble yet, so we left it unchanged.
Not sure what you are asking, but generally Prisma selects a OpenSSL version that works. We have only very few reports of this not working properly. |
Progress update: It seems like the change was introduced between 5.0.0 and 5.1.0. You can see that the build result is different between commits on this pull request: haaksmash/prisma-20905#3 time for some bisecting! |
It seems clear to me that something changed within Prisma's
This is what I expect to happen in Ubuntu 22.04; OpenSSL 3 is what's documented by Github. But in 5.1.0, the logs are different:
Could it be as simple as this change...? - return dirContents.find((value) => value.startsWith('libssl.so') && !value.startsWith('libssl.so.0'))
+ return dirContents.find((value) => value.startsWith('libssl.so.') && !value.startsWith('libssl.so.0')) It looks like that commit was partially reverted, in 27f4f0f, but it didn't revert that SSL matching string! @janpio wdyt? I think this is a pretty likely candidate! |
Yes, although I would assume that was intentional. Let's see. |
Thanks for the repro @haaksmash it was super useful to figure out a work-around! |
Do you understand why this one line change causes this problem on Nix systems? |
@janpio i haven't reproduced this in a non-nix context, but I suspect that the problem is that, within the nix shell, there is no lower version than 3 of OpenSSL available. Before this change, the prisma client would find After this change, it doesn't find I wonder if it would find |
@OzTK i'm not sure what your risk tolerance is, but it seems like I'm going to hem and haw for a little longer before deciding to use that patch in my own production system, but it does seem pretty innocuous... |
Help me out quickly: If it finds that file, shouldn't it then also be available? |
@janpio confusion is totally understandable! The This is why the build passes in your fork without the nix shell; released from the restrictions imposed by nix, the prisma client is able to access whatever is in the ubuntu image. |
Uh, does that mean that relying on the presence of files is not a good indicator with Nix OS? |
It's only an indicator that something put that file there. In a nix-shell, there is no access to binaries, libraries, or environment variables that have not been explicitly added. In a true NixOS system, I think this file would not be present in I don't think it's possible for a nix-shell, from within another operating system, to hide files, or any other possible outputs of a program. It can't know whether For my situation, I wouldn't say that the problem is that OpenSSL 1.1.x is not available; the problem is that Prisma stops looking for |
I think Prisma thinks different here: By the way: You can probably route around this whole detection logic by using the environment variables that directly provide the path to the engine files to Prisma. Maybe that is a workaround for you? |
Ah, that's helpful context, thank you---although it seems that the Prisma client is really determined to use 1.1.x, even if I only include Good tip about the environment variables; I'll explore that option, too! |
@janpio maybe I've missed some key part of the environment variables documentation, but I'm not seeing how to use them to work around the inferred OpenSSL version. It doesn't look like the query engine I'd need ( Run ls -al "node_modules/@prisma/engines"
drwxr-xr-x 4 runner docker 4096 Oct 13 23:45 .
drwxr-xr-x 5 runner docker 4096 Oct 13 23:45 ..
-rw-r--r-- 1 runner docker 11357 Oct 13 23:45 LICENSE
-rw-r--r-- 1 runner docker 740 Oct 13 23:45 README.md
drwxr-xr-x 3 runner docker 4096 Oct 13 23:45 dist
-rwxr-xr-x 1 runner docker 14796416 Oct 13 23:45 libquery_engine-debian-openssl-1.1.x.so.node
-rw-r--r-- 1 runner docker 1099 Oct 13 23:45 package.json
-rwxr-xr-x 1 runner docker 19523224 Oct 13 23:45 schema-engine-debian-openssl-1.1.x
drwxr-xr-x 2 runner docker 4096 Oct 13 23:45 scripts Since that's the case, the edit: seems like I'm not sure I understand the difference between |
You use Your required engine for Prisma Client should actually be in the built Prisma Client, which is probably in
I understand, but that does not exist right now. |
Haha! Sorry, I'm sure it feels like I'm repeating earlier comments in this thread. Just consider it voting with my issue comments for Prisma's future roadmap! It seems very likely that the combination of Thank you very much for coming on this nix-shell journey with me, @janpio ❤️ |
Please let me know if it works out. And sorry I did not get this idea earlier. I would suggest that you open a separate issue that we can actually turn into a feature request to reflect what you are looking for as an easier way to control this behavior. I am unsure if we should keep an issue open to describe the general Nix problem - I am still not sure if the problem there is with Nix (showing the file but not allowing its usage), or with Prisma (somehow relying on the presence of a file that should not indicate some functionality to be available). |
Personally, I think it's still worth keeping something open on the Prisma side, since there was a way to infer the working version of OpenSSL ( It's also kind of a catch-22 to suggest I think it would be more complicated, but it'd be be nice to have a more specifically informative error message for folks in my situation; something like:
I will report back! At least, when the |
I think that is actually simpler: |
I'm not sure if it's 100% related but I'm trying to run an app with a nixpacks-generated Docker image. I get this error: PrismaClientInitializationError: Unable to require(`/app/node_modules/.prisma/client/libquery_engine-debian-openssl-3.0.x.so.node`).
Prisma cannot find the required `libssl` system library in your system. Please install openssl-3.0.x and try again.
Details: libssl.so.3: cannot open shared object file: No such file or directory
at new e (/app/node_modules/@prisma/client/runtime/library.js:26:1871)
at /app/node_modules/@prisma/client/runtime/library.js:114:10094
at processTicksAndRejections (:61:76) Here is the output of 2024-02-06T09:22:14.392Z prisma:get-platform Found distro info:
{
"targetDistro": "debian",
"familyDistro": "debian",
"originalDistro": "ubuntu"
}
2024-02-06T09:22:14.392Z prisma:get-platform Trying platform-specific paths for "debian" (and "ubuntu")
2024-02-06T09:22:14.393Z prisma:get-platform Found libssl.so file using platform-specific paths: libssl.so.3
2024-02-06T09:22:14.394Z prisma:get-platform The parsed libssl version is: 3.0.x
prisma : 5.8.0
@prisma/client : 5.8.0
Computed binaryTarget : debian-openssl-3.0.x
Operating System : linux
Architecture : x64
Node.js : v20.9.0
Query Engine (Node-API) : libquery-engine 0a83d8541752d7582de2ebc1ece46519ce72a848 (at ../../node_modules/@prisma/engines/libquery_engine-debian-openssl-3.0.x.so.node)
Schema Engine : schema-engine-cli 0a83d8541752d7582de2ebc1ece46519ce72a848 (at ../../node_modules/@prisma/engines/schema-engine-debian-openssl-3.0.x)
Schema Wasm : @prisma/prisma-schema-wasm 5.8.0-37.0a83d8541752d7582de2ebc1ece46519ce72a848
Default Engines Hash : 0a83d8541752d7582de2ebc1ece46519ce72a848
Studio : 0.497.0 This is the output of
Node.js version: v20 (I've also tried with Node.js v18) Prisma version: 5.8.0 It uses I just don't know how to debug that further. Related issue: railwayapp/nixpacks#1030 |
prisma:get-platform
step infers incorrect libssl version in github action under ubuntu-latestprisma:get-platform
step infers incorrect libssl version in github action under ubuntu-latest
Bug description
Updating from 4.12 to 5.2, github actions began to fail when building the project (
npx prisma migrate reset --force
) with this error (fromDEBUG=prisma:*
):It seems like
prisma:get-platform
thinks thenative
target should be 1.1:I originally thought this was a problem with generating the client, but in the action's logs that seems to be done successfully:
This does not happen when developing locally (on
darwin-arm64
); thenative
binaryTarget
is doing its job as expected.How to reproduce
Full disclosure: I haven't minimized this. But in our case, the only change was upgrading the prisma version from 4.12.0 to 5.2.0. Previously our builds were humming along happily!
Expected behavior
I expect the OpenSSL version to be inferred as 3.0.x on a system that only has OpenSSL 3 available.
Prisma information
Environment & setup
ubuntu-latest
, which I think is 22.04Because of the node version, I wonder if it's related to #19124... but we've been on this node version for a while! However, we're not doing anything in parallel (afaik!), and the build doesn't progress to tests, so I've seen no evidence of segfaulting yet.
Prisma Version
The text was updated successfully, but these errors were encountered: