- Owner: @divyenduz
- Stakeholders: @mavilein @timsuchanek
- State:
- Spec: Stable ✅
- Implementation: In Progress 🚧
Binaries are the artifacts generated by compiling Prisma's core (written in Rust).
- Context
- Binaries
- Binary Files
- Binary Protocols
- Use Case: Prisma CLI
- Use Case: Prisma Client JS Generator
- How to Fetch Binaries
- Runtime
- Example Scenarios
- 1. Development machine is Mac but the deployment platform is AWS lambda.
- 2. Deterministically choose the binary-based a runtime environment variable
- 3. Development machine is Mac but we need a custom binary in production
- 4. Development machine is a Raspberry Pi and the deployment platform is AWS Lambda
Binaries are at the core of Prisma Client JS, Migrate and Prisma CLI via the Prisma SDK.
-> Whoever has the original shoud update this to include the introspection binary.
They are, however, compiled for a specific platform, that leads to the following requirements:
- Minimal configuration, simple mental model.
- Possibility of a deterministic binary resolution both locally and production setup.
- Easy setup of development and deployment workflows.
The binaries are used in multiple use cases, some of which are noted below:
Binaries are the artifacts generated by compiling the Prisma's core (written in Rust). The following binaries are generated:
Prisma query engine binary has the following use cases:
A generator like Photon
- Uses this binary to run queries against a data source (at runtime of generated code).
- Binary is downloaded when Photon is generated.
Prisma CLI
- Generation uses this binary to fetch internal schema representation (at the time of running
generateCLI command). - Binary is downloaded when the CLI is installed.
Prisma introspection engine binary has the following use cases:
The CLI uses the binary to:
- List available databases in a database reachable by a given connection string.
- Request metadata about specific databases.
- Introspect a database to generate a Prisma schema.
Prisma migration engine binary has the following use cases:
A generator like prisma-test-utils
- Uses this binary to perform migrations (at runtime of generated code)
- Binary is downloaded when
prisma-test-utilsis generated.
Prisma CLI
- Migrate commands use the binary to perform migrations or calculate pending migrations (at the time of running various
migratecommands likeup,saveetc). - Binary is downloaded when the CLI is installed.
VSCode Prisma extension uses this binary for providing code formatting features.
Prisma provides multiple binaries for various platforms and operating systems.
We need different binaries because operating systems such as Windows and Linux work fundamentally different. Some Linux distributions have the same issue. On top of that, different versions of Linux distributions have different versions of OpenSSL installed, which is why we need different binaries for those as well.
We build multiple binaries on common operating systems with different combinations of OpenSSL versions. This results in a few binaries, which work on a large selection of operating systems, distributions, and cloud platforms.
Initially, we built specific binaries for given platforms (e.g. Netlify), which resulted in many binaries which couldn't be shared with similar platforms and it was hard to maintain.
The following table lists the pre-built binaries provided by Prisma. These include Windows and Mac, and multiple variations for Linux distributions.
The binaries are published under the following URL schema.
- Latest:
<base url>/<branch>/latest/<platform family>/<binary>(.<file extension>).gz
e.g.https://binaries.prisma.sh/master/latest/darwin/prisma.gz - Specific commit:
<base url>/<branch>/<commit>/<platform family>/<binary>(.<file extension>).gz
e.g.https://binaries.prisma.sh/master/4028eec09329a14692b13f06581329fddb7b2876/darwin/prisma.gz
The commit SHAs can be found in our prisma-engines repository.
Notes
- The
<base url>ishttps://prisma-builds.s3-eu-west-1.amazonaws.com. <commit>and<branch>always correspond to the git branch and commit the binary was built for.- The binary names are:
- prisma
- migration-engine
- introspection-engine
- prisma-fmt
| Build name | Build OS | OpenSSL | Downloads (.gz) |
|---|---|---|---|
darwin |
Mac | n/a | prisma migration introspection prisma-fmt |
windows |
Windows | n/a | prisma.exe migration.exe introspection.exe prisma-fmt.exe |
debian-openssl-1.0.x |
Debian 8 | 1.0.x | prisma migration introspection prisma-fmt |
debian-openssl-1.1.x |
1.1.x | prisma migration introspection prisma-fmt | |
rhel-openssl-1.0.x |
CentOS 6 | 1.0.x | prisma migration introspection prisma-fmt |
rhel-openssl-1.1.x |
1.1.x | prisma migration introspection prisma-fmt |
This table shows which Linux distributions are compatible with our pre-built binaries.
| OS | Version | Build | Status ? | OpenSSL [1] | Comment |
|---|---|---|---|---|---|
| Debian | 8 (Jessie) | debian |
(:heavy_check_mark:) | 1.0.x | [2] install nodejs or openssl |
| 9 (Stretch) | (:heavy_check_mark:) | 1.1.x | [2] install nodejs or openssl |
||
| 10 (Buster) | (:heavy_check_mark:) | 1.1.x | [2] install nodejs or openssl |
||
| Ubuntu | 14.04 (trusty) | debian |
(:heavy_check_mark:) | 1.0.x | [2] install nodejs or openssl |
| 16.04 (xenial) | (:heavy_check_mark:) | 1.0.x | [2] install nodejs or openssl |
||
| 18.04 (bionic) | (:heavy_check_mark:) | 1.1.x | [2] install nodejs or openssl |
||
| 19.04 (disco) | (:heavy_check_mark:) | 1.1.x | [2] install nodejs or openssl |
||
| CentOS | 6 | rhel |
✔️ | 1.0.x | |
| 7 | ✔️ | 1.0.x | |||
| Fedora | 28 | rhel |
✔️ | 1.1.x | |
| 29 | ✔️ | 1.1.x | |||
| 30 | ✔️ | 1.1.x | |||
| Linux Mint | 18 | debian |
(:heavy_check_mark:) | 1.0.x | [2] install nodejs or openssl |
| 19 | (:heavy_check_mark:) | 1.1.x | [2] install nodejs or openssl |
||
| Arch Linux | 2019.09.01 | debian |
✔️ | 1.1.x | |
| Alpine | * | n/a | ❌ | ? | see prisma/photonjs#173 |
This is also continuously tracked on our build system.
| Build | Base OS | Status ? | Comment | |
|---|---|---|---|---|
| Netlify | debian |
Ubuntu 16.04/18.04 | ✔️ | Use the latest build image |
| Codesandbox | debian |
Debian 8 Jessie | ✔️ | |
| Zeit | rhel |
Amazon Linux (CentOS) | ✔️ | |
| Lambda | rhel |
Amazon Linux (CentOS) | ✔️ | Use the NodeJS 10 image or higher |
- ✔️ Works out of the box with no configuration at all.
- (:heavy_check_mark:) You may need to install dependencies.
- ❌ Currently unsupported.
-
The OpenSSL column in the tables above describe what the default OpenSSL version is on a given OS and version. It may not be installed per default, but it's always packaged through the native package manager (e.g. apt for Ubuntu), and in most cases you can simply install the package
openssl. -
You need dependencies on your machine to run the binary successfully. You can use your OS package manager, for example apt-get or yum. Check the table above which dependency you need exactly and then install it, e.g.
sudo apt-get install nodejs.
All downloaded binaries must follow the naming convention outlined by the Table of Binaries.
This includes both binaries downloaded for a generator and downloaded for CLI commands.
In case a binary for your platform is not listed in the Pre-built Binary Targets. Please follow this section of the docs to build a custom binary.
Prisma uses OpenSSL to enable a secure communication to remote databases. This means OpenSSL has to be either shipped with the binary ("static linking") or exist on the users's machine ("dynamic linking").
As it is critical to always run the latest OpenSSL due to security reasons, Prisma dynamically links OpenSSL so the user can update OpenSSL (usually with their native operating system package manager) independent of Prisma. Installing OpenSSL downloads a few files in common directories, which are usually named after the package name libssl, such as libssl.so.10, and the Prisma binaries automatically find and use these files. If they do not exist on the user's system or are incompatible, Prisma will crash and exits with an error which describes which file is missing.
Prisma query engine binary uses GraphQL over HTTP.
Prisma migration engine binary uses JSON RPC over stdio.
Prisma migration engine binary uses JSON RPC over stdio.
This is covered in the Prisma Engine Runtime (for JavaScript/TypeScript) spec.
Environment variable to configure the binary for CLI (like prisma2 migrate or prisma2 generate):
| Environment Variable | Description | Behavior |
|---|---|---|
PRISMA_MIGRATION_ENGINE_BINARY |
(optional) Overrides the resolution path for migration engine binary for migrate commands. |
Can be a relative (from CWD) or an absolute path to the binary |
PRISMA_INTROSPECTION_ENGINE_BINARY |
(optional) Overrides the resolution path for introspection engine binary. | Can be a relative (from CWD) or an absolute path to the binary |
PRISMA_QUERY_ENGINE_BINARY |
(optional) Overrides the resolution path for query engine binary for generate command. |
Can be a relative (from CWD) or an absolute path to the binary |
PRISMA_FMT_BINARY |
(optional) Overrides the resolution path for format binary for format command. |
Can be a relative (from CWD) or an absolute path to the binary |
- CLI binaries can only be overridden by a path to a custom binary. It does not alter download behavior, it just overrides the binary path provided for respective commands. This means that using known binaries is not possible.
-
If the environment variable path to a custom binary is not found, the respective CLI command should throw.
-
If the environment variable path to a custom binary exists but the binary is incompatible with the current platform, the respective CLI command should throw.
As we do not have precompiled binaries for ARM architecture yet, the user would compile binaries manually for Prisma query engine and Prisma migration engine.
export PRISMA_INTROSPECTION_ENGINE_BINARY=<path to compiled introspection engine binary>
export PRISMA_QUERY_ENGINE_BINARY=<path to compiled query engine binary>
export PRISMA_MIGRATION_ENGINE_BINARY=<path to compiled migration engine binary>
export PRISMA_FMT_BINARY=<path to compiled format binary>Then prisma2 migrate and prisma2 generate would use the respective compiled binaries.
2. We are using CLI in a build system from a provider for which we do not have a working pre-compiled binary
Since overriding CLI binary is an environment variable and these providers might not always allow compiling a binary. There will be no work around such a situation except us making the default downloaded binary for that provider work. We want to support all major providers out of the box and this use case should be rare.
Fields on the generator block to configure the availability of binaries for generators (like Prisma Client JS, nexus, etc):
| Field | Description |
|---|---|
binaryTargets |
(optional) An array of binaries that are required by the application, string for known binary targets. These are downloaded at generation time. Note that custom binary paths should not be provided in the binaryTargets. |
Environment variable to configure a specific binary for the generated code's runtime:
| Environment Variable | Description |
|---|---|
PRISMA_MIGRATION_ENGINE_BINARY |
(optional) A string literal with a known binary name (like darwin or linux-glibc-libssl1.0.2" or path to a custom binary |
PRISMA_INTROSPECTION_ENGINE_BINARY |
(optional) A string literal with a known binary name (like darwin or linux-glibc-libssl1.0.2" or path to a custom binary |
PRISMA_QUERY_ENGINE_BINARY |
(optional) A string literal with a known binary name (like darwin or linux-glibc-libssl1.0.2" or path to a custom binary |
PRISMA_FMT_BINARY |
(optional) A string literal with a known binary name (like darwin or linux-glibc-libssl1.0.2" or path to a custom binary |
-
Both
binaryTargetsfield andPRISMA_QUERY_ENGINE_BINARYenvironment variable are optional. Here are some scenarios-
binaryTargetsis not provided.generator client { provider = "prisma-client-js" }We download and use the binary for the current platform.
-
Field
binaryTargetsprovided with a single valuegenerator client { provider = "prisma-client-js" binaryTargets = ["native"] }We download the provided binary and resolve it at runtime.
-
Field
binaryTargetsprovided with multiple valuesgenerator client { provider = "prisma-client-js" binaryTargets = ["native", "linux-glibc-libssl1.0.2"] }We download both binaries and resolve the correct binary at runtime by detecting the platform.
-
Field
binaryTargetsprovided with multiple values andPRISMA_QUERY_ENGINE_BINARYenvironment variable provided.generator client { provider = "prisma-client-js" binaryTargets = ["native", "linux-glibc-libssl1.0.2"] }PRISMA_QUERY_ENGINE_BINARY=native # On LocalPRISMA_QUERY_ENGINE_BINARY=linux-glibc-libssl1.0.2 # In ProductionWe download all the binaries specified by
binaryTargetsand use the binary specified byPRISMA_QUERY_ENGINE_BINARYenvironment variable.
Note: In production setups with a dedicated CI which has a different platform than the deployment machine (think CircleCI, Netlify), we can configure
binaryTargetsto only include the required binaries:generator client { provider = "prisma-client-js" binaryTargets = env("BINARY_TARGETS") }With environment variable:
BINARY_TARGETS=["linux-glibc-libssl1.1.0"] # In CI for CircleCI
BINARY_TARGETS=["linux-glibc-libssl1.0.2"] # In production for Netlify
A configuration like
binaryTargets = ["native", "linux-glibc-libssl1.0.2"]is only needed when the development machine is also the machine responsible to build for production but the platform in production is different, likeAWS lambda,now, etc. -
-
Known binaries specified in
binaryTargetsdownloads specified known binaries to an OS cache path and copies it to generator path ongenerate. -
Not all generators require all the binaries, the generators spec outlines the generator API that defines which binaries are needed.
Note: pinned binary in this section refers to binary specified via PRISMA_QUERY_ENGINE_BINARY environment variable.
-
If the pinned binary is a known binary and it is not found during the generated code's runtime, it should throw.
-
If the pinned binary is a known binary but does not work for the current platform, try other known binaries from
binaryTargets. This would make the use cases work where build machine is different from deploy machine, like in the case of zeit's now. -
If the pinned binary is a known binary but not listed in the
binaryTargets, throw because these binaries won't be downloaded. -
If the pinned binary is a custom binary but does not work for the current platform or if its path does not exist, generated code's runtime should throw.
In the scenario where binaryTargets field is defined but no pinnedBinaryTarget field is defined, we resolve the binary at runtime by detecting the platform. This can be achieved by generating code similar to this pseudo-code in Photon.
function detectPlatform(): string { ... }
const binaryTarget = process.env.PRISMA_QUERY_ENGINE_BINARY
const binaries = {
'mac': <path>,
'lambda': <path>,
}
let binaryPath
if (!binaryTarget) {
const inferredPlatform = detectPlatform()
binaryPath = binaries[inferredPlatform]
} else {
binaryPath = binaries[binaryTarget]
}We can use binaryTargets without a PRISMA_QUERY_ENGINE_BINARY environment variable. Which binary to use will be resolved at runtime, see Runtime for more details.
generator client {
provider = "prisma-client-js"
binaryTargets = ["native", "linux-glibc-libssl1.0.2"]
}We can use binaryTargets and provide a specific binary using PRISMA_QUERY_ENGINE_BINARY environment variable.
generator client {
provider = "prisma-client-js"
binaryTargets = ["native", "linux-glibc-libssl1.0.2"]
}PRISMA_QUERY_ENGINE_BINARY=native # On LocalPRISMA_QUERY_ENGINE_BINARY=linux-glibc-libssl1.0.2 # In ProductionWe define the binaryTargets and use one of the binaryTargets.
We use "native" (binary of the current platform) in binaryTargets and provide PRISMA_QUERY_ENGINE_BINARY in production.
generator client {
provider = "prisma-client-js"
binaryTargets = ["native"]
}PRISMA_QUERY_ENGINE_BINARY=custom-prisma-binary # In productionAs we do not have precompiled binaries for ARM architecture yet, the user would compile binaries manually for Prisma query engine and use it by providing PRISMA_QUERY_ENGINE_BINARY environment variable
generator client {
provider = "prisma-client-js"
binaryTargets = ["linux-glibc-libssl1.0.2"]
}PRISMA_QUERY_ENGINE_BINARY=./custom-query-engine-binary # On localUnset PRISMA_QUERY_ENGINE_BINARY in production for it to pick the only binary specified in binaryTargets.