Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(*): polishing upstream work on the data proxy [DPGA, 5] #13629

Merged
merged 39 commits into from
Jun 7, 2022
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
fd0821b
remove 3.4.1 fallback from data proxy client
millsp Jun 2, 2022
5aadcaf
only create an edge directory if dataProxy is true
millsp Jun 2, 2022
54e4c51
infer the data proxy version from the engine version
millsp Jun 3, 2022
089304c
apply review
millsp Jun 3, 2022
d14aa39
Merge branch 'chore/data-proxy-ga-4' into integration/chore/data-prox…
millsp Jun 3, 2022
cab67ed
Merge branch 'chore/data-proxy-ga-4' into integration/chore/data-prox…
millsp Jun 3, 2022
25d5a43
Merge branch 'chore/data-proxy-ga-4' into integration/chore/data-prox…
millsp Jun 3, 2022
dceabd0
restore fix for umd lz-string via esbuild injection
millsp Jun 3, 2022
41e6298
Merge branch 'integration/chore/data-proxy-ga-5' of github.com:prisma…
millsp Jun 3, 2022
4d0b741
add missing data proxy url check to prisma init
millsp Jun 3, 2022
752d757
aggregate the ambient env to the loaded one
millsp Jun 3, 2022
5cf1374
log the select data proxy client version in debug
millsp Jun 3, 2022
21294c6
make buildInjectableEdgeEnv more digestible
millsp Jun 3, 2022
b951a42
include "edge" in the publish of the package
millsp Jun 3, 2022
e94e596
update comment about default throw files
millsp Jun 3, 2022
1f33f60
check that metrics cannot be used with --data-proxy
millsp Jun 3, 2022
227da0a
throw error instead of exiting the process (for testing)
millsp Jun 3, 2022
dfc6fb7
fix the default import location via module not the path
millsp Jun 3, 2022
cfff964
remove unnecessary logic for building NTF annotations
millsp Jun 3, 2022
3476eb9
rename network to request error and append info
millsp Jun 3, 2022
779a3db
update snapshots
millsp Jun 3, 2022
1de120d
make the prisma client edge point to browser field
millsp Jun 3, 2022
f4eac36
obfuscate require in a way that does not break jest
millsp Jun 3, 2022
b87e240
only assign to globalThis if it actually exists
millsp Jun 3, 2022
961b7f6
fix env priority, env var injection, and constructor promises
millsp Jun 4, 2022
4a4d8ef
do not create edge folder but just the file
millsp Jun 4, 2022
3c335ef
debug vercel edge functions via logging
millsp Jun 4, 2022
3170776
fix wrong path to runtime after removal of edge folder
millsp Jun 4, 2022
2aaf6e0
include edge.js in the package.json
millsp Jun 4, 2022
347765e
make esbuild discard all evals and new Function
millsp Jun 4, 2022
3d2b349
properly remove and polyfill calls to Function
millsp Jun 4, 2022
091f807
cleanup
millsp Jun 6, 2022
736ce04
fix engine mode display for generate
millsp Jun 6, 2022
985c6ae
create a dts redirect for custom outdir
millsp Jun 6, 2022
727b290
further simplify logic for injectable edge env
millsp Jun 6, 2022
0755229
remove unnecessary path.join
millsp Jun 6, 2022
fa0b011
make getClientVersion async
millsp Jun 7, 2022
ce8ce89
Merge branch 'chore/data-proxy-ga-4' into integration/chore/data-prox…
millsp Jun 7, 2022
2df7160
Merge branch 'integration/chore/data-proxy-ga-5' of github.com:prisma…
millsp Jun 7, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 3 additions & 0 deletions helpers/compile/plugins/fill-plugin/fillers/function.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export const fn = () => {}

fn.prototype = fn
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Polyfill for Vercel which disallows Function.

5 changes: 4 additions & 1 deletion packages/cli/src/Init.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import type { ConnectorType } from '@prisma/generator-helper'
import type { Command } from '@prisma/sdk'
import {
arg,
canConnectToDatabase,
checkUnsupportedDataProxy,
Command,
format,
getCommandWithExecutor,
HelpError,
Expand Down Expand Up @@ -134,6 +135,8 @@ export class Init implements Command {
return this.help()
}

await checkUnsupportedDataProxy('init', args, false)
Copy link
Member Author

@millsp millsp Jun 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prisma init tests if it can connect to the db, so we need to make sure it fails if prisma:// is in use.


/**
* Validation
*/
Expand Down
1 change: 1 addition & 0 deletions packages/client/edge.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from '.prisma/client'
3 changes: 3 additions & 0 deletions packages/client/edge.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module.exports = {
...require('.prisma/client/edge'),
}
1 change: 0 additions & 1 deletion packages/client/edge/index.d.ts

This file was deleted.

3 changes: 0 additions & 3 deletions packages/client/edge/index.js

This file was deleted.

14 changes: 14 additions & 0 deletions packages/client/helpers/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ import type { BuildOptions } from '../../../helpers/compile/build'
import { build } from '../../../helpers/compile/build'
import { fillPlugin } from '../../../helpers/compile/plugins/fill-plugin/fillPlugin'

const fillPluginPath = path.join('..', '..', 'helpers', 'compile', 'plugins', 'fill-plugin')
const functionPolyfillPath = path.join(fillPluginPath, 'fillers', 'function.ts')

// we define the config for runtime
const nodeRuntimeBuildConfig: BuildOptions = {
name: 'runtime',
Expand All @@ -13,6 +16,8 @@ const nodeRuntimeBuildConfig: BuildOptions = {
bundle: true,
define: {
NODE_CLIENT: 'true',
// that fixes an issue with lz-string umd builds
'define.amd': 'false',
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored this as it's still needed, otherwise it causes issues with webpack.

},
}

Expand All @@ -36,9 +41,18 @@ const edgeRuntimeBuildConfig: BuildOptions = {
define: {
// that helps us to tree-shake unused things out
NODE_CLIENT: 'false',
// that fixes an issue with lz-string umd builds
'define.amd': 'false',
},
plugins: [
fillPlugin({
// we remove eval and Function for vercel
eval: { define: 'undefined' },
Function: {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-define Function as being fn (to rename it), and inject fn.

define: 'fn',
inject: functionPolyfillPath,
},

// TODO no tree shaking on wrapper pkgs
'@prisma/get-platform': { contents: '' },
// removes un-needed code out of `chalk`
Expand Down
2 changes: 2 additions & 0 deletions packages/client/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@
"runtime",
"scripts",
"generator-build",
"edge.js",
"edge.d.ts",
"index.js",
"index.d.ts",
"index-browser.js"
Expand Down
11 changes: 5 additions & 6 deletions packages/client/scripts/postinstall.js
Original file line number Diff line number Diff line change
Expand Up @@ -197,21 +197,20 @@ function run(cmd, params, cwd = process.cwd()) {
/**
* Copies our default "throw" files into the default generation folder. These
* files are dummy and informative because they just throw an error to let the
* user know that they have forgotten to run `prisma generate`.
* user know that they have forgotten to run `prisma generate` or that they
* don't have a a schema file yet. We only add these files at the default
* location `node_modules/.prisma/client`.
*/
async function createDefaultGeneratedThrowFiles() {
try {
const dotPrismaClientDir = path.join(__dirname, '../../../.prisma/client')
const defaultNodeIndexPath = path.join(dotPrismaClientDir, 'index.js')
const defaultNodeIndexDtsPath = path.join(dotPrismaClientDir, 'index.d.ts')
const defaultBrowserIndexPath = path.join(dotPrismaClientDir, 'index-browser.js')
const defaultEdgeIndexPath = path.join(dotPrismaClientDir, 'edge.js')
const defaultEdgeIndexDtsPath = path.join(dotPrismaClientDir, 'edge.d.ts')
await makeDir(dotPrismaClientDir)

const dotPrismaClientEdgeDir = path.join(dotPrismaClientDir, 'edge')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed anymore, instead of having a full folder, we just have one file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Isomorphism is not needed because bundling for the edge means that we always bundle for the browser)

const defaultEdgeIndexPath = path.join(dotPrismaClientEdgeDir, 'index.js')
const defaultEdgeIndexDtsPath = path.join(dotPrismaClientEdgeDir, 'index.d.ts')
await makeDir(dotPrismaClientEdgeDir)

if (!fs.existsSync(defaultNodeIndexPath)) {
await copyFile(path.join(__dirname, 'default-index.js'), defaultNodeIndexPath)
}
Expand Down
6 changes: 3 additions & 3 deletions packages/client/src/generation/TSClient/Generatable.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
export interface Generatable {
toJS?(edge?: boolean): string | Promise<string>
toTS(): string | Promise<string>
toTS(edge?: boolean): string | Promise<string>
toBrowserJS?(): string | Promise<string>
toTSWithoutNamespace?(): string | Promise<string>
}
Expand All @@ -13,6 +13,6 @@ export function BrowserJS(gen: Generatable): string | Promise<string> {
return gen.toBrowserJS?.() ?? ''
}

export function TS(gen: Generatable): string | Promise<string> {
return gen.toTS()
export function TS(gen: Generatable, edge = false): string | Promise<string> {
return gen.toTS(edge)
}
5 changes: 4 additions & 1 deletion packages/client/src/generation/TSClient/TSClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,10 @@ ${buildNFTAnnotations(dataProxy, engineType, platforms, relativeOutdir)}
`
return code
}
public toTS(): string {
public toTS(edge = false): string {
// edge exports the same ts definitions as the index
if (edge === true) return `export * from './index'`

const prismaClientClass = new PrismaClientClass(
this.dmmf,
this.options.datasources,
Expand Down
16 changes: 3 additions & 13 deletions packages/client/src/generation/generateClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -123,17 +123,8 @@ export async function buildClient({

// we only generate the edge client if `--data-proxy` is passed
if (dataProxy === true) {
fileMap[path.join('edge', 'index.js')] = await JS(edgeTsClient, true)
fileMap[path.join('edge', 'package.json')] = JSON.stringify(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed anymore, instead of having a full folder, we just have one file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Isomorphism is not needed because bundling for the edge means that we always bundle for the browser)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, so that means in isomporphic setups we will be bundling the whole client for the browser, right? In previous setup it was only enums, version info and a bunch of stubs, so significantly smaller amount of code. Is that what we want?

Copy link
Member Author

@millsp millsp Jun 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you deploy on Cloudflare Workers, it is by default bundled via the browser field. The same applies to Vercel Edge Functions, they always chose browser field. That means we cannot have such isomorphism, but just a browser build, hence the single file. So yes, if you bundle @prisma/client/edge into your frontend, you get the non-needed runtime with it, unfortunately. The best way to tackle this would be to have esm modules so that tree-shaking can take place (but that's a bigger project). In the meantime @prisma/client can be used.

{
name: '.prisma/client/edge',
main: 'index.js',
types: '../index.d.ts',
browser: '../index-browser.js',
},
null,
2,
)
fileMap[path.join('edge.js')] = await JS(edgeTsClient, true)
millsp marked this conversation as resolved.
Show resolved Hide resolved
fileMap[path.join('edge.d.ts')] = await TS(edgeTsClient, true)
}

return {
Expand Down Expand Up @@ -222,7 +213,6 @@ export async function generateClient(options: GenerateClientOptions): Promise<vo
}

await makeDir(finalOutputDir)
await makeDir(path.join(finalOutputDir, 'edge'))
await makeDir(path.join(outputDir, 'runtime'))
// TODO: why do we sometimes use outputDir and sometimes finalOutputDir?
// outputDir: /home/millsp/Work/prisma/packages/client
Expand Down Expand Up @@ -435,7 +425,7 @@ async function getGenerationDirs({ testMode, runtimeDirs, generator, outputDir }
const _runtimeDirs = {
// if we have an override, we use it, but if not then use the defaults
node: runtimeDirs?.node || (useDefaultOutdir ? '@prisma/client/runtime' : './runtime'),
edge: runtimeDirs?.edge || (useDefaultOutdir ? '@prisma/client/runtime' : '../runtime'),
edge: runtimeDirs?.edge || (useDefaultOutdir ? '@prisma/client/runtime' : './runtime'),
}

const finalOutputDir = useDefaultOutdir ? await getDefaultOutdir(outputDir) : outputDir
Expand Down
86 changes: 33 additions & 53 deletions packages/client/src/generation/utils/buildInjectableEdgeEnv.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type { InternalDatasource } from '../../runtime/utils/printDatasources'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I simplified the code and updated the comments, hopefully more comprehensive.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(this code is only for the edge client)


type LoadedEnv = {
type InjectableEnv = {
parsed: {
[x: string]: string | undefined
}
Expand All @@ -9,90 +9,70 @@ type LoadedEnv = {
/**
* Builds an injectable environment for the data proxy edge client. It's useful
* because it is designed to run in browser-like environments where things like
* `fs`, `process.env`, and .env file loading are not available. The injectable
* env is the default fallback when `tryLoadEnvs` wasn't called by the client.
* `fs`, `process.env`, and .env file loading are not available. It is the place
* where we make collect the env vars for the edge client. To understand this
* better, take a look at the generated code in the edge client.
* @see {@link declareInjectableEdgeEnv}
* @param edge
* @param datasources
* @returns
*/
export function buildInjectableEdgeEnv(edge: boolean, datasources: InternalDatasource[]) {
if (edge === true) {
const envVarNames = getSelectedEnvVarNames(datasources)
const loadedEnv = getEmptyEnvObjectForVars(envVarNames)

return declareInjectableEdgeEnv(loadedEnv)
return declareInjectableEdgeEnv(datasources)
}

return ``
}

/**
* Determines which env vars we are interested in in the case of the data proxy
* client. Right now, we are only interested in env vars from the `datasource`.
* Creates the necessary declarations to embed env vars directly inside of the
* generated client. We abuse a custom `JSON.stringify` to generate the code.
* @param datasources to find env vars in
* @returns
*/
function getSelectedEnvVarNames(datasources: InternalDatasource[]) {
return datasources.reduce((acc, datasource) => {
if (datasource.url.fromEnvVar) {
return [...acc, datasource.url.fromEnvVar]
}
function declareInjectableEdgeEnv(datasources: InternalDatasource[]) {
const envVarNames = getSelectedEnvVarNames(datasources)

return acc
}, [] as string[])
}

/**
* Like `tryLoadEnvs` but we only retain a subset of the env vars that will be
* used in the case of the data proxy client. And we discard the `message` prop.
* @param envVarNames to be selected from the load
* @returns
*/
function getEmptyEnvObjectForVars(envVarNames: string[]) {
const selectedEnv: LoadedEnv = { parsed: {} }
const injectableEdgeEnv: InjectableEnv = { parsed: {} }

// we create a base env with empty values for env names
for (const envVarName of envVarNames) {
/** note that we willingly keep `undefined` values because
* we need that later in {@link declareInjectableEdgeEnv} **/
selectedEnv.parsed[envVarName] = undefined
injectableEdgeEnv.parsed[envVarName] = undefined
}

return selectedEnv
}

/**
* Creates the necessary declarations to embed env vars directly inside of the
* generated client. We abuse a custom `JSON.stringify` replacer to transform
* {@link loadedEnv} into a piece of code. The goal of this is to take that and
* generate a new object which re-prioritizes the loading of the environment.
*
* By generating an output with `${key} || process.env.${key} || '${value}'` it
* would yield something like `DB_URL || process.env.DB_URL || undefined`:
* - For Cloudflare Workers `DB_URL` will be the first to be found (runtime)
* - `process.env.DB_URL` will be undefined
* - For Vercel `process.env.DB_URL` is replaced by webpack by a value
* - `DB_URL` will be undefined at anytime
* - If none of them were provided, we fallback to the default `undefined`
* @param loadedEnv
*/
function declareInjectableEdgeEnv(loadedEnv: LoadedEnv) {
// abuse a custom replacer to create the injectable env
const injectableEdgeEnvDeclaration = JSON.stringify(
loadedEnv,
injectableEdgeEnv,
(key, value) => {
if (key === '') return value
if (key === 'parsed') return value

// for cloudflare workers, an env var is a global js variable
const cfwEnv = `typeof global !== 'undefined' && global['${key}']`
const vercelEnv = `process.env['${key}']`
const dotEnv = value ? `'${value}'` : 'undefined'
// for vercel edge functions, it's injected statically at build
const vercelEnv = `process.env.${key}`

return `${cfwEnv} || ${vercelEnv} || ${dotEnv}`
return `${cfwEnv} || ${vercelEnv} || undefined`
},
2,
).replace(/"/g, '') // remove quotes to make code

return `
config.injectableEdgeEnv = ${injectableEdgeEnvDeclaration}`
}

/**
* Determines which env vars we are interested in in the case of the data proxy
* client. Right now, we are only interested in env vars from the `datasource`.
* @param datasources to find env vars in
* @returns
*/
function getSelectedEnvVarNames(datasources: InternalDatasource[]) {
return datasources.reduce((acc, datasource) => {
if (datasource.url.fromEnvVar) {
return [...acc, datasource.url.fromEnvVar]
}

return acc
}, [] as string[])
}
34 changes: 3 additions & 31 deletions packages/client/src/generation/utils/buildNFTAnnotations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,11 @@ export function buildNFTAnnotations(
}

const engineAnnotations = map(platforms, (platform) => {
return buildNFTEngineAnnotation(engineType, platform, relativeOutdir)
const engineFilename = getQueryEngineFilename(engineType, platform)
return engineFilename ? buildNFTAnnotation(engineFilename, relativeOutdir) : ''
}).join('\n')

const schemaAnnotations = buildNFTSchemaAnnotation(engineType, relativeOutdir)
const schemaAnnotations = buildNFTAnnotation('schema.prisma', relativeOutdir)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just shrinking the code here.


return `${engineAnnotations}${schemaAnnotations}`
}
Expand Down Expand Up @@ -76,32 +77,3 @@ function buildNFTAnnotation(fileName: string, relativeOutdir: string) {
path.join(__dirname, ${JSON.stringify(fileName)});
path.join(process.cwd(), ${JSON.stringify(relativeFilePath)})`
}

/**
* Build an annotation for the prisma client engine files
* @param engineType
* @param platform
* @param relativeOutdir
* @returns
*/
function buildNFTEngineAnnotation(engineType: ClientEngineType, platform: Platform, relativeOutdir: string) {
const engineFilename = getQueryEngineFilename(engineType, platform)

if (engineFilename === undefined) return ''

return buildNFTAnnotation(engineFilename, relativeOutdir)
}

/**
* Build an annotation for the prisma schema files
* @param engineType
* @param relativeOutdir
* @returns
*/
function buildNFTSchemaAnnotation(engineType: ClientEngineType, relativeOutdir: string) {
if (engineType === ClientEngineType.Library || engineType === ClientEngineType.Binary) {
return buildNFTAnnotation('schema.prisma', relativeOutdir)
}

return ''
}
2 changes: 1 addition & 1 deletion packages/client/src/runtime/getPrismaClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ declare global {
}

// used by esbuild for tree-shaking
globalThis.NODE_CLIENT = true
typeof globalThis === 'object' ? (globalThis.NODE_CLIENT = true) : 0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to check if this exists always.


function isReadonlyArray(arg: any): arg is ReadonlyArray<any> {
return Array.isArray(arg)
Expand Down
2 changes: 1 addition & 1 deletion packages/engine-core/src/common/Engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ export interface EngineConfig {
showColors?: boolean
logQueries?: boolean
logLevel?: 'info' | 'warn'
env?: Record<string, string>
env: Record<string, string>
flags?: string[]
clientVersion?: string
previewFeatures?: string[]
Expand Down