Preview feature feedback: @prisma/adapter-neon / Neon Serverless Driver support

[janpio]

Support for the Neon Serverless Driver is in preview behind the driverAdapters preview feature.

Please share your feedback about @prisma/adapter-neon released in v5.4.0 in this issue.

• If you encounter a bug, please open a bug report in this repo.
• If the feature is working well for you, please share this in a comment below or leave a 👍 on this issue.
• If you have any questions, don't hesitate to ask them in the Prisma Slack community

[menewman]

👋 Hi, @janpio - This feature is very exciting! In non-edge runtimes, it seems like it's working as expected from some initial testing.

However, I was curious about whether this works in Vercel Edge runtimes, and it seems like it does not if we use setup code similar to what is provided in the blog article:

// Import needed packages
import { Pool, neonConfig } from '@neondatabase/serverless';
import { PrismaNeon } from '@prisma/adapter-neon';
import { PrismaClient } from '@prisma/client';
import dotenv from 'dotenv';
import ws from 'ws';

// Setup
dotenv.config()
neonConfig.webSocketConstructor = ws;
const connectionString = `${process.env.DATABASE_URL}`;

// Init prisma client
const pool = new Pool({ connectionString });
const adapter = new PrismaNeon(pool);
const prisma = new PrismaClient({ adapter });

// Use Prisma Client as normal

^This doesn't work in edge runtimes (regardless of whether we import from @prisma/client or @prisma/client/edge). The specific error I see is around the websocket library (whether I use undici or ws), since not all of the necessary dependencies for it are available in the edge runtime.

From the TypeScript types (and in the source code of the Prisma client), I see that there is a PrismaNeonHTTP adapter we could also import/use instead, a bit more like this:

  const connectionString = `${process.env.DATABASE_URL}`;
  const connection = neon(connectionString, {
    arrayMode: false,
    fullResults: true,
  });

  const adapter = new PrismaNeonHTTP(connection);
  return new PrismaClient({ adapter });

That looks promising, but I can't quite get it work when passing it to the Prisma client constructor from @prisma/client/edge -- I consistently get this error:

Error validating datasource `db`: the URL must start with the protocol `prisma://`

I just wanted to confirm whether there is a way to use the Neon serverless driver in edge runtimes, or if that's specifically something that you wouldn't expect to work yet. I wasn't sure from the announcement whether this new support for the Neon serverless driver was meant to include support for edge or not (I'm assuming not, but I'd love to be wrong).

Thank you in advance! Again, very exciting stuff, and I'm glad the team has been working on enhanced support for serverless/edge.

[janpio]

Hey, thanks for trying it out and the feedback! We really appreciate it.

To your Edge question:

Prisma Client does not work in Edge Function deployments yet. To quote the launch blog post:

Prisma Client and the new driver adapters feature currently do not support Edge Functions deployments such as CloudFlare Workers or Vercel Edge Functions yet. If you want to test the Neon or PlanetScale Serverless Driver in Edge Functions environments, make sure to submit a response to our survey so that you will be considered for Early Access to our edge function support as well. We'll have news soon.

And to add a bit of explanation that I just posted here:

The release of Driver Adapters is the first step to bring us to being able to allow usage of Prisma Client [in Edge Function environments]. Our classic database drivers use TCP, which is not available or just via some experimental approaches - so we needed to develop support for HTTP based drivers first (like this one from PlanetScale). Now we can focus on making it possible to run Prisma Client directly there, using these Driver Adapters.

So we are working on making both Neon Driver Adapters, WS and HTTP, work in environments like Vercel Edge. You can follow #20566 for updates as well.

[menewman]

Awesome, thank you for confirming that it's not expected to work yet, and also that it's something the team is working on! That's pretty much what I expected, but I had a small hope that maybe I was just doing it wrong and it was already supposed to work.

(And sorry that I missed it in the conclusion of the blog post; you did write it there pretty clearly 😅)

I've responded to the survey & subscribed to that GitHub discussion, so I'll stay tuned for updates 🎉

[kokokenada]

I'm giving this a try. I'm currently in development and using cockroachdb on AWS Lambda and I'm noticing some cold start slowness and I'm hoping this might be a great fix. (Any one care to opine on the tradeoffs between cockroachdb and neon?) Anyways, I'm trying to run locally and am making some progress. Neon docs seem sketchy on the running locally, but I've pieced together that I can run docker-compose with this https://github.com/neondatabase/neon/tree/main/docker-compose and reach postgres with DATABASE_URL="postgresql://cloud_admin:cloud_admin@localhost:55433/postgres?sslmode=disable&schema=public" (Can't seem to find the port for the Neon console). My integration tests run fine when connecting directly to Neon without the pooling. When I add adapter to the prisma connection, I get a runtime error:

Error: connect ECONNREFUSED 127.0.0.1:443
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16)

I guess it's looking to be https. I tried this change to no effect:

// Setup
neonConfig.webSocketConstructor = ws;
neonConfig.useSecureWebSocket = false; // new - did not resolve
const connectionString = `${process.env.DATABASE_URL}`;

// Init prisma client
const pool = new Pool({
  connectionString,
});
const adapter = new PrismaNeon(pool);

Any ideas on how to run the pooling locally (for testing purposes).

[kokokenada]

Further to this, I'm trying to connect to the cloud instance and ran into some unexpected errors. Running the app node server locally connecting to a cloud instance of Neon. Resulted in:

    PrismaClientUnknownRequestError: 
    Invalid `prisma.quarantineMessage.deleteMany()` invocation in
    /home/kenono/apps/tripmatch/packages/data-access/src/lib/data-util.ts:208:36

      205 }
      206 
      207 public static async reset() {
    → 208   await prisma.quarantineMessage.deleteMany(
    Error occurred during query execution:
    ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(PostgresError { code: "42P05", message: "prepared statement \"s0\" already exists", severity: "ERROR", detail: None, column: None, hint: None }), transient: false })

      at Cn.handleRequestError (../../node_modules/@prisma/client/runtime/library.js:123:6989)
      at Cn.handleAndLogRequestError (../../node_modules/@prisma/client/runtime/library.js:123:6206)
      at Cn.request (../../node_modules/@prisma/client/runtime/library.js:123:5926)

(this was during the cleanup phase of the integration tests)

Trying to run the app itself using AWS Lambda resulted in:

2023-10-15T09:59:52.622-04:00Copy2023-10-15T13:59:52.622Z	38ac1fa5-58e9-430e-9211-c2d3a85db57d	INFO	prisma:error Unsupported column type: 1184 | 2023-10-15T13:59:52.622Z 38ac1fa5-58e9-430e-9211-c2d3a85db57d INFO prisma:error Unsupported column type: 1184
-- | --
  | 2023-10-15T09:59:52.623-04:00 | �[31m[Nest] 8 - �[39m10/15/2023, 1:59:52 PM �[31m ERROR�[39m �[38;5;3m[ExceptionsHandler] �[39m�[31mUnsupported column type: 1184�[39m
  | 2023-10-15T09:59:52.623-04:00CopyError: Unsupported column type: 1184 | Error: Unsupported column type: 1184
  | 2023-10-15T09:59:52.623-04:00 | at fieldToColumnType (/var/task/dist/packages/server/main.js:69044:13)

The tables etc were created no problem and the seed.ts was run correctly.

[janpio]

ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(PostgresError { code: "42P05", message: "prepared statement \"s0\" already exists", severity: "ERROR", detail: None, column: None, hint: None }), transient: false })

Hmm, that is unexpected and weird. Smells a bit like it might be using the driver adapter, and this might be coming from the usual driver, but I have no idea how to confirm that 🤔

| 2023-10-15T09:59:52.623-04:00CopyError: Unsupported column type: 1184 | Error: Unsupported column type: 1184

This is an actual bug in the preview version I think. Does that query use any "not super basic" types?
We're definitely going to improve on the types support a lot more in the next release(s).

[janpio]

Did you figure out what was going on @kokokenada?

[janpio]

Ping again @kokokenada

[kokokenada]

Thanks so much for the follow up. I switched back to cockroach and haven't had a chance to try Neon again. I'll pipe up if/when I manage to do so

[mjyoung]

Does the neon adapter respect the ?schema param in the connection string?

I have a connection string with ?schema=staging.

const prisma = new PrismaClient();
await prisma.$queryRaw`SHOW search_path;`;
// search_path is 'staging' as expected

const prisma = new PrismaClient({ adapter })
await prisma.$queryRaw`SHOW search_path;`;
// search_path is 'public' as NOT expected

[janpio]

No, currently we indeed do not handle the schema at all for raw queries. We take the connections or pool with what whatever configuration you provide. (For our own SQL queries we generate from Prisma Client queries, we qualify the table names explicitly with the schema - hence that is not a problem.)

I am honestly not sure yet how we should deal with this. Modifying the provided connection or pool feels wrong. On the other hand, as you experienced, the behavior can be unexpected. Ideas?

[mjyoung]

Granted I may be ignorant in how these prisma adapters work under the hood, but given the same DB string passed in via a DATABASE_URL with a schema=staging parameter, it seems reasonable to assume same queries (raw or not) should return same results.

[janpio]

schema=foo is not part of any connection string standard, but something we added for the default database drivers that Prisma uses. So when you use the Neon serverless driver, it doesn't do anything with it. And that is exactly the problem we have, as with a connection pool it is also not enough to just set the value once - but each new connection needs to be modified. We'll need to see if there is a configuration for Neon and pg that might automate this. Then we could document that this needs to be applied.

[mjyoung]

Ahhh that makes a lot of sense, thank you. Yeah not sure the best way to handle the schema situation. For now I've moved each individual schema for staging and production into two environment-specific DBs so that it just uses the public schema regardless of environment. Unblocked on this but definitely will follow along for updates. Thanks @janpio!

[iqbal125]

This is cool. Im using Vercel Storage Postgres which itself uses Neon database under the hood.

Will this work with Vercel Postgres?

[janpio]

Probably. Let us know when you try it out.

(Vercel technically has its own variant of it as @vercel/postgres, maybe that is a drop-in replacement that can also be used with this driver adapter - not sure: https://vercel.com/docs/storage/vercel-postgres/sdk)

[timothywadecook]

@iqbal125 did you have any issues using the neon adapter with Vercel Postgres? I'm thinking of using the same

[timothywadecook]

Does the Prisma Client need to be initialized in each request handler when using the prisma adapter or does the adapter handle connecting, using, and closing the objects?

From Neon Driver Docs link:
"The Pool and Client objects must be connected, used, and closed within a single request handler. Don't create the objects outside a request handler; don't create them in one handler and try to reuse them in another; and to avoid exhausting available connections, don't forget to close them."

I am using Prisma with a Nextjs app on Vercel using Vercel Postgres (neon). I tried following the docs for using prisma with neon serverless driver and tried running locally and I get the following error.

TypeError: bufferUtil.mask is not a function
    at module.exports.mask (webpack-internal:///(rsc)/./node_modules/ws/lib/buffer-util.js:96:29)
    at Sender.frame (webpack-internal:///(rsc)/./node_modules/ws/lib/sender.js:112:13)
    at Sender.send (webpack-internal:///(rsc)/./node_modules/ws/lib/sender.js:324:35)
    at WebSocket.send (webpack-internal:///(rsc)/./node_modules/ws/lib/websocket.js:371:22)
    at Timeout.eval [as _onTimeout] (webpack-internal:///(rsc)/./node_modules/@neondatabase/serverless/index.js:3090:25)
    at listOnTimeout (node:internal/timers:573:17)
    at process.processTimers (node:internal/timers:514:7)

I did not initialize the pool or prisma client inside of my handlers, I simply modified the file where I was previously instantiating prisma client:

import { neonConfig, Pool, PoolConfig } from "@neondatabase/serverless";
import { PrismaNeon } from "@prisma/adapter-neon";
import { PrismaClient } from "@prisma/client";
import ws from "ws";

import { env } from "@/env.mjs";

// types
declare global {
  var prisma: PrismaClient | undefined;
}

// setup
neonConfig.webSocketConstructor = ws;
const poolConfig: PoolConfig = { connectionString: env.POSTGRES_URL_NON_POOLING };

// instantiate
const pool = new Pool(poolConfig); // log:console.log?
const adapter = new PrismaNeon(pool);

const prisma = global.prisma || new PrismaClient({ adapter, log: ["info", "warn", "error"] });

// cache on global for HOT RELOAD in dev
if (process.env.NODE_ENV === "development") global.prisma = prisma;

export const db = prisma;

[iqbal125]

@timothywadecook Yeah Im gonna look into this and get back into it this week.

[timothywadecook]

Getting the following error every couple minutes in prod:

Uncaught Exception {"errorType":"error","errorMessage":"**terminating connection due to administrator command**","trace":["error: terminating connection due to administrator command"," at a.parseErrorMessage (/var/task/.next/server/chunks/343.js:9:15118)"," at a.handlePacket (/var/task/.next/server/chunks/343.js:9:11799)"," at a.parse (/var/task/.next/server/chunks/343.js:9:10450)"," at e.<anonymous> (/var/task/.next/server/chunks/343.js:9:15790)"," at e.emit (/var/task/.next/server/chunks/343.js:4:6560)"," at O.<anonymous> (/var/task/.next/server/chunks/343.js:6:5080)"," at g (/var/task/.next/server/chunks/343.js:28:15838)"," at O.i (/var/task/.next/server/chunks/343.js:28:16151)"," at O.emit (node:events:517:28)"," at g.W (/var/task/.next/server/chunks/343.js:29:10664)"]} Error: Runtime exited with error: exit status 129 (Uncaught Exception)

from Neon Docs: Link
The "terminating connection due to administrator command" error is typically encountered when running a query from a connection that has sat idle long enough for the compute endpoint to suspend due to inactivity.

Notes:
My Neon db is set to suspend compute after 30 seconds (default was 5 minutes).
I am instantiating prisma client in a single file and sharing across the app as mentioned previously in thread (consistent with recommended best practices for Prisma with Next - but contrary to Neon serverless driver instructions).

@janpio Any thoughts?

[iqbal125]

@timothywadecook Did you try adding pgbouncer=true to the connection url?

https://neon.tech/docs/guides/prisma#use-connection-pooling-with-prisma

[iqbal125]

@timothywadecook I think Im just going to go the direct Neon route. Vercel offers Neon integration so might be easier to do that.

[timothywadecook]

thanks @iqbal125 adding pgbouncer back to the connection url appears to have solved the issue.

[timothywadecook]

I did have a type error in the same place, and realized I needed to update Prisma Client version to latest / same as prisma adapter.

On Sat, Dec 30, 2023 at 1:56 PM iqbal125 ***@***.***> wrote: @timothywadecook <https://github.com/timothywadecook> Are you getting this type error in Vscode by any chance? image.png (view on web) <https://github.com/prisma/prisma/assets/24860061/d17771b6-d4b6-43f5-972c-577f0d048170> — Reply to this email directly, view it on GitHub <#21346 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJX24FGLPVDCV2YUHNKGWQ3YMBPUJAVCNFSM6AAAAAA5S6FPBSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TSOBQGEZDA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Timothy W Cook 678-371-3074 ***@***.***

[iqbal125]

Thanks I was getting that same error as you, for it said "t.mask" is not a function, but installing bufferUtil still fixed it, thanks for that.

[iqbal125]

@timothywadecook Seems to be working so far, will do more testing.

[iqbal125]

@janpio Do you have any techniques on how to test this? and also test nextjs caching features.

[NicHaley]

Installing bufferUtil fixed it for me as well.

[Jolg42]

This is documented in Neon's docs now:

If you encounter a TypeError: bufferUtil.mask is not a function error when building your application, this is likely due to a missing dependency that the ws module requires when using Client and Pool constructs. You can address this requirement by installing the bufferutil package: npm i bufferutil --save-dev

From https://neon.tech/docs/guides/prisma

[dani-z]

Hey 👋
I am trying to use this in a TRPC configuration but I am keep seeing this error, any ideas how I could fix this? I am indeed calling the mutation from a client component in this case.

prisma:error All attempts to open a WebSocket to connect to the database failed. Please refer to https://github.com/neondatabase/serverless/blob/main/CONFIG.md#websocketconstructor-typeof-websocket--undefined. Details: ws does not work in the browser. Browser clients must use the native WebSocket object
>>> tRPC Error on 'ad.create'  [TRPCError: All attempts to open a WebSocket to connect to the database failed. Please refer to https://github.com/neondatabase/serverless/blob/main/CONFIG.md#websocketconstructor-typeof-websocket--undefined. Details: ws does not work in the browser. Browser clients must use the native WebSocket object] {
  code: 'INTERNAL_SERVER_ERROR',
  name: 'TRPCError'
}

LE: Digging into it a bit, now that /lib/prisma.ts I have definitely does not run on the client so I am unsure what is happening. Also, replacing ws with native WebSocket seems to be fixing the issue, I just did not yet test it from a server component.

neonConfig.webSocketConstructor = WebSocket;

Thanks 🙏

[janpio]

Best open a bug report issue with a reproduction of what you are doing, then we can take a look. Lots of moving parts here it seems - so that is better solved via its own issue. Thanks! (And don't forget to post a link to the issue here when you created it)

[LucasWinkler]

Did you ever figure out a solution to this? I get the same WebSocket error while using middleware or nextauth with database strategy, or nextauth with jwt strategy but calling my database in the callbacks.

Edit: Sorry I just read that you tried the native WebSocket. I'm currently testing this myself.

Edit 2: Tried the native WebSocket and the error was gone. Tried it in prod and my website was extremely slow and things weren't working correctly. Tried to swap out WebSocket and ws based on node env but it did not solve it. I currently rely on ws it seems or else my project times out on vercel but I can't get past this same ws error

[adamchipperfield]

I have the following client in my Next.js app, connecting to a local Postgres database:

import { Pool } from '@neondatabase/serverless'
import { PrismaNeon } from '@prisma/adapter-neon'
import { PrismaClient } from '@prisma/client'

export const db = new PrismaClient({
  adapter: new PrismaNeon(
    new Pool({ connectionString: process.env.DATABASE_URL })
  )
})

When I try to use this in a Server Action, I get a connection error (I confirmed that the database is up and running by switching back to the original Prisma client config where everything worked):

Error: connect ECONNREFUSED 127.0.0.1:5433

When I use it in middleware (edge runtime), I get a rather vague error:

Error: {"source":"edge-server","message":"","stack":"Error\n    at eval (webpack-internal:///(middleware)/./node_modules/@neondatabase/serverless/index.mjs:1355:74)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async PrismaNeon.performIO (webpack-internal:///(middleware)/./node_modules/@prisma/adapter-neon/dist/index.mjs:385:79)\n    at async PrismaNeon.queryRaw (webpack-internal:///(middleware)/./node_modules/@prisma/adapter-neon/dist/index.mjs:347:17)\n    at async eval (webpack-internal:///(middleware)/./node_modules/@prisma/client/runtime/wasm.js:2:7458)"}

Any help would be appreciated! It's just a hobby project for now, so thought I'd give these new adapters a try and provide some feedback 👍

[janpio]

The Neon serverless driver can not talk to any local PostgreSQL. It has to talk to a Neon Postgre, Vercel Postgres - or a local proxy in front of your database that offer the Websocket API that @neondatabase/serverless requires. See my parallel response here: #21394 (comment)

[aaly00]

Facing the same issue on vercel serverless function with neon serverless driver.

Very tricky to troubleshoot.

Uncaught Exception: TypeError: t.mask is not a function
at e.exports.mask (/var/task/.next/server/chunks/8067.js:32:55843)
at d.frame (/var/task/.next/server/chunks/8067.js:32:74646)
at d.send (/var/task/.next/server/chunks/8067.js:32:76715)
at R.send (/var/task/.next/server/chunks/8067.js:33:9194)
at Timeout._onTimeout (/var/task/.next/server/chunks/8067.js:52:953)
at listOnTimeout (node:internal/timers:569:17)
at process.processTimers (node:internal/timers:512:7)
Node.js process exited with exit status: 129. The logs above can help with debugging the issue.
Unknown application error occurred

[Jolg42]

Hi @aaly00
I don't know all the details about this yet, but this should help you:

If you encounter a TypeError: bufferUtil.mask is not a function error when building your application, this is likely due to a missing dependency that the ws module requires when using Client and Pool constructs. You can address this requirement by installing the bufferutil package: npm i bufferutil --save-dev

From https://neon.tech/docs/guides/prisma

Let us know if that helped.