Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any way to connect from localhost? #33

Closed
cosbgn opened this issue Jul 13, 2023 · 6 comments
Closed

Any way to connect from localhost? #33

cosbgn opened this issue Jul 13, 2023 · 6 comments

Comments

@cosbgn
Copy link

cosbgn commented Jul 13, 2023

I would like to use an offline local instance of postgress while developing on my computer, I'm trying something like this:

import { neon, neonConfig } from '@neondatabase/serverless'
import { drizzle } from 'drizzle-orm/neon-http'
 import schema from "../database/schema.js"
neonConfig.fetchConnectionCache = true
let cached_db = null

const useDb = () => {
	if (!cached_db){
                const uri = process.dev ? 'postgresql://localhost:5432/dbname' : process.env.NEON_URI
		const client = neon(uri)
		cached_db = drizzle(client, {schema})
	}
	return cached_db
}

export const db = useDb()

But I keep getting an error that my uri is wrong

@cosbgn cosbgn changed the title Any whay to connect from localhost? Any way to connect from localhost? Jul 13, 2023
@jawj
Copy link
Collaborator

jawj commented Jul 13, 2023

If you were connecting via WebSockets (by importing Client or Pool), this blog post by Gal at Vercel would have you covered: https://gal.hagever.com/posts/running-vercel-postgres-locally (it's just as applicable to Neon as to Vercel Postgres).

But: you're connecting via http. So I think you have two options:

(1) You could use plain 'pg' + 'drizzle-orm/node-postgres' (instead of our driver and 'drizzle-orm/neon-http') for development.

(2) You could run our open-source proxy configured to talk to a local Postgres DB. This is a bit more involved, but would look something like the following.

In your app, you'll need to set the (new) fetchEndpoint configuration option to conditionally switch the port to something non-privileged. For example:

import { neonConfig } from '@neondatabase/serverless';
neonConfig.fetchEndpoint = host => `https://${host}:${host === 'db.localtest.me' ? 4444 : 443}/sql`;  // fixed: thanks @marbemac

Then, set up the proxy like so:

# install Rust if not already available
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# get and build the proxy
git clone https://github.com/neondatabase/neon.git
cd neon
cargo build --bin proxy

# still in `neon` directory, create a self-signed cert for *.localtest.me (see https://readme.localtest.me/)
openssl req -new -x509 -days 365 -nodes -text -out server.crt -keyout server.key -subj "/CN=*.localtest.me"

# start local postgres -- ensure you have an appropriate DB + user set up with either `trust` or `scram-sha-256` auth

# still in `neon` directory, run the proxy (substituting in an appropriate postgres:// connection string)
./target/debug/proxy -c server.crt -k server.key --auth-backend=postgres \
  --auth-endpoint=postgres://user:password@db.localtest.me/main --wss=0.0.0.0:4444  # or: --wss='[::1]:4444'

# run your app, accepting the self-signed cert
NODE_TLS_REJECT_UNAUTHORIZED=0 node myfile.js

I'm aware this is a bit painful, and we're looking into ways we can make this simpler.

@cosbgn
Copy link
Author

cosbgn commented Jul 21, 2023

Thanks!

@cosbgn cosbgn closed this as completed Jul 21, 2023
@marbemac
Copy link

marbemac commented Aug 5, 2023

@jawj a couple of things I ran into when trying to get neon http working locally:

  1. Small typo in your fetchEndpoint fn example - need /sql at the end:
neonConfig.fetchEndpoint = host => `https://${host}:${host === 'db.localtest.me' ? 4444 : 443}/sql`;
  1. Node 17+ now prefers ipv6 by default when doing ip resolving (Unable to connect to establish a TCP connection (ECONNREFUSED) nodejs/node#40702 (comment)). So db.localtest.me turns into ::1, which ends up failing out as you can see in the screenshot below.
Screenshot 2023-08-05 at 2 34 55 PM

Two options:

  • Can configure at runtime with dns.setDefaultResultOrder('ipv4first');. However this doesn't work in edge runtimes like vercel or cloudflare, because no node:dns module.
  • Configure when running the app, so it becomes NODE_OPTIONS=--dns-result-order=ipv4first NODE_TLS_REJECT_UNAUTHORIZED=0 next dev

Soo yeah, overall quite a bit of work use local postgres with next dev (edge runtime) during local development. Tough ask to get rest of the dev team cloning and building the neon http proxy, setting up the certs, etc. Would be incredible if there was a docker image kind of like the neon wsproxy that we could pop into our docker-compose that worked with less configuration 🙏 ❤️ 😄.

@jawj
Copy link
Collaborator

jawj commented Aug 7, 2023

Thanks @marbemac.

  • I've fixed the code example to add the final /sql.
  • I think you can simply switch the argument from --wss=0.0.0.0:4444 to --wss='[::1]:4444' if IPV6 DNS is a problem.
  • I agree, we need to find a rather more painless solution for this!

@marbemac
Copy link

marbemac commented Aug 7, 2023

I think you can simply switch the argument from --wss=0.0.0.0:4444 to --wss='[::1]:4444' if IPV6 DNS is a problem.

❤️ this worked.

@TimoWilhelm
Copy link

Maybe this helps someone else that ends up here:

I've taken the steps described in #33 (comment) and created a Docker compose project to simplify the setup process for a local neon proxy + PostgreSQL DB.

You can find the repo here: https://github.com/TimoWilhelm/local-neon-http-proxy

The container also includes a small Caddy reverse proxy to setup the upstream TLS connection so the code does not need to trust the self-signed certificate of the neon proxy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants