Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support PostgreSQL #10

Open
metaskills opened this issue Dec 29, 2019 · 5 comments
Open

Support PostgreSQL #10

metaskills opened this issue Dec 29, 2019 · 5 comments

Comments

@metaskills
Copy link
Member

@metaskills metaskills commented Dec 29, 2019

FORWARD: I'm not sure this work is worth it given RDS proxy is in preview. https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-rds-proxy-available-in-preview/ However, if someone does want to push on this work, here are some thoughts since Aurora Serverless does support PostgreSQL.

During our initial work (#6) we did leave a good module & client structure in place so that we could mixing PostgreSQL client hacks as needed. My biggest question is, how would we handle loading, move the requires to the core ConnectionHandler?

@Genkilabs

This comment has been minimized.

Copy link

@Genkilabs Genkilabs commented Jan 7, 2020

This is very interesting. Currently the docs for the RDS Proxy preview include the points:

  • Currently, RDS Proxy is only available for the MySQL engine family. This engine family includes RDS MySQL 5.6 and 5.7, and Aurora versions 1 and 2. Proxy support for PostgreSQL databases isn't available in the public preview.
  • You can't use RDS Proxy with Aurora Serverless clusters.

Meaning, RDS Proxy is not really a viable option for people who won't/can't convert from PostgreSQL (or won't be for some time).

Right now, this gem is the closest thing I have come across for Rails -> RDS Serverless PSQL ...

@metaskills

This comment has been minimized.

Copy link
Member Author

@metaskills metaskills commented Jan 7, 2020

So for "RDS Proxy" this gem would be moot right? My understanding is that it uses the same mysql/pg clients and protocols. Is that incorrect? I've not dug too deep on it quite yet.

@Genkilabs

This comment has been minimized.

Copy link

@Genkilabs Genkilabs commented Jan 14, 2020

I believe that is their plan, but there is no timeline up yet that I could find.
We needed something now, so the pattern we ended up going with is:

Rails <-> net-ssh-gateway <-> Bastion AWS instance <-> Aurora Serverless

It let us use the default postgresql AR adapter at the cost of slightly more AWS moving parts.

For those who find their way here before RDS Proxy is available for their use case...
Basically, you can connect to serverless as normal (without this gem) if you connect through a TCP tunnel. For their own reasons, Amazon decided that for their serverless RDS product it would be locked down to ONLY allow access from within the same VPC, regardless of what the user wants to setup.

So, this approach to connect from an external ActiveRecord AbstractAdapter is to make an EC2 instance (or autoscaling group of them) and a security group that opens only one TCP port and has best practice keys/certs. Then use that bastion/jump-box to tunnel through.
The image can be just the default AWS Linux 2 box at size nano. But I recommend for something public facing folks make their own image and use the new Launch Template in combination with and Auto Scaling Group and a Load Balancer. Then benchmark and size up/out the boxes appropriately.

It takes some fiddling with the dashboards so I'm thinking to write up a gist when I get the time...
^_^

@metaskills

This comment has been minimized.

Copy link
Member Author

@metaskills metaskills commented Jan 14, 2020

Whoa, that's some absolute, next level, righteous hacks!

@joelvh

This comment has been minimized.

Copy link

@joelvh joelvh commented Jan 21, 2020

Hey @Genkilabs I started using RDS Serverless PostgreSQL from Lambda and ran into an issue where some queries would hang. I thought it was Sequel (stripped down our Lambda function from using Rails), but it might seem it's the pg gem.

I had hoped to try the RDS proxy or Data API, but your research above was informative about the former... leaving the latter as an alternative to pg if that's really the issue in my case.

Did you ever run into behavior where queries hung? If not, maybe it's related to Lambda and maybe how it access RDS Serverless over the network. I haven't found others reporting this issue, so I'm hoping to narrow it down to a configuration issue.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.