Add support for Aurora Serverless cluster (via Data API?)

[victorlin]

Currently, I am using sqlalchemy-aurora-data-api + pandas.DataFrame.to_sql() to update data on an Aurora Serverless DB cluster:

from sqlalchemy import create_engine

df = pd.DataFrame(...)
cluster_arn = 'arn:aws:rds:us-east-1:123456789012:cluster:mydbcluster'
secret_arn = 'arn:aws:secretsmanager:us-east-1:123456789012:secret:mysecret'

engine = create_engine('postgresql+auroradataapi://:@/my_db',
    connect_args=dict(aurora_cluster_arn=cluster_arn, secret_arn=secret_arn))
con = engine.connect()
df.to_sql('table_name', con, if_exists='append', index=False)

This is already pretty simple, but I'm wondering if awswrangler could simplify the connection a bit, e.g.:

import awswrangler as wr

cluster_name = 'mydbcluster'
dbname = 'my_db'
secret_arn = 'arn:aws:secretsmanager:us-east-1:123456789012:secret:mysecret'

con = wr.postgresql.connect(
    aurora_cluster=cluster_name,
    dbname=dbname,
    secret_id=secret_arn)
wr.postgresql.to_sql(df, 'table_name', schema='public', con=con)

I have already tried the current wr.postgresql.connect() but there's some additional VPC connection overhead which I couldn't get around. sqlalchemy-aurora-data-api leverages the Aurora Serverless Data API which is publicly accessible.

[kukushking]

Agree, that would be a nice feature.

[kukushking]

Closing the issue as the functionality is covered by #798 and will be included in the next release. Thanks again to @pwithams for contribution!

I'll also take this opportunity to advertise and invite you folks @pwithams @victorlin to our AWS Data Wrangler Community Slack. This is a good place for direct communication with maintainers and top contributors.