Skip to content

SecretsManager engine : aurora-postgresql support #3187

@senorcinco

Description

@senorcinco

Describe the bug

I'm trying to use a SecretsManager secret which is programmatically created with Terraform. The Aurora PostgreSQL engine comes back as aurora-postgresql. However, this causes the SDK to throw an invalid database error. Does it make sense to add the Aurora engines as valid databases types?

When I manually create an Aurora DB SecretsManager secret and use the "Credentials for Amazon RDS database" option, the engine is set to postgresql.

Terraform Aurora PostgreSQL resource:

$ terraform state show module.analytics-db.aws_rds_cluster.aurora
# module.analytics-db.aws_rds_cluster.aurora:
resource "aws_rds_cluster" "aurora" {
    ...
    endpoint                              = "analytics.cluster-c5lm123456789.us-east-1.rds.amazonaws.com"
    engine                                = "aurora-postgresql"
    port                                  = 5432
    ...

SecretsManager secret:

{
   "engine": "aurora-postgresql", 
   "host": "analytics.cluster-c5lm123456789.us-east-1.rds.amazonaws.com", 
   "username": "svc_lambda", 
   "password": "XXXXXXXX", 
   "port": 5432, 
   "dbInstanceIdentifier": "analytics", 
   "dbname": "analytics123"
}

SDK Error:

Invalid connection type (aurora-postgresql. It must be a postgresql connection.)
Traceback (most recent call last):
  File "lambda_handler.py", line 290, in <module>
    lambda_handler('', '')
  File "lambda_handler.py", line 248, in lambda_handler
    raise e
  File "lambda_handler.py", line 245, in lambda_handler
    db_con = open_db()
  File "lambda_handler.py", line 25, in open_db
    con = wr.postgresql.connect(secret_id=DBSECRET,ssl_context=ssl_context)
  File "/home/user/.local/lib/python3.10/site-packages/awswrangler/_utils.py", line 126, in inner
    return func(*args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/awswrangler/postgresql.py", line 229, in connect
    raise exceptions.InvalidDatabaseType(
awswrangler.exceptions.InvalidDatabaseType: Invalid connection type (aurora-postgresql. It must be a postgresql connection.)

Valid RDS Engines:

$ aws rds describe-db-engine-versions --query 'DBEngineVersions[*].[Engine]' --output text | sort -u
aurora-mysql
aurora-postgresql
custom-sqlserver-ee
custom-sqlserver-se
custom-sqlserver-web
db2-ae
db2-se
docdb
mariadb
mysql
neptune
oracle-ee
oracle-ee-cdb
oracle-se2
oracle-se2-cdb
postgres
sqlserver-ee
sqlserver-ex
sqlserver-se
sqlserver-web

How to Reproduce

Create a secret with engine set to aurora-postgresql.

import awswrangler as wr
import ssl

DBSECRET="arn:aws:secretsmanager:us-east-1:123456789012:secret:analytics/svc_lambda-12345"

ssl_context = ssl.create_default_context()
ssl_context.verify_mode = ssl.CERT_REQUIRED
ssl_context.load_verify_locations('./global-bundle.pem')
con = wr.postgresql.connect(secret_id=DBSECRET,ssl_context=ssl_context)

Expected behavior

SDK supports all valid RDS engine types.

Your project

No response

Screenshots

No response

OS

Linux

Python version

3.13

AWS SDK for pandas version

3.12.1

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions