Skip to content

Add GlueCatalogCreateDatabaseOperator#65977

Merged
vincbeck merged 1 commit intoapache:mainfrom
john-jac:add-glue-catalog-create-database
Apr 28, 2026
Merged

Add GlueCatalogCreateDatabaseOperator#65977
vincbeck merged 1 commit intoapache:mainfrom
john-jac:add-glue-catalog-create-database

Conversation

@john-jac
Copy link
Copy Markdown
Contributor

What

Add a new operator for managing databases in the AWS Glue Data Catalog.

The Glue Data Catalog is the centralized metadata repository used by Athena, Redshift Spectrum, EMR, and Lake Formation. Currently there are no operators for catalog database management — users must use PythonOperator with raw boto3 calls.

How

  • New GlueCatalogCreateDatabaseOperator using AwsBaseOperator[AwsBaseHook] pattern (no custom hook)
  • Supports description, location_uri, parameters, catalog_id, and tags
  • Returns the database name
  • Added to existing AWS Glue integration in provider.yaml

Files

  • operators/glue_catalog.py — operator
  • test_glue_catalog.py — unit tests (3 tests)
  • example_glue_catalog.py — system test
  • glue_catalog.rst — docs
  • provider.yaml + get_provider_info.py — updated

Testing

  • Unit tests: 3 passed
  • mypy: no issues
  • Static checks (prek): 57 passed, 0 PR-related failures

Comment thread providers/amazon/src/airflow/providers/amazon/aws/operators/glue_catalog.py Outdated
Comment thread providers/amazon/tests/system/amazon/aws/example_glue_catalog.py
@john-jac john-jac force-pushed the add-glue-catalog-create-database branch 2 times, most recently from faa5579 to 75764ad Compare April 27, 2026 22:19
Comment thread providers/amazon/tests/system/amazon/aws/example_glue_catalog.py Outdated
@john-jac john-jac force-pushed the add-glue-catalog-create-database branch from 75764ad to b40e776 Compare April 27, 2026 22:57
- New operator to create databases in the AWS Glue Data Catalog
- Supports description, location_uri, parameters, catalog_id, and tags
- Uses AwsBaseOperator[AwsBaseHook] pattern (no custom hook)
- Unit tests, system test, and docs included
- provider.yaml and get_provider_info.py updated
@john-jac john-jac force-pushed the add-glue-catalog-create-database branch from b40e776 to e67b7dd Compare April 27, 2026 23:49
@vincbeck vincbeck merged commit 7503189 into apache:main Apr 28, 2026
94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants