Skip to content

Add GlueCatalogCreateTableOperator#66362

Merged
vincbeck merged 1 commit intoapache:mainfrom
john-jac:add-glue-catalog-create-table
May 4, 2026
Merged

Add GlueCatalogCreateTableOperator#66362
vincbeck merged 1 commit intoapache:mainfrom
john-jac:add-glue-catalog-create-table

Conversation

@john-jac
Copy link
Copy Markdown
Contributor

@john-jac john-jac commented May 4, 2026

What

Add GlueCatalogCreateTableOperator to create tables in AWS Glue Data Catalog databases.

Why

Tables are the most commonly used Glue Catalog resource in data pipelines — defining schema for Athena queries, Spark jobs, and ETL workflows. This extends the existing database operators to cover the full catalog hierarchy (database → table).

What was done

  • Operator: GlueCatalogCreateTableOperator with if_exists idempotency, table_input for schema definition, optional catalog_id. Added to existing glue_catalog.py.
  • Unit tests: Added to existing test_glue_catalog.py — happy path, skip-existing, fail-on-conflict, template fields
  • System test: Updated example_glue_catalog.py — added table creation between database create/delete
  • Docs: Updated glue_catalog.rst with new section

@john-jac john-jac requested a review from o-nikolas as a code owner May 4, 2026 15:09
@boring-cyborg boring-cyborg Bot added area:dev-tools area:providers backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch kind:documentation provider:amazon AWS/Amazon - related issues labels May 4, 2026
@john-jac john-jac force-pushed the add-glue-catalog-create-table branch 3 times, most recently from 2f69e69 to d4c0189 Compare May 4, 2026 15:31
Copy link
Copy Markdown
Contributor

@vincbeck vincbeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check which files are being committed before creating the PR

Copy link
Copy Markdown
Contributor

@vincbeck vincbeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check which files are being committed before creating the PR

@john-jac john-jac force-pushed the add-glue-catalog-create-table branch 5 times, most recently from ba724dc to 906103c Compare May 4, 2026 16:40
@vincbeck
Copy link
Copy Markdown
Contributor

vincbeck commented May 4, 2026

Static checks are failing

@john-jac john-jac force-pushed the add-glue-catalog-create-table branch from 906103c to 444e8a7 Compare May 4, 2026 17:59
Comment thread providers/amazon/tests/system/amazon/aws/example_glue_catalog.py
@john-jac john-jac force-pushed the add-glue-catalog-create-table branch 2 times, most recently from 18606ad to 5c2dee9 Compare May 4, 2026 18:30
@john-jac john-jac force-pushed the add-glue-catalog-create-table branch from 5c2dee9 to 5e11189 Compare May 4, 2026 19:28
@vincbeck vincbeck removed the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label May 4, 2026
@vincbeck vincbeck merged commit e9691ee into apache:main May 4, 2026
94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants