Python script that will update S3 file paths in Iceberg metadata files (metadata.json + AVRO)

Example using the Iceberg register_table command with AWS Glue and Glue Data Catalog

Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with Amazon Data Firehose and DMS

Process DynamoDB change streams via. AWS Glue w Iceberg to keep a copy of a collection in S3 upto date

Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with AWS Glue Streaming and DMS

A poc open framework to manage data ingestion into apache iceberg tables

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming using Amazon MSK Serverless and MSK Connect (Debezium)

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming using Amazon MSK and MSK Connect (Debezium)

This is a collecton of Amazon CDK projects to show how to directly ingest streaming data from Amazon Mananged Service for Apache Kafka (MSK) and MSK Serverless into Apache Iceberg table in S3 with AWS Glue Streaming.

Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3

Sample code to collect Apache Iceberg metrics for table monitoring

A tool for learning Iceberg table format

Run an open-source data LakeHouse locally using Docker Compose

Automated setup of Apache Iceberg on Amazon S3 using Terraform and AWS Glue Data Catalog. Explore the power of a Lakehouse architecture for data management and analysis, featuring schema discovery, metadata management, and efficient querying with Amazon Athena.

"Apache Iceberg Connector for AWS Glue를 이용하여 데이터레이크 CRUD 하기" 포스팅 내용 실습 프로젝트