AWS::Glue::Crawler support for triggering off of S3 event notifications

[Ricapar]

Name of the resource

AWS::Glue::Crawler

Resource name

No response

Description

AWS recently announced support for having Glue jobs trigger off of S3 event notifications off of an SQS queue [1]. This new feature exciting as it should greatly improve crawler performance for data lakes where there is a large amount of data changing, as well as helping eliminate some unnecessary Lambdas in certain architectures.

The feature doesn't seem to be yet supported by CloudFormation, per the docs for the S3Target structure defined within a Glue Crawler [2].

[1] https://aws.amazon.com/about-aws/whats-new/2021/10/aws-glue-crawlers-amazon-s3-notifications/
[2] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-crawler-s3target.html

Other Details

No response

[kgduong]

CloudFormation Support For S3 Event isn't currently available.

S3 Event Crawler's integration with CloudFormation is in scope and in the works. We plan on releasing this coverage to Cloudformation some later this year. Thank you for patience.