Skip to content

aws-samples/glue-workflow-aws-cdk

main
Switch branches/tags
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
Aug 26, 2021
Aug 26, 2021
Sep 27, 2021
Sep 30, 2021
Aug 26, 2021
Aug 26, 2021
Aug 26, 2021
Aug 26, 2021
Aug 26, 2021

Glue Workflow


Stability: Stable

This is a stable example. It should successfully build out of the box

This examples does is built on Construct Libraries marked "Stable" and does not have any infrastructure prerequisites to build.


This example creates a Glue Workflow containing multiple crawlers, glue jobs and triggers for the workflow. The workflow is manually triggered, but the script can be updated to run on a cron schedule.

Prerequisites

Prior to deployment, issue command aws configure and then follow the prompts to setup your AWS credentials and define the target account for the deployment.

Build

To build this app, you need to be in this example's root folder. Then run the following:

npm install -g aws-cdk
npm install
npm run build

This will install the AWS CDK, the necessary dependencies, and then build your TypeScript files and your CloudFormation template.

Deploy

  1. Run cdk bootstrap to bootstrap the stack and create the S3 bucket that will store the jobs' scripts.
  2. Run cdk deploy --all. This will deploy / redeploy your Stack to your AWS Account.

The --all arguement is required to deploy both stacks in this example.

After the deployment, browse to the Glue Console and manually launch the newly created Glue Workflow.

Synthesize Cloudformation Template

To see the Cloudformation template generated by the CDK, run cdk synth, then check the output file in the "cdk.out" directory.