This is an AWS Lake Formation project for CDK development with Python.
This project shows how to grant AWS Lake Formation permissions on Data Catalog Resources.
The cdk.json
file tells the CDK Toolkit how to execute your app.
This project is set up like a standard Python project. The initialization
process also creates a virtualenv within this project, stored under the .venv
directory. To create the virtualenv it assumes that there is a python3
(or python
for Windows) executable in your path with access to the venv
package. If for any reason the automatic creation of the virtualenv fails,
you can create the virtualenv manually.
To manually create a virtualenv on MacOS and Linux:
$ python3 -m venv .venv
After the init process completes and the virtualenv is created, you can use the following step to activate your virtualenv.
$ source .venv/bin/activate
If you are a Windows platform, you would activate the virtualenv like this:
% .venv\Scripts\activate.bat
Once the virtualenv is activated, you can install the required dependencies.
(.venv) $ pip install -r requirements.txt
At this point you can now synthesize the CloudFormation template for this code.
(.venv) $ cdk synth -c database_name=lf_testdb
Use cdk deploy
command to create the stack shown above.
(.venv) $ cdk deploy -c database_name=lf_testdb --require-approval never
ℹ️ Replace
lf_testdb
to your own database name in Glue Data Catalog.
To add additional dependencies, for example other CDK libraries, just add
them to your setup.py
file and rerun the pip install -r requirements.txt
command.
After all CDK stacks are successfully deployed, you can see a list of permissions on lf_testdb
to GlueJobRoleExample
IAM Role.
aws lakeformation list-permissions | jq -r '.PrincipalResourcePermissions[] | select(.Principal.DataLakePrincipalIdentifier | endswith("GlueJobRoleExample"))'
ℹ️
GlueJobRoleExample
is an IAM Role created by this project.
Delete the CloudFormation stack by running the below command.
(.venv) $ cdk destroy -c database_name=lf_testdb --force --all
cdk ls
list all stacks in the appcdk synth
emits the synthesized CloudFormation templatecdk deploy
deploy this stack to your default AWS account/regioncdk diff
compare deployed stack with current statecdk docs
open CDK documentation
Enjoy!
aws lakeformation list-permissions
Returns a list of the principal permissions on the resource, filtered by the permissions of the caller. See moreaws lakeformation grant-permissions
Grants permissions to the principal to access metadata in the Data Catalog and data organized in underlying data storage such as Amazon S3. See moreaws lakeformation revoke-permissions
Revokes permissions to the principal to access metadata in the Data Catalog and data organized in underlying data storage such as Amazon S3. See moreaws lakeformation list-resources
Lists the resources registered to be managed by the Data Catalog. See more
- AWS Lake Formation Workshop
- AWS Lake Formation - Create a data lake administrator
- AWS Lake Formation - Granting Data Catalog permissions using the named resource method
- AWS Lake Formation Permissions Reference
- Troubleshooting Lake Formation
- Known issues for AWS Lake Formation
- AWS Lake Formation - Working with other AWS services