AWS Cost & Usage Reports

This Terraform module stands up a Cost and Usage Report, together with necessary services making the CUR data queryable in Athena.

Overview

The overall architecture looks like the illustration below

1. AWS delivers Cost and Usage Reports data to the S3 bucket continuously
2. Whenever new CUR data is delivered, a Glue Crawler makes sure the newly available CUR data is processed and made available in the Data Catalog
3. Athena provides an SQL interface to the CUR data, using the Data Catalog as its data source
4. QuickSight visualizes the data returned from querying Athena

Requirements

Name	Version
terraform	>= 0.13, < 2.0
archive	~> 2.0
aws	~> 3.29

Providers

Name	Version
aws.cur	~> 3.29
archive	~> 2.0
aws	~> 3.29

Inputs

Name	Description	Type	Default	Required
use_existing_s3_bucket	Whether to use an existing S3 bucket or create a new one. Regardless, s3_bucket_name must contain the name of the bucket.	bool	n/a	yes
s3_bucket_name	Name of the S3 bucket into which CUR will put the cost data.	string	n/a	yes
s3_use_existing_kms_key	Whether to use an existing KMS CMK for S3 SSE.	bool	n/a	yes
s3_kms_key_alias	Alias for the KMS CMK, existing or otherwise.	string	""	no
report_name	Name of the Cost and Usage Report which will be created.	string	n/a	yes
report_frequency	How often the Cost and Usage Report will be generated. HOURLY or DAILY.	string	n/a	yes
report_versioning	Whether reports should be overwritten or new ones should be created.	string	n/a	yes
report_format	Format for report. Valid values are: textORcsv, Parquet. If Parquet is used, then Compression must also be Parquet.	string	n/a	yes
report_compression	Compression format for report. Valid values are: GZIP, ZIP, Parquet. If Parquet is used, then format must also be Parquet.	string	n/a	yes
report_additional_artifacts	A list of additional artifacts. Valid values are: REDSHIFT, QUICKSIGHT, ATHENA. When ATHENA exists within additional_artifacts, no other artifact type can be declared and report_versioning must be OVERWRITE_REPORT.	set(string)	n/a	yes
s3_bucket_prefix	Prefix in the S3 bucket to put reports.	string	""	no
cur_role_arn	ARN of the role to assume in order to provision the Cost and Usage Reports S3 bucket in us-east-1.	string	""	no
cur_role_session_name	Session name to use when assuming cur_role_arn.	string	""	no
lambda_log_group_retention_days	Number of days to retain logs from the Lambda function, which ensures Glue Crawler runs when new CUR data is available.	number	14	no
glue_crawler_create_log_group	Whether to create a CloudWatch Log Group for the Glue Crawler. Crawlers share Log Group, and this gives the option of managing the Log Group with retention through this module.	bool	true	no
glue_crawler_log_group_retention_days	Number of days to retain logs from the Glue Crawler, which populates the Athena table whenever new CUR data is available.	number	14	no
tags	Tags which will be applied to provisioned resources.	map(string)	{}	no

Outputs

Name	Description
s3_bucket_name	Name of S3 bucket used for storing CUR data. This may be provisioned by this module or not.
s3_bucket_prefix	Prefix used for storing CUR data inside the S3 bucket.
s3_bucket_arn	ARN of S3 bucket used for storing CUR data. This may be provisioned by this module or not.
s3_bucket_region	Region where the S3 bucket used for storing CUR data is provisioned. This may be provisioned by this module or not.
report_name	Name of the provisioned Cost and Usage Report.
lambda_crawler_trigger_arn	ARN of the Lambda function responsible for triggering the Glue Crawler when new CUR data is uploaded into the S3 bucket.
lambda_crawler_trigger_role_arn	ARN of the IAM role used by the Lambda function responsible for starting the Glue Crawler.
crawler_arn	ARN of the Glue Crawler responsible for populating the Catalog Database with new CUR data.
crawler_role_arn	ARN of the IAM role used by the Glue Crawler responsible for populating the Catalog Database with new CUR data.
glue_catalog_database_name	Name of the Glue Catalog Database which is populated with CUR data.

References

It is based on AWS: Query and Visualize AWS Cost and Usage. Check out the blog post and the linked resources for an explanation of the concepts.

For more information about Cost & Usage Reports in general, see AWS: What are Cost and Usage Reports?

Requirements

Name	Version
terraform	>= 0.13, < 2.0
archive	~> 2.0
aws	~> 3.29

Providers

Name	Version
aws.cur	~> 3.29
aws	~> 3.29
archive	~> 2.0

Inputs

Name	Description	Type	Default	Required
use_existing_s3_bucket	Whether to use an existing S3 bucket or create a new one. Regardless, s3_bucket_name must contain the name of the bucket.	bool	n/a	yes
s3_bucket_name	Name of the S3 bucket into which CUR will put the cost data.	string	n/a	yes
s3_use_existing_kms_key	Whether to use an existing KMS CMK for S3 SSE.	bool	n/a	yes
s3_kms_key_alias	Alias for the KMS CMK, existing or otherwise.	string	""	no
report_name	Name of the Cost and Usage Report which will be created.	string	n/a	yes
report_frequency	How often the Cost and Usage Report will be generated. HOURLY or DAILY.	string	n/a	yes
report_versioning	Whether reports should be overwritten or new ones should be created.	string	n/a	yes
report_format	Format for report. Valid values are: textORcsv, Parquet. If Parquet is used, then Compression must also be Parquet.	string	n/a	yes
report_compression	Compression format for report. Valid values are: GZIP, ZIP, Parquet. If Parquet is used, then format must also be Parquet.	string	n/a	yes
report_additional_artifacts	A list of additional artifacts. Valid values are: REDSHIFT, QUICKSIGHT, ATHENA. When ATHENA exists within additional_artifacts, no other artifact type can be declared and report_versioning must be OVERWRITE_REPORT.	set(string)	n/a	yes
s3_bucket_prefix	Prefix in the S3 bucket to put reports.	string	""	no
cur_role_arn	ARN of the role to assume in order to provision the Cost and Usage Reports S3 bucket in us-east-1.	string	""	no
cur_role_session_name	Session name to use when assuming cur_role_arn.	string	null	no
lambda_log_group_retention_days	Number of days to retain logs from the Lambda function, which ensures Glue Crawler runs when new CUR data is available.	number	14	no
glue_crawler_create_log_group	Whether to create a CloudWatch Log Group for the Glue Crawler. Crawlers share Log Group, and this gives the option of managing the Log Group with retention through this module.	bool	true	no
glue_crawler_log_group_retention_days	Number of days to retain logs from the Glue Crawler, which populates the Athena table whenever new CUR data is available.	number	14	no
tags	Tags which will be applied to provisioned resources.	map(string)	{}	no

Outputs

Name	Description
s3_bucket_name	Name of S3 bucket used for storing CUR data. This may be provisioned by this module or not.
s3_bucket_prefix	Prefix used for storing CUR data inside the S3 bucket.
s3_bucket_arn	ARN of S3 bucket used for storing CUR data. This may be provisioned by this module or not.
s3_bucket_region	Region where the S3 bucket used for storing CUR data is provisioned. This may be provisioned by this module or not.
report_name	Name of the provisioned Cost and Usage Report.
lambda_crawler_trigger_arn	ARN of the Lambda function responsible for triggering the Glue Crawler when new CUR data is uploaded into the S3 bucket.
lambda_crawler_trigger_role_arn	ARN of the IAM role used by the Lambda function responsible for starting the Glue Crawler.
crawler_arn	ARN of the Glue Crawler responsible for populating the Catalog Database with new CUR data.
crawler_role_arn	ARN of the IAM role used by the Glue Crawler responsible for populating the Catalog Database with new CUR data.
glue_catalog_database_name	Name of the Glue Catalog Database which is populated with CUR data.