cloudfront: support for cache and origin request policies

[zarnovican]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Amazon recently introduced CloudFront Cache Policy, Origin Request Policy. In short, caching configuration can be factored out of distribution config. Distribution can then reference existing, shared, policy. Several important features were added as well. Like decoupling headers contributing to caching key from the headers in origin request. New "Viewer" headers were added as well.

New or Affected Resource(s)

• aws_cloudfront_distribution - CF has to be able to switch to policy-based config (per behavior)
• aws_cloudfront_cache_policy - new resource(s). Name is to be defined.
• aws_cloudfront_origin_request_policy - new resource(s). Name is to be defined.

Potential Terraform Configuration

# custom policy
resource "aws_cloudfront_cache_policy" "test1" {
  name = "test1"
  comment = ".."
  ...
}

resource "aws_cloudfront_distribution" "foo" {
  ..
  ordered_cache_behavior {
    ..
    cache_policy              = aws_cloudfront_cache_policy.test1.name
    origin_request_policy = "Managed-AllViewer"               # AWS-managed policy
  }
}

References

• https://aws.amazon.com/blogs/networking-and-content-delivery/amazon-cloudfront-announces-cache-and-origin-request-policies/
• https://aws.amazon.com/about-aws/whats-new/2020/07/cloudfront-geolocation-headers/

[ewbankkit]

Requires AWS SDK v1.33.8:

• Update module aws/aws-sdk-go to v1.33.21 #14000

[roberth-k]

I'll pick this up.

[lfwelty]

are aws_cloudfront_cache_policy and aws_cloudfront_origin_request_policy ready to use?

[bvilnis]

Come across the need for this at work recently. Any word on when cache policies will be able to Terraform?

[timbaileyjones]

I need both OriginRequesPolicies AND CachePolicies.

[t0rr3sp3dr0]

Any updates on this?

[ashwini-hopin]

Would be nice to know when this change will be ready for use :)

[maneja805]

It would be great if can share details about it, as this is one of the most important features for CloudFront automation for maintaining multiple behaviors cache(via pre-defined cache policies ids- available in cloud formation) via terraform.

[amerinero]

One more waiting for this feature

[ItamarLevin]

also looking forward to it.

[lpruthvi]

Waiting for this

[breathingdust]

Hi all! 👋 Just wanted to direct you to our public roadmap for this quarter (Nov-Jan) in which this item has been mentioned.

Due to the significant community interest in support for this feature, we will be looking at merging existing contributions soon. 

We appreciate all the contributions and feedback thus far.

Look out for support in the provider soon!

[EthanDavis]

Any word on when this will be released?

[jessefarinacci]

I think to be effective we'd also want data.aws_cloudfront_cache_policy and data.aws_cloudfront_origin_request_policy so that the polices can be managed outside of any particular distribution/module, and then utilized by specific distributions. The data filter would likely just be the name field and that'd be sufficient to be useful, but by id would also be nice, too, and be more in line with the existing cli functionality. Thanks!

[bill-rich]

@EthanDavis I am currently working on this one, and I expect it will be released in the next week or two.

@jessefarinacci Agreed. I'm working on covering the data sources for the new policies also.

[pen-pal]

@bill-rich any alternatives for this until it get's released?

[garthkerr]

You can temporarily use aws_cloudfront_distribution in combination with lifecycle.ignore_changes and a null resource with the distribution state. Not ideal but will work for local runs until the provider is updated.

resource "null_resource" "cloudfront_config" {
  triggers = {
    config_sha1 = sha1(file("./null-cloudfront.json"))
  }

  provisioner "local-exec" {
    command = "./null-cloudfront-apply.sh"
  }

  depends_on = [aws_cloudfront_distribution.main]
}

# whatever your resource name is here and < pipe the state file or pull remote state
# you could also just reference the identifier directly
CF_DISTRIBUTION=$(jq -r \
  '.resources[]
    | select(.type == "aws_cloudfront_distribution" and .name == "main")
    | .instances[0].attributes.id' \
  < terraform.tfstate)

# fetch the current state to create a backup, make modifications
aws cloudfront get-distribution-config --id "${CF_DISTRIBUTION}" > null-cloudfront.json

# fetch the etag of the current version
CF_ETAG=$(aws cloudfront get-distribution-config --id "${CF_DISTRIBUTION}" | jq -r '.ETag')

aws cloudfront update-distribution \
  --id "${CF_DISTRIBUTION}" \
  --distribution-config file://null-cloudfront.json \
  --if-match "${CF_ETAG}"

[pen-pal]

Hey @garthkerr thank you for that.
But since it's in production and the state are in remote (s3 and dynamodb), how I should I approach this?
PS: I am quite new in terraform so please mind the question. 😅

[garthkerr]

@M-A-N-I-S-H-K this won't work for remote runs. But if it's just remote state, you can pipe it in like:

CF_DISTRIBUTION=$(terraform state pull | jq -r '.resources[] | select(.type == "aws_cloudfront_distribution" and .name == "main") | .instances[0].attributes.id')
CF_ETAG=$(aws cloudfront get-distribution-config --id "${CF_DISTRIBUTION}" | jq -r '.ETag')

# make changes to `null-cloudfront.json` as needed.
aws cloudfront get-distribution-config --id "${CF_DISTRIBUTION}" > null-cloudfront.json

# update the distribution
aws cloudfront update-distribution --id "${CF_DISTRIBUTION}" --distribution-config file://null-cloudfront.json --if-match "${CF_ETAG}"

I have this wrapped in a null_resource for running the script with apply:

resource "null_resource" "cloudfront_config" {
  triggers = {
    config_sha1 = sha1(file("./null-cloudfront.json"))
  }

  provisioner "local-exec" {
    command = "./null-cloudfront-apply.sh"
  }

  depends_on = [aws_cloudfront_distribution.main]
}

[pen-pal]

Eagerly waiting for it to be merged.

[abcfy2]

Hi all! Just wanted to direct you to our public roadmap for this quarter (Nov-Jan) in which this item has been mentioned.

Due to the significant community interest in support for this feature, we will be looking at merging existing contributions soon. 

We appreciate all the contributions and feedback thus far.

Look out for support in the provider soon!

I can't find this feature in ROADMAP any more, did you give up to support this feature ?

[lejeunen]

@abcfy2 It's still there, first item under Workflow Improvements

[abcfy2]

@lejeunen Oh, the default branch is main now, and this feature does not find in main branch READMAP.md: https://github.com/hashicorp/terraform-provider-aws/blob/main/ROADMAP.md

[ofhouse]

No need to worry about the roadmap, aws_cloudfront_origin_request_policy resource was released yesterday:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_origin_request_policy

And I expect that support for cache policies is following soon: #17336

Edit:
For everybody who wants to try the new resource aws_cloudfront_origin_request_policy:
We have to wait until cache policies are released, since request policies can only used together with cache policies.
(So the release of request policies without cache policies is useless at this point)

InvalidArgument: The parameter Origin request policy cannot be attached without a cache policy

[ewbankkit]

• aws_cloudfront_cache_policy resource and data source released in Terraform AWS Provider v3.28.0
• aws_cloudfront_origin_request_policy resource and data source released in Terraform AWS Provider v3.27.0

[ebdxflr]

How do i use an aws_cloudfront_origin_request_policy after its created inside my cache behavior. For the **aws_cloudfront_cache_policy ** I can use cache_policy_id but what can I use with aws_cloudfront_origin_request_policy @ewbankkit ?

[ofhouse]

How to add the origin_request_policy_id to the CloudFront distribution is currently missing in the documentation, but is fully implemented.
Here is an example how to integrate aws_cloudfront_origin_request_policy & aws_cloudfront_cache_policy into a CloudFront resource:

resource "aws_cloudfront_origin_request_policy" "this" {
  ...
}

resource "aws_cloudfront_cache_policy" "this" {
  ...
}

resource "aws_cloudfront_distribution" "distribution" {
  ...

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "origin_id"

    viewer_protocol_policy = "redirect-to-https"
    compress               = true

    origin_request_policy_id = aws_cloudfront_origin_request_policy.this.id
    cache_policy_id          = aws_cloudfront_cache_policy.this.id
  }
}

[ebdxflr]

one last question, how do i go about importing existing aws_cloudfront_cache_policy and aws_cloudfront_origin_request_policy? thanks for the previous reply

[ofhouse]

Existing resources should be importable via the corresponding data resources aws_cloudfront_origin_request_policy and aws_cloudfront_cache_policy:

data "aws_cloudfront_origin_request_policy" "this" {
  id = "acba4595-bd28-49b8-b9fe-13317c0390fa"
}

data "aws_cloudfront_cache_policy" "this" {
  id = "658327ea-f89d-4fab-a63d-7e88639e58f6"
}

resource "aws_cloudfront_distribution" "distribution" {
  ...

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "origin_id"

    viewer_protocol_policy = "redirect-to-https"
    compress               = true

    origin_request_policy_id = data.aws_cloudfront_origin_request_policy.this.id
    cache_policy_id          = data.aws_cloudfront_cache_policy.this.id
  }
}

So you could e.g. use managed policies from AWS:

• You can find managed origin policies here: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-origin-request-policies.html
• and managed Cache policies here: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-cache-policies.html

[ebdxflr]

im looking to import policies created manually in the console, not using the data sources :)

LE:
looks like using the policy ID works:

terraform import aws_cloudfront_cache_policy.example $policy_id_from_console

[pen-pal]

Hy team,

I have a previously created cloudfront distribution with forwarded_values values enabled.

    forwarded_values {
      query_string = true

      cookies {
        forward = "none"
      }
    }

Now when I try to implement cache_policy_id = xxxxx to be distribution by setting

    forwarded_values {
      query_string = none
      headers = null

      cookies {
        forward = "none"
      }
    }

Once I apply this, I get the following error.

Warning: Applied changes may be incomplete

The plan was created with the -target option in effect, so some changes
requested in the configuration may have been ignored and the output values may
not be fully updated. Run the following command to verify that no other
changes are pending:                                                               
    terraform plan                                                                 

Note that the -target option is not suitable for routine use, and is provided
only for exceptional situations such as recovering from errors or mistakes, or
when Terraform specifically suggests to use it as part of an error message.


Error: error updating CloudFront Distribution (EQ3US9107CM6F): InvalidArgument: The parameter ForwardedValues cannot be used when a cache policy is associated to the cache behavior.
        status code: 400, request id: 05af011a-723d-40db-b3b1-94a0cf6881f6

But when I create new distribution with only cache_policy_id = xxxxx it works fine.

Is there a way to implement cache_policy_id = xxxxx id on the previously created cloudfront distribution by disabling forwarded_values ??

[ElieSangeunLee]

FYI, I tried to use Managed policies provided by Amazon and below codes worked. I don't know why but when I used the id instead of name inside the data block, the cloudfront resource ignores origin_request_policy_id and cache_policy_id arguments at terraform plan and terraform apply

data "aws_cloudfront_origin_request_policy" "this" {
  name = "Managed-CORS-S3Origin"
}

data "aws_cloudfront_cache_policy" "this" {
  name = "Managed-CachingOptimized"
}

resource "aws_cloudfront_distribution" "distribution" {
  ...

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "origin_id"

    viewer_protocol_policy = "redirect-to-https"
    compress               = true

    origin_request_policy_id = data.aws_cloudfront_origin_request_policy.this.id
    cache_policy_id          = data.aws_cloudfront_cache_policy.this.id
  }
}

[hussamhadi]

I've used the terraform-aws module, in that case I needed to add the cache/origin-request policies to the depends_on attribute, otherwise the module would be created before the policies are read and the module would set those policies to null:

data "aws_cloudfront_origin_request_policy" "managed-allviewer" {
  name = "Managed-AllViewer"
}

data "aws_cloudfront_cache_policy" "managed-cache-disabled" {
  name = "Managed-CachingDisabled"
}

module "cdn" {
  source = "terraform-aws-modules/cloudfront/aws"
  depends_on = [
    data.aws_cloudfront_origin_request_policy.managed-allviewer,
    data.aws_cloudfront_cache_policy.managed-cache-disabled
]
...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!