Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to create aws_waf_ipset from a list of IPs #9814

Closed
jmulloy-simplisafe opened this issue Nov 2, 2016 · 7 comments

Comments

Projects
None yet
9 participants
@jmulloy-simplisafe
Copy link

commented Nov 2, 2016

I would like to use terraform to load an AWS WAF whitelist and load the same IP whitelist into consul so our API apps can pick it up to enforce access by IP. I can't find any way to load a list of IPs into an aws_waf_ipset resource. This problem seems to have been solved for security groups by adding a separate aws_security_group_rule resource that can be iterated via count. Perhaps this would be the easiest solution for this problem.

I've also been trying to do the reverse, getting the IPs out of a manually configured aws_waf_ipset resource and turning it into a list that I could load into consul, but I can't get that to work either.

Also the documentation for this resource doesn't make it clear that repeating the stanza is how you add multiple IPs to the resource.

Terraform Version

0.7.8

Affected Resource(s)

  • aws_waf_ipset

Terraform Configuration Files

resource "aws_waf_ipset" "foobar_whitelist" {
  name = "tf-whitelist-test"
  ip_set_descriptors {
    type = "IPV4"
    value = "10.10.10.10/32"
  }
  ip_set_descriptors {
    type = "IPV4"
    value = "20.20.20.20/32"
  }
}
@sharmaanshul2102

This comment has been minimized.

Copy link
Contributor

commented Nov 3, 2016

@jmulloy-simplisafe yes adding multiple stanza will add multiple ips

@rmccubbin

This comment has been minimized.

Copy link

commented Feb 23, 2017

Just to note you can also use an array as follows:

resource "aws_waf_ipset" "foobar_whitelist" {
  name = "tf-whitelist-test"
  ip_set_descriptors = [ {
    type = "IPV4"
    value = "10.10.10.10/32"
  },
  {
    type = "IPV4"
    value = "20.20.20.20/32"
  } ]
}

However the original problem that we can't go from a list of IPs to a list of ip_set_descriptors still seems to remain. For example I have > 1000 ips which I would like to blacklist. I can read those in from a file using "${split("\n", file("ips.txt"))}" but I am struggling to find a way to turn that list into the ip_set_descriptors. I looked at formatList and templating but it seems like those would just create a list of strings e.g. ["{ value=\"10.10.10.10/32\"...}"] rather than an actual object. If ip_set_descriptors were its own resource we could use count operations or if there were some means of creating objects for a list that would also solve the problem.

@allangomesptc

This comment has been minimized.

Copy link

commented May 8, 2017

I'm looking forward to having this feature. I have a long list of IPs to add and I had to write an external python script using boto3 to create that ipset.

@mgladdish

This comment has been minimized.

Copy link

commented May 20, 2017

I ended up using https://github.com/cbroglie/terrastache to get around this deeply annoying limitation

@butla

This comment has been minimized.

Copy link

commented Sep 19, 2017

I resolved that by putting a list of maps in my .tfvars file, like this:

whitelisted_ips = [
    { value = "1.2.3.4/32", type="IPV4"},
    { value = "2.3.4.5/28", type="IPV4"},
]

And assigning that to ip_set_descriptors in the module:

variable "whitelisted_ips" { type = "list" }

resource "aws_waf_ipset" "whitelisted_ips" {
  name = "WhitelistedIps"
  ip_set_descriptors = "${var.whitelisted_ips}"
}

Having to put type="IPV4" on every single line is a bit offensive to me as a programmer, but it works :) And it's doesn't require as much engineering effort as starting to use templates to pre-process Terraform scripts.

@rmccubbin I guess #7034 will be our salvation :)

@allandrick

This comment has been minimized.

Copy link

commented Oct 16, 2018

I use established lists in a repo based module which are used for security groups and other functions elsewhere which are in the format of:

cidr_blocks = [
  "X.X.X.X/Y",
  "X.X.X.X/Y",
]

which I would love to re-use and/or reference in WAF IP Sets. I can get Terraform to render the required format in JSON which looks fine in an output variable, but can't get it to work as a parameter to ip_set_descriptor in the aws_wafregional_ipset resource.

data "template_file" "template" {
  template = <<TXT
{
  "type": "$${type}",
  "value": "$${value}"
}
TXT
  count    = "${length(module.offices.cidr_blocks)}"
  vars {
    type = "IPV4"
    value = "${module.offices.cidr_blocks[count.index]}"
  }
}

The output looks good:

output "json" {
  value = "${data.template_file.template.*.rendered}"
}

renders to:

json = [
    {
  "type": "IPV4",
  "value": "X.X.X.X/Y"
}
,
    {
  "type": "IPV4",
  "value": "X.X.X.X/Y"
}
]

but when I try to use it:

resource "aws_wafregional_ipset" "test" {
  name = "test"

  ip_set_descriptor = ["${data.template_file.data_json.*.rendered}"]
}

I get

Error: aws_wafregional_ipset.test: "ip_set_descriptor.0.type": required field is not set
Error: aws_wafregional_ipset.test: "ip_set_descriptor.0.value": required field is not set

If I don't surround the ip_set_descriptor value with [] I get:

Error: aws_wafregional_ipset.test: ip_set_descriptor: should be a list
@allandrick

This comment has been minimized.

Copy link

commented Oct 16, 2018

I ended up using template_file to create a CloudFormation template to create the WAF Regional IP Sets and then referenced them by ID in my TF hardcoded elsewhere. This was much simpler than duplicating my IP lists.

# in my offices module
output "cidr_blocks" {
  value = [
    "X.X.X.X/Y",
   "X.X.X.X/Y",
  ]
}

# ... in my ipsets-cf.tf - I load the offices module

data "template_file" "ipsets" {
  template = <<JSON
{
  "Type": "$${type}",
  "Value": "$${value}"
}
JSON
  count = "${length(module.offices.cidr_blocks)}"
  vars {
    type  = "IPV4"
    value = "${module.offices.cidr_blocks[count.index]}"
  }
}

data "template_file" "waf_ipset" {
  template = <<JSON
{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "AWS WAF Template to load WAF IP Sets",
  "Resources": {
    "WAFRegionalIPSet": {
      "Type": "AWS::WAFRegional::IPSet",
      "Properties": {
        "Name": "Offices",
        "IPSetDescriptors": [
          $${ipsets}
        ]
      }
    }
  }
}
JSON
  vars {
    ipsets = "${join(",", data.template_file.ipsets.*.rendered)}"
  }
}

resource "aws_cloudformation_stack" "waf-ipset" {
  name = "waf-ipset"
  template_body = "${data.template_file.waf_ipset.rendered}"
}

Hopefully this helps someone and saves them 4 hours of googling / fighting interpolation limitations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.