New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

formatlist iteration count or interpolation range function #2769

Open
jasonkeene opened this Issue Jul 17, 2015 · 10 comments

Comments

Projects
None yet
8 participants
@jasonkeene

jasonkeene commented Jul 17, 2015

I want to be able to do a formatlist that includes a count of the iteration over the list:

formatlist("node-%d=http://%s:2380", COUNT, openstack_compute_floatingip_v2.demo.*.address)

There are two solutions. Have a custom printf format verb for the current iteration:

formatlist("node-%i=http://%s:2380", openstack_compute_floatingip_v2.demo.*.address)

It could easily be checked for here:

fmtargs := make([]interface{}, len(varargs))
for i := 0; i < n; i++ {
for j, arg := range varargs {
switch arg := arg.(type) {
default:
fmtargs[j] = arg
case []string:
fmtargs[j] = arg[i]
}
}
list[i] = fmt.Sprintf(format, fmtargs...)

Or just provide a generic range function that will produce a list of integers:

formatlist("node-%d=http://%s:2380", range(length(openstack_compute_floatingip_v2.demo)), openstack_compute_floatingip_v2.demo.*.address)

I don't mind implementing this since I need it. I'd prefer the former.

@jasonkeene jasonkeene changed the title from Interpolation Range Function or formatlist iteration count to formatlist iteration count or interpolation range function Jul 17, 2015

@jasonkeene

This comment has been minimized.

Show comment
Hide comment
@jasonkeene

jasonkeene Jul 17, 2015

Note, count.index wouldn't work in my case since it's tied to the resource (in this case the instance) and this string needs to be created for each instance.

jasonkeene commented Jul 17, 2015

Note, count.index wouldn't work in my case since it's tied to the resource (in this case the instance) and this string needs to be created for each instance.

@jasonkeene

This comment has been minimized.

Show comment
Hide comment
@jasonkeene

jasonkeene Jul 17, 2015

Another solution I thought of is to provide the count index on the resource like this:

formatlist("node-%d=http://%s:2380", openstack_compute_floatingip_v2.demo.*.index, openstack_compute_floatingip_v2.demo.*.address)

It would be nice to have this on every resource that does count.

jasonkeene commented Jul 17, 2015

Another solution I thought of is to provide the count index on the resource like this:

formatlist("node-%d=http://%s:2380", openstack_compute_floatingip_v2.demo.*.index, openstack_compute_floatingip_v2.demo.*.address)

It would be nice to have this on every resource that does count.

@jasonkeene

This comment has been minimized.

Show comment
Hide comment
@jasonkeene

jasonkeene Jul 17, 2015

@phinze gave me a workaround for this:

resource "template_file" "address-lines" {
    filename = "./address-line"
    count = "${var.size}"
    vars = {
        machine_num = "${count.index+1}"
        ip_address = "${element(openstack_compute_floatingip_v2.demo.*.address, count.index)}"
    }
}

resource "template_file" "user-data" {
    filename = "./user-data"
    count = "${var.size}"
    vars = {
        ...
        initial_cluster = "${join(", ", template_file.address-lines.*.rendered)}"
    }
}

I ran into the issue that my template data in ./address-line had a \n at the end due to vim that I had to remove. Obviously this is just a work around.

jasonkeene commented Jul 17, 2015

@phinze gave me a workaround for this:

resource "template_file" "address-lines" {
    filename = "./address-line"
    count = "${var.size}"
    vars = {
        machine_num = "${count.index+1}"
        ip_address = "${element(openstack_compute_floatingip_v2.demo.*.address, count.index)}"
    }
}

resource "template_file" "user-data" {
    filename = "./user-data"
    count = "${var.size}"
    vars = {
        ...
        initial_cluster = "${join(", ", template_file.address-lines.*.rendered)}"
    }
}

I ran into the issue that my template data in ./address-line had a \n at the end due to vim that I had to remove. Obviously this is just a work around.

@jjn2009

This comment has been minimized.

Show comment
Hide comment
@jjn2009

jjn2009 May 26, 2016

The problem I am having is that my user_data and my virtual machines cannot be dependent on each other, thus I cannot output a config line into my user data which reflects each machine I am going to create. If I had a range function then I could create a list derived from my num_masters variable (for example) which is passed to formatlist, which then constructs a string that contains each DNS name of the nodes I am going to deploy (given some canonicalized DNS scheme). num_masters would also be passed to count in my digitalocean_droplet for virtual machine creation and digitalocean_record for subdomain creation.

jjn2009 commented May 26, 2016

The problem I am having is that my user_data and my virtual machines cannot be dependent on each other, thus I cannot output a config line into my user data which reflects each machine I am going to create. If I had a range function then I could create a list derived from my num_masters variable (for example) which is passed to formatlist, which then constructs a string that contains each DNS name of the nodes I am going to deploy (given some canonicalized DNS scheme). num_masters would also be passed to count in my digitalocean_droplet for virtual machine creation and digitalocean_record for subdomain creation.

@vallard

This comment has been minimized.

Show comment
Hide comment
@vallard

vallard Nov 4, 2016

any updates on this or was another way to do this implemented? Thanks!

vallard commented Nov 4, 2016

any updates on this or was another way to do this implemented? Thanks!

@vallard

This comment has been minimized.

Show comment
Hide comment
@vallard

vallard Nov 4, 2016

For my workaround, using openstack_compute_instance_v2 resources, when I created them I added a meta variable:

resource "openstack_compute_instance_v2" "kube-worker" {
  name = "${var.worker_name}${format(var.count_format, count.index+1) }"
  count = "${var.worker_count}"
  image_name = "${var.kube_image}"
  flavor_name = "${var.kube_flavor}"
  metadata {
    kube_role = "worker"
    worker_number = "${count.index}"
  }
  network {
    name = "${var.network}"
  }
  key_pair = "${var.key_pair}"
  security_groups = ["${var.security_group}"]
}

Then when I needed to merge the lists I used the metadata.worker_number variable in formatlist:

static_routes = "${join("\n", formatlist("up route add -net %s.%s.0/24 gw %s dev %s", var.cluster_nets_prefix, openstack_compute_instance_v2.kube-worker.*.metadata.worker_number,openstack_compute_instance_v2.kube-worker.*.access_ip_v4, var.if_dev))}"

In this case the *.metadata.worker_number provided the index I needed.

vallard commented Nov 4, 2016

For my workaround, using openstack_compute_instance_v2 resources, when I created them I added a meta variable:

resource "openstack_compute_instance_v2" "kube-worker" {
  name = "${var.worker_name}${format(var.count_format, count.index+1) }"
  count = "${var.worker_count}"
  image_name = "${var.kube_image}"
  flavor_name = "${var.kube_flavor}"
  metadata {
    kube_role = "worker"
    worker_number = "${count.index}"
  }
  network {
    name = "${var.network}"
  }
  key_pair = "${var.key_pair}"
  security_groups = ["${var.security_group}"]
}

Then when I needed to merge the lists I used the metadata.worker_number variable in formatlist:

static_routes = "${join("\n", formatlist("up route add -net %s.%s.0/24 gw %s dev %s", var.cluster_nets_prefix, openstack_compute_instance_v2.kube-worker.*.metadata.worker_number,openstack_compute_instance_v2.kube-worker.*.access_ip_v4, var.if_dev))}"

In this case the *.metadata.worker_number provided the index I needed.

@blalor

This comment has been minimized.

Show comment
Hide comment
@blalor

blalor Jan 20, 2017

Contributor

I stumbled upon the template_file solution myself, but it would really be nice to have a first-class range function, or something like formatlist.

Contributor

blalor commented Jan 20, 2017

I stumbled upon the template_file solution myself, but it would really be nice to have a first-class range function, or something like formatlist.

@bloo

This comment has been minimized.

Show comment
Hide comment
@bloo

bloo Jul 12, 2017

@blalor ran into this as well - seems like a common use case

bloo commented Jul 12, 2017

@blalor ran into this as well - seems like a common use case

@r1ckr

This comment has been minimized.

Show comment
Hide comment
@r1ckr

r1ckr Jan 26, 2018

Any progress on the range function?

r1ckr commented Jan 26, 2018

Any progress on the range function?

@apparentlymart apparentlymart added config and removed core labels Feb 19, 2018

@apparentlymart

This comment has been minimized.

Show comment
Hide comment
@apparentlymart

apparentlymart Feb 19, 2018

Contributor

Hi all! Sorry for the long silence here.

Our current project is integrating an improved version of the configuration language parser and interpreter that addresses some limitations and adds some new features. Once we have completed our current work it should be possible to address the given use-case using the new "for expression" construct:

# NOT YET IMPLEMENTED; details may change before final release

formatlist(
  "node-%d=http://%s:2380",
  [for i, v in openstack_compute_floatingip_v2.demo.*.address: i],
  openstack_compute_floatingip_v2.demo.*.address
)

Alternatively, this expression can be turned "inside-out", which seems more readable/understandable:

# NOT YET IMPLEMENTED; details may change before final release

[
  for i, v in openstack_compute_floatingip_v2.demo.*.address:
  format("node-%d=http://%s:2380", i, v)
]

Integrating the improved implementation to get these new features is the current focus for the Terraform team at HashiCorp, and will be included in a forthcoming major release.

It is likely that a range function will still be added for other reasons, but I think it will be preferable to use the "for expression" construct for this sort of use-case where something needs to be evaluated for each index/value pair.

Contributor

apparentlymart commented Feb 19, 2018

Hi all! Sorry for the long silence here.

Our current project is integrating an improved version of the configuration language parser and interpreter that addresses some limitations and adds some new features. Once we have completed our current work it should be possible to address the given use-case using the new "for expression" construct:

# NOT YET IMPLEMENTED; details may change before final release

formatlist(
  "node-%d=http://%s:2380",
  [for i, v in openstack_compute_floatingip_v2.demo.*.address: i],
  openstack_compute_floatingip_v2.demo.*.address
)

Alternatively, this expression can be turned "inside-out", which seems more readable/understandable:

# NOT YET IMPLEMENTED; details may change before final release

[
  for i, v in openstack_compute_floatingip_v2.demo.*.address:
  format("node-%d=http://%s:2380", i, v)
]

Integrating the improved implementation to get these new features is the current focus for the Terraform team at HashiCorp, and will be included in a forthcoming major release.

It is likely that a range function will still be added for other reasons, but I think it will be preferable to use the "for expression" construct for this sort of use-case where something needs to be evaluated for each index/value pair.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment