New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

depends_on cannot be used in a module #10462

Open
chrisrlong opened this Issue Dec 1, 2016 · 75 comments

Comments

Projects
None yet
@chrisrlong

chrisrlong commented Dec 1, 2016

Hi there,

Terraform Version

0.8.0 rc1+

Affected Resource(s)

module

Terraform Configuration Files

module "legacy_site" {
  source = "../../../../../modules/site"
  name = "foo-site"
  health_check_target = "TCP:443"
  azs = "${var.azs}"
  instance_count = "${var.instance_count}"
  vpc = "apps"
  region = "${var.region}"
  environment = "${var.environment}"
  run_list = "hs_site_foo"

  #rds_complete = "${module.rds.db_instance_id}"
  #elasticache_cache_complete = "${module.elasticache_cache.elasticache_id}"
  #elasticache_sessions_complete = "${module.elasticache_sessions.elasticache_id}"

  depends_on = [
  "module.rds",
  "module.elasticache_sessions"
  ]

}

Debug Output

Error loading Terraform: module root: module legacy_site: depends_on is not a valid parameter
module root: module legacy_site: depends_on is not a valid parameter

Expected Behavior

I am trying to use the new depends_on instead of the above outputs, so I create and provision my app once I know database and caches are built.

Actual Behavior

Nothing as terraform errors out as above.

Steps to Reproduce

  1. terraform apply

References

depends_on can reference modules. This allows a resource or output to depend on everything within a module. (#10076)

@stack72

This comment has been minimized.

Show comment
Hide comment
@stack72

stack72 Dec 1, 2016

Contributor

Hi @chrisrlong

This is very strange - we introduced depends_on for modules in 0.8.0-beta2

can you run terraform version and post the output here for me?

Paul

Contributor

stack72 commented Dec 1, 2016

Hi @chrisrlong

This is very strange - we introduced depends_on for modules in 0.8.0-beta2

can you run terraform version and post the output here for me?

Paul

@stack72 stack72 added bug core labels Dec 1, 2016

@chrisrlong

This comment has been minimized.

Show comment
Hide comment
@chrisrlong

chrisrlong Dec 1, 2016

Hi @stack72,

I have complied master from version b15b8bd (Terraform v0.8.0-dev (b15b8bd+CHANGES)), which should include beta2 features. (I did this to get 10337 and 10338 fixes)

It seems it would work if you set a resource to depend on a module, but you cannot set a module to depend_on another module......

Thanks for the great tool btw ;)

Chris

chrisrlong commented Dec 1, 2016

Hi @stack72,

I have complied master from version b15b8bd (Terraform v0.8.0-dev (b15b8bd+CHANGES)), which should include beta2 features. (I did this to get 10337 and 10338 fixes)

It seems it would work if you set a resource to depend on a module, but you cannot set a module to depend_on another module......

Thanks for the great tool btw ;)

Chris

@stack72

This comment has been minimized.

Show comment
Hide comment
@stack72

stack72 Dec 1, 2016

Contributor

Hi @chrisrlong

thanks for getting back to me - you are indeed correct - modules cannot (currently) depend_on other modules but can depend on a resource only

Will change the tag on this from bug to enhancement :)

Keep an eye out for it soon ™️

Thanks

Paul

Contributor

stack72 commented Dec 1, 2016

Hi @chrisrlong

thanks for getting back to me - you are indeed correct - modules cannot (currently) depend_on other modules but can depend on a resource only

Will change the tag on this from bug to enhancement :)

Keep an eye out for it soon ™️

Thanks

Paul

@mitchellh

This comment has been minimized.

Show comment
Hide comment
@mitchellh

mitchellh Dec 1, 2016

Member

Retagging as enhancement, since this isn't broken functionality, its functionality that doesn't exist yet. :)

Member

mitchellh commented Dec 1, 2016

Retagging as enhancement, since this isn't broken functionality, its functionality that doesn't exist yet. :)

@mitchellh mitchellh added enhancement and removed bug labels Dec 1, 2016

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Jan 3, 2017

Contributor

My use case: the slave virtual machine depends on master virtual machine to exist

Contributor

sheerun commented Jan 3, 2017

My use case: the slave virtual machine depends on master virtual machine to exist

@shantanugadgil

This comment has been minimized.

Show comment
Hide comment
@shantanugadgil

shantanugadgil Jan 7, 2017

I am seeing the same error:
module root: module example: depends_on is not a valid parameter

My Terraform version is:
Terraform v0.8.2

My use case is that I want a module to depend on a resource.

shantanugadgil commented Jan 7, 2017

I am seeing the same error:
module root: module example: depends_on is not a valid parameter

My Terraform version is:
Terraform v0.8.2

My use case is that I want a module to depend on a resource.

@arehmandev

This comment has been minimized.

Show comment
Hide comment
@arehmandev

arehmandev Jan 9, 2017

Contributor

Error loading Terraform: module root: module etcdbastion: depends_on is not a valid parameter.
^ Seeing it here too

Contributor

arehmandev commented Jan 9, 2017

Error loading Terraform: module root: module etcdbastion: depends_on is not a valid parameter.
^ Seeing it here too

@cemo

This comment has been minimized.

Show comment
Hide comment
@cemo

cemo Jan 10, 2017

@mitchellh I believe this one can solve unnested modules problem too. There were some issues regarding this #10883 before. This solution would be intuitive and address these issues.

cemo commented Jan 10, 2017

@mitchellh I believe this one can solve unnested modules problem too. There were some issues regarding this #10883 before. This solution would be intuitive and address these issues.

@fieryvova

This comment has been minimized.

Show comment
Hide comment
@fieryvova

fieryvova Feb 14, 2017

Waiting for this feature to be added 👍

fieryvova commented Feb 14, 2017

Waiting for this feature to be added 👍

@pseudobeer

This comment has been minimized.

Show comment
Hide comment
@pseudobeer

pseudobeer commented Feb 23, 2017

+1

@avatar4d

This comment has been minimized.

Show comment
Hide comment
@avatar4d

avatar4d commented Feb 23, 2017

+1

@panmanphil

This comment has been minimized.

Show comment
Hide comment
@panmanphil

panmanphil Feb 23, 2017

+1 This would really help modularizing our code

panmanphil commented Feb 23, 2017

+1 This would really help modularizing our code

@shoelessrob

This comment has been minimized.

Show comment
Hide comment
@shoelessrob

shoelessrob commented Feb 23, 2017

+1

@mschenck

This comment has been minimized.

Show comment
Hide comment
@mschenck

mschenck commented Feb 23, 2017

+1

@cpanayides

This comment has been minimized.

Show comment
Hide comment
@cpanayides

cpanayides commented Feb 24, 2017

+1

@derBroBro

This comment has been minimized.

Show comment
Hide comment
@derBroBro

derBroBro Feb 25, 2017

Contributor

+1

Contributor

derBroBro commented Feb 25, 2017

+1

@waffleshop

This comment has been minimized.

Show comment
Hide comment
@waffleshop

waffleshop commented Feb 28, 2017

+1

@laiph

This comment has been minimized.

Show comment
Hide comment
@laiph

laiph commented Mar 2, 2017

+1

@mqasim1983

This comment has been minimized.

Show comment
Hide comment
@mqasim1983

mqasim1983 commented Mar 7, 2017

+1

@seyeda

This comment has been minimized.

Show comment
Hide comment
@seyeda

seyeda commented Mar 7, 2017

+1

@AirbornePorcine

This comment has been minimized.

Show comment
Hide comment
@AirbornePorcine

AirbornePorcine Mar 10, 2017

Anyone have a good workaround for this in the meantime?

AirbornePorcine commented Mar 10, 2017

Anyone have a good workaround for this in the meantime?

@jacobwgillespie

This comment has been minimized.

Show comment
Hide comment
@jacobwgillespie

jacobwgillespie Mar 10, 2017

My pseudo workaround is to add a list variable inside the module:

variable "depends_on" { default = [], type = "list" }

Then when using the module, pass it a computed value from the resource I want it to depend on:

module "something" {
  depends_on = ["${aws_instance.instance.private_ip}"]
}

jacobwgillespie commented Mar 10, 2017

My pseudo workaround is to add a list variable inside the module:

variable "depends_on" { default = [], type = "list" }

Then when using the module, pass it a computed value from the resource I want it to depend on:

module "something" {
  depends_on = ["${aws_instance.instance.private_ip}"]
}
@AirbornePorcine

This comment has been minimized.

Show comment
Hide comment
@AirbornePorcine

AirbornePorcine Mar 10, 2017

Thanks, that doesn't entirely fit what I think this issue is talking about though - making a module able to depend on another one. I don't have any resources that the module should depend on - it should depend on another module being created.

I've tried doing stuff like this:
module 1:

output "wait_for_cluster" { value = "ref to a resource that gets created last by this module" }

module 2:

variable "wait_for_cluster" {}

Main terraform template:

module "1" {

}

module "2" {
 wait_for_cluster = "${module.1.wait_for_cluster}"
}

But this doesn't do anything - my module two is still created at basically the same time as module 1.

AirbornePorcine commented Mar 10, 2017

Thanks, that doesn't entirely fit what I think this issue is talking about though - making a module able to depend on another one. I don't have any resources that the module should depend on - it should depend on another module being created.

I've tried doing stuff like this:
module 1:

output "wait_for_cluster" { value = "ref to a resource that gets created last by this module" }

module 2:

variable "wait_for_cluster" {}

Main terraform template:

module "1" {

}

module "2" {
 wait_for_cluster = "${module.1.wait_for_cluster}"
}

But this doesn't do anything - my module two is still created at basically the same time as module 1.

@b-dean

This comment has been minimized.

Show comment
Hide comment
@b-dean

b-dean Mar 10, 2017

@AirbornePorcine In your second module you need to actually use the wait_for_cluster variable somewhere that establishes a dependency. Such as in a template or a trigger on a null_resource, etc.

See #1178 (comment) from @kristjanelias

The only change I'd make to his workaround is to maybe make the dummy dependency resource use a trigger so it changes when the instance changes.

resource "null_resource" "dummy_dependency" {
  triggers {
    dependency_id = "${aws_instance.instance.id}"
  }
}

b-dean commented Mar 10, 2017

@AirbornePorcine In your second module you need to actually use the wait_for_cluster variable somewhere that establishes a dependency. Such as in a template or a trigger on a null_resource, etc.

See #1178 (comment) from @kristjanelias

The only change I'd make to his workaround is to maybe make the dummy dependency resource use a trigger so it changes when the instance changes.

resource "null_resource" "dummy_dependency" {
  triggers {
    dependency_id = "${aws_instance.instance.id}"
  }
}
@AirbornePorcine

This comment has been minimized.

Show comment
Hide comment
@AirbornePorcine

AirbornePorcine Mar 10, 2017

Ah, that makes sense! Should have known that Terraform would optimize away my variable if I didn't use it. :) I tried out your suggestion with the trigger and that didn't do it, but setting the depends_id on a tag on some resource in module 2 seems to have done it. Hacky but oh well I guess.

AirbornePorcine commented Mar 10, 2017

Ah, that makes sense! Should have known that Terraform would optimize away my variable if I didn't use it. :) I tried out your suggestion with the trigger and that didn't do it, but setting the depends_id on a tag on some resource in module 2 seems to have done it. Hacky but oh well I guess.

@dmikalova

This comment has been minimized.

Show comment
Hide comment
@dmikalova

dmikalova Sep 6, 2017

@apparentlymart Do you expect that the aws_ecs_service depends_on race condition will be satisfied with cross module variables?

ie moduleA outputs aws_iam_role_policy.moduleA.id and moduleB is declared using that output - is that enough for a aws_ecs_service in moduleB to not have a race condition, or is a null_resource necessary?

dmikalova commented Sep 6, 2017

@apparentlymart Do you expect that the aws_ecs_service depends_on race condition will be satisfied with cross module variables?

ie moduleA outputs aws_iam_role_policy.moduleA.id and moduleB is declared using that output - is that enough for a aws_ecs_service in moduleB to not have a race condition, or is a null_resource necessary?

@ozbillwang

This comment has been minimized.

Show comment
Hide comment
@ozbillwang

ozbillwang Sep 11, 2017

In my case, I need set depends_on to a resource in a module, any work around for me?

module "app" {
   source = "./module-xxxx"

   depends_on = ["aws_ecs_task_definition.app"]

   ecs_task_definition_family = "${aws_ecs_task_definition.app.family}"
   ecs_task_definition_arn    = "${aws_ecs_task_definition.app.arn}"

}

I got the same error:

* module root: module app: depends_on is not a valid parameter

And if I don't add depends_on

resource "aws_ecs_task_definition" "app" {
...
}

module "app" {
   source = "./module-xxxx"

   ecs_task_definition_family = "${aws_ecs_task_definition.app.family}"
   ecs_task_definition_arn    = "${aws_ecs_task_definition.app.arn}"
}  

Always get error that resource is not found

* module.app.var.ecs_task_definition_family: Resource 'aws_ecs_task_definition.app' not found for variable 'ecs_task_definition.app.family'
* module.app.var.ecs_task_definition_arn: Resource 'aws_ecs_task_definition.app' not found for variable 'ecs_task_definition.app.arn'

ozbillwang commented Sep 11, 2017

In my case, I need set depends_on to a resource in a module, any work around for me?

module "app" {
   source = "./module-xxxx"

   depends_on = ["aws_ecs_task_definition.app"]

   ecs_task_definition_family = "${aws_ecs_task_definition.app.family}"
   ecs_task_definition_arn    = "${aws_ecs_task_definition.app.arn}"

}

I got the same error:

* module root: module app: depends_on is not a valid parameter

And if I don't add depends_on

resource "aws_ecs_task_definition" "app" {
...
}

module "app" {
   source = "./module-xxxx"

   ecs_task_definition_family = "${aws_ecs_task_definition.app.family}"
   ecs_task_definition_arn    = "${aws_ecs_task_definition.app.arn}"
}  

Always get error that resource is not found

* module.app.var.ecs_task_definition_family: Resource 'aws_ecs_task_definition.app' not found for variable 'ecs_task_definition.app.family'
* module.app.var.ecs_task_definition_arn: Resource 'aws_ecs_task_definition.app' not found for variable 'ecs_task_definition.app.arn'
@thomasbiddle

This comment has been minimized.

Show comment
Hide comment
@thomasbiddle

thomasbiddle Sep 11, 2017

@ozbillwang You shouldn't need any depends_on setup here. Looks like you're referencing your variable incorrectly inside your module.

thomasbiddle commented Sep 11, 2017

@ozbillwang You shouldn't need any depends_on setup here. Looks like you're referencing your variable incorrectly inside your module.

@ozbillwang

This comment has been minimized.

Show comment
Hide comment
@ozbillwang

ozbillwang Sep 12, 2017

@thomasbiddle

Thanks for helping, but my codes are fine. Currently I fixed it with below work around.

data "template_file" "family" {
  template = "${aws_ecs_task_definition.app.family}"

  depends_on = ["aws_ecs_task_definition.app"]
}

data "template_file" "arn" {
  template = "${aws_ecs_task_definition.app.arn}"

  depends_on = ["aws_ecs_task_definition.app"]
}

module "app" {
   source = "./module-xxxx"

   aws_ecs_task_definition_family    = "${data.template_file.family.rendered}"
   aws_ecs_task_definition_arn       = "${data.template_file.arn.rendered}"
}

It works, but every time, when run terraform plan, always report there is one resource module.app.aws_ecs_service.app to be changed.

But when you apply it, no change at all.

ozbillwang commented Sep 12, 2017

@thomasbiddle

Thanks for helping, but my codes are fine. Currently I fixed it with below work around.

data "template_file" "family" {
  template = "${aws_ecs_task_definition.app.family}"

  depends_on = ["aws_ecs_task_definition.app"]
}

data "template_file" "arn" {
  template = "${aws_ecs_task_definition.app.arn}"

  depends_on = ["aws_ecs_task_definition.app"]
}

module "app" {
   source = "./module-xxxx"

   aws_ecs_task_definition_family    = "${data.template_file.family.rendered}"
   aws_ecs_task_definition_arn       = "${data.template_file.arn.rendered}"
}

It works, but every time, when run terraform plan, always report there is one resource module.app.aws_ecs_service.app to be changed.

But when you apply it, no change at all.

@FernandoMiguel

This comment has been minimized.

Show comment
Hide comment
@FernandoMiguel

FernandoMiguel Oct 26, 2017

@apparentlymart in my case, I have a module that creates security groups and then calls it again to create the rules for each SG, passing the corresponding SG_id.
Some SG rules reference other SG_id for inbound targets.

the code that creates the rules depends on the SG already existing.

FernandoMiguel commented Oct 26, 2017

@apparentlymart in my case, I have a module that creates security groups and then calls it again to create the rules for each SG, passing the corresponding SG_id.
Some SG rules reference other SG_id for inbound targets.

the code that creates the rules depends on the SG already existing.

@rafaelmagu

This comment has been minimized.

Show comment
Hide comment
@rafaelmagu

rafaelmagu Oct 31, 2017

I'm seeing a very similar issue, but am trying to reference a module output in a resource in the root:

# creates one mount target per subnet
resource "aws_efs_mount_target" "web" {
  count           = "${length(module.web-vpc.internal_subnets)}"
  file_system_id  = "${aws_efs_file_system.web.id}"
  subnet_id       = "${element(module.web-vpc.internal_subnets, count.index)}"
  security_groups = ["${aws_security_group.efs.id}"]
}

module.web-vpc.internal_subnets is a list.

Attempting to plan this will render the following error:

Error running plan: 1 error(s) occurred:

* aws_efs_mount_target.web: aws_efs_mount_target.web: value of 'count' cannot be computed

rafaelmagu commented Oct 31, 2017

I'm seeing a very similar issue, but am trying to reference a module output in a resource in the root:

# creates one mount target per subnet
resource "aws_efs_mount_target" "web" {
  count           = "${length(module.web-vpc.internal_subnets)}"
  file_system_id  = "${aws_efs_file_system.web.id}"
  subnet_id       = "${element(module.web-vpc.internal_subnets, count.index)}"
  security_groups = ["${aws_security_group.efs.id}"]
}

module.web-vpc.internal_subnets is a list.

Attempting to plan this will render the following error:

Error running plan: 1 error(s) occurred:

* aws_efs_mount_target.web: aws_efs_mount_target.web: value of 'count' cannot be computed
@rafaelmagu

This comment has been minimized.

Show comment
Hide comment
@rafaelmagu

rafaelmagu Oct 31, 2017

I was able to work around that by changing count to 3, plan/apply-ing, then reverting the count to the dynamic value.

rafaelmagu commented Oct 31, 2017

I was able to work around that by changing count to 3, plan/apply-ing, then reverting the count to the dynamic value.

@GeoffMillerAZ

This comment has been minimized.

Show comment
Hide comment
@GeoffMillerAZ

GeoffMillerAZ Nov 7, 2017

Another part to this:

Consider each of these modules is for user management of accounts. the parent module creates groups and roles and allows users in this account to assume roles in other accounts. It takes a list of the children account IDs to setup allowing this. The child modules create roles with no groups. This is because I want to sign into the parent account and assume roles into other accounts controlled by what group you are placed in the parent account. When you run terraform plan you get:

Error output:

Error refreshing state: 1 error(s) occurred:

* module.child_b.provider.aws.${var.iam_alias}-${local.region}: The role "arn:aws:iam::999fakeaccount999:role/myrole" cannot be assumed.

There is a provider in the child module that assumes the myrole in the child account. But it can't until the module.parent allows it to. It works fine if I comment out the child module then run it. This is why the error only references child_b and not child_a; I did the commenting out workaround to get it to work for child_a. But due to #9653 I can't run a destroy on these either without manual intervention. So, I'm trying to make sure that permissions checks are taken into consideration with the addition of depends_on to modules. It would also be nice if you could pass a map into a provider key of a module that way I could define my providers outside the module and have them specified by map keys inside the module but that's a side note.

Snippet of my code

// local.config is a non-flat map that holds flags and account numbers 
// and default regions and stuff

module "parent" { // creates groups that allow assume role into other accounts
  source    = "./parent"
  config    = "${local.config}"
  children  = "${values(local.children_map)}"
  iam_alias = "repay_testing_main"
  role      = "myrole"
}

module "child_a" {
  source           = "./child"
  config           = "${local.config}"
  parent_iam_alias = "${module.parent.iam_alias}"
  iam_alias        = "two"
  role             = "myrole"
}

module "child_b" {
  source           = "./child"
  config           = "${local.config}"
  parent_iam_alias = "${module.parent.iam_alias}"
  iam_alias        = "three"
  role             = "myrole"
}

GeoffMillerAZ commented Nov 7, 2017

Another part to this:

Consider each of these modules is for user management of accounts. the parent module creates groups and roles and allows users in this account to assume roles in other accounts. It takes a list of the children account IDs to setup allowing this. The child modules create roles with no groups. This is because I want to sign into the parent account and assume roles into other accounts controlled by what group you are placed in the parent account. When you run terraform plan you get:

Error output:

Error refreshing state: 1 error(s) occurred:

* module.child_b.provider.aws.${var.iam_alias}-${local.region}: The role "arn:aws:iam::999fakeaccount999:role/myrole" cannot be assumed.

There is a provider in the child module that assumes the myrole in the child account. But it can't until the module.parent allows it to. It works fine if I comment out the child module then run it. This is why the error only references child_b and not child_a; I did the commenting out workaround to get it to work for child_a. But due to #9653 I can't run a destroy on these either without manual intervention. So, I'm trying to make sure that permissions checks are taken into consideration with the addition of depends_on to modules. It would also be nice if you could pass a map into a provider key of a module that way I could define my providers outside the module and have them specified by map keys inside the module but that's a side note.

Snippet of my code

// local.config is a non-flat map that holds flags and account numbers 
// and default regions and stuff

module "parent" { // creates groups that allow assume role into other accounts
  source    = "./parent"
  config    = "${local.config}"
  children  = "${values(local.children_map)}"
  iam_alias = "repay_testing_main"
  role      = "myrole"
}

module "child_a" {
  source           = "./child"
  config           = "${local.config}"
  parent_iam_alias = "${module.parent.iam_alias}"
  iam_alias        = "two"
  role             = "myrole"
}

module "child_b" {
  source           = "./child"
  config           = "${local.config}"
  parent_iam_alias = "${module.parent.iam_alias}"
  iam_alias        = "three"
  role             = "myrole"
}
@bluemalkin

This comment has been minimized.

Show comment
Hide comment
@bluemalkin

bluemalkin Nov 14, 2017

Any update on implementing this feature please ?

bluemalkin commented Nov 14, 2017

Any update on implementing this feature please ?

@melchit20

This comment has been minimized.

Show comment
Hide comment
@melchit20

melchit20 Dec 5, 2017

I need this plz

melchit20 commented Dec 5, 2017

I need this plz

@gaui

This comment has been minimized.

Show comment
Hide comment
@gaui

gaui Dec 16, 2017

Can't resources have depends_on on other resources in different modules? I understand that modules can't depend on other modules, but resources should be able to depend on other resources, despite of modules.

I have two modules under my application module, docker-swarm and docker-app. Before running app I need to provision and initialize the Docker Swarm.

So in my docker-app module I have a resource aws_instance.docker-app and he has depends_on = ["module.applications.docker-swarm.aws_instance.swarm-manager"] but I always get:

Error: module root:
  module applications.root:
  module docker-app.applications: 1 error(s) occurred:

* aws_instance.docker-app: resource depends on non-existent module 'applications.docker-swarm'

gaui commented Dec 16, 2017

Can't resources have depends_on on other resources in different modules? I understand that modules can't depend on other modules, but resources should be able to depend on other resources, despite of modules.

I have two modules under my application module, docker-swarm and docker-app. Before running app I need to provision and initialize the Docker Swarm.

So in my docker-app module I have a resource aws_instance.docker-app and he has depends_on = ["module.applications.docker-swarm.aws_instance.swarm-manager"] but I always get:

Error: module root:
  module applications.root:
  module docker-app.applications: 1 error(s) occurred:

* aws_instance.docker-app: resource depends on non-existent module 'applications.docker-swarm'
@pixelicous

This comment has been minimized.

Show comment
Hide comment
@pixelicous

pixelicous Dec 25, 2017

#16983 Is this the same error?? This is causing us issues.. do not understand why i cannot just depend on resources from other modules as well, why do we need the output, let me tag resources as public or private. It doesn't work using output anyways..

pixelicous commented Dec 25, 2017

#16983 Is this the same error?? This is causing us issues.. do not understand why i cannot just depend on resources from other modules as well, why do we need the output, let me tag resources as public or private. It doesn't work using output anyways..

@ThomasObenaus

This comment has been minimized.

Show comment
Hide comment
@ThomasObenaus

ThomasObenaus Jan 27, 2018

my use-case

I want to use the output of one module (a list) in order to generate a number of resources. The number of resources to be created should match the size of the modules list.

code snippet

module bar

This module provides just a list of sg_ids. One is of a generated resource and one is hardcoded.

output "sg_ids" {
  value = [
    "${aws_security_group.sg_bar.id}",
    "sg-b91c79d3",
  ]
}

data "aws_vpc" "default" {
  default = true
}

resource "aws_security_group" "sg_bar" {
  vpc_id      = "${data.aws_vpc.default.id}"
  name        = "sg_bar"
  description = "bar sg"
}

using module bar

data "aws_vpc" "default" {
  default = true
}

provider "aws" {
  region  = "eu-central-1"
  profile = "playground"
}

# module that exports a list of security-groups
module "bar" {
  source = "../bar"
}

resource "aws_security_group" "sg_foo" {
  vpc_id      = "${data.aws_vpc.default.id}"
  name        = "sg_foo"
  description = "foo sg"
}

# add one sg_rule to for each of the security-groups of module bar
resource "aws_security_group_rule" "sg_foo_rule" {
  count                    = "${length(module.bar.sg_ids)}"
  type                     = "ingress"
  from_port                = "8080"
  to_port                  = "8080"
  protocol                 = "tcp"
  source_security_group_id = "${element(module.bar.sg_ids,count.index)}"
  security_group_id        = "${aws_security_group.sg_foo.id}"
}

Error output

------------------------------------------------------------------------

Error: Error running plan: 1 error(s) occurred:

* aws_security_group_rule.sg_foo_rule: aws_security_group_rule.sg_foo_rule: value of 'count' cannot be computed

For me it seems that count is evaluated prior to the setup of the modules resources. Since the error does not occur if I hard-code both sg-ids in the output variable of module bar.
i.e.:

output "sg_ids" {
  value = [
    "sg-b91c79d5"",
    "sg-b91c79d3",
  ]
}

ThomasObenaus commented Jan 27, 2018

my use-case

I want to use the output of one module (a list) in order to generate a number of resources. The number of resources to be created should match the size of the modules list.

code snippet

module bar

This module provides just a list of sg_ids. One is of a generated resource and one is hardcoded.

output "sg_ids" {
  value = [
    "${aws_security_group.sg_bar.id}",
    "sg-b91c79d3",
  ]
}

data "aws_vpc" "default" {
  default = true
}

resource "aws_security_group" "sg_bar" {
  vpc_id      = "${data.aws_vpc.default.id}"
  name        = "sg_bar"
  description = "bar sg"
}

using module bar

data "aws_vpc" "default" {
  default = true
}

provider "aws" {
  region  = "eu-central-1"
  profile = "playground"
}

# module that exports a list of security-groups
module "bar" {
  source = "../bar"
}

resource "aws_security_group" "sg_foo" {
  vpc_id      = "${data.aws_vpc.default.id}"
  name        = "sg_foo"
  description = "foo sg"
}

# add one sg_rule to for each of the security-groups of module bar
resource "aws_security_group_rule" "sg_foo_rule" {
  count                    = "${length(module.bar.sg_ids)}"
  type                     = "ingress"
  from_port                = "8080"
  to_port                  = "8080"
  protocol                 = "tcp"
  source_security_group_id = "${element(module.bar.sg_ids,count.index)}"
  security_group_id        = "${aws_security_group.sg_foo.id}"
}

Error output

------------------------------------------------------------------------

Error: Error running plan: 1 error(s) occurred:

* aws_security_group_rule.sg_foo_rule: aws_security_group_rule.sg_foo_rule: value of 'count' cannot be computed

For me it seems that count is evaluated prior to the setup of the modules resources. Since the error does not occur if I hard-code both sg-ids in the output variable of module bar.
i.e.:

output "sg_ids" {
  value = [
    "sg-b91c79d5"",
    "sg-b91c79d3",
  ]
}
@sgendler-stem

This comment has been minimized.

Show comment
Hide comment
@sgendler-stem

sgendler-stem Feb 7, 2018

Another use-case

I have a module which creates an ECS service, sets up autoscaling, cloudwatch alarms, etc. It takes the task definition as an input, which tends to be where the differences between similar services lie. Different services need different access to other AWS resources, however, which means I need to create a task role and assign policies to it. In most contexts, I would have the service module define the role and return the arn and name as outputs, to have policies attached after the module returns. However, because it may sometimes be necessary to have certain policies in place BEFORE the service starts up (to access a KMS key to decrypt secrets in the config file, for example), and the service module waits for the service to start. For this reason, I need to create the role, attach various policies, and then run the service module. However, creating the role is a separate act from attaching a policy to it, so the service module ends up dependent on the role because the role arn is passed in, but it starts executing before the policy attachment resource has completed. It would be easy enough to make the service module depend on the policy attachment resource, except that isn't possible. So I have to workaround it by passing an output of the attachment resource to the module. I'm sure the race between policy attachment and actually needing it will almost always work in my favour, but it will inevitably not be guaranteed and it WILL happen, probably the first time I'm not around to babysit a service launch in-person.

# Have to define the task role and pass it to the stem-job module because the
# job may not be able to start up if permissions aren't granted first, so we
# cannot attach permissions to the role after the service is started in the
# module
data "aws_iam_policy_document" "ecs_task_role" {
  statement {
    effect = "Allow"
    actions = ["sts:AssumeRole"]
    
    principals {
      type = "Service"
      identifiers = ["ecs-tasks.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "ecs_task_role" {
  name = "${var.service_name}-${var.environment}-task-role"
  assume_role_policy = "${data.aws_iam_policy_document.ecs_task_role.json}"
}

# attach a policy to grant access to some resource
resource "aws_iam_role_policy_attachment" "test-attach" {
    role       = "${aws_iam_role.ecs_task_role.name}"
    policy_arn = "${aws_iam_policy.policy.arn}"
}

module "job" {
  source = "../../modules/stem-job"

  service_name = "${var.service_name}"
  environment = "${var.environment}"
  ecs_cluster_name = "${data.terraform_remote_state.ecs.ecs_cluster_name}"
  ecs_cluster_arn = "${data.terraform_remote_state.ecs.ecs_cluster_arn}"
  container_definition = "${data.template_file.container_definition.rendered}"

  # IMPORTANT: see comment on depends_on field
  ecs_task_role_arn = "${aws_iam_role.ecs_task_role.arn}"

  # If other policies are added to the task role in separate aws_iam_role_policy
  # resources, add them as dependencies here, so that the module doesn't get
  # instantiated until after the role is populated
  depends_on = ["aws_iam_role_policy_attachment.test-attach"]

  desired_number_of_tasks = "${var.desired_number_of_tasks}"
  min_number_of_tasks = "${var.min_number_of_tasks}"
  max_number_of_tasks = "${var.max_number_of_tasks}"
}

sgendler-stem commented Feb 7, 2018

Another use-case

I have a module which creates an ECS service, sets up autoscaling, cloudwatch alarms, etc. It takes the task definition as an input, which tends to be where the differences between similar services lie. Different services need different access to other AWS resources, however, which means I need to create a task role and assign policies to it. In most contexts, I would have the service module define the role and return the arn and name as outputs, to have policies attached after the module returns. However, because it may sometimes be necessary to have certain policies in place BEFORE the service starts up (to access a KMS key to decrypt secrets in the config file, for example), and the service module waits for the service to start. For this reason, I need to create the role, attach various policies, and then run the service module. However, creating the role is a separate act from attaching a policy to it, so the service module ends up dependent on the role because the role arn is passed in, but it starts executing before the policy attachment resource has completed. It would be easy enough to make the service module depend on the policy attachment resource, except that isn't possible. So I have to workaround it by passing an output of the attachment resource to the module. I'm sure the race between policy attachment and actually needing it will almost always work in my favour, but it will inevitably not be guaranteed and it WILL happen, probably the first time I'm not around to babysit a service launch in-person.

# Have to define the task role and pass it to the stem-job module because the
# job may not be able to start up if permissions aren't granted first, so we
# cannot attach permissions to the role after the service is started in the
# module
data "aws_iam_policy_document" "ecs_task_role" {
  statement {
    effect = "Allow"
    actions = ["sts:AssumeRole"]
    
    principals {
      type = "Service"
      identifiers = ["ecs-tasks.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "ecs_task_role" {
  name = "${var.service_name}-${var.environment}-task-role"
  assume_role_policy = "${data.aws_iam_policy_document.ecs_task_role.json}"
}

# attach a policy to grant access to some resource
resource "aws_iam_role_policy_attachment" "test-attach" {
    role       = "${aws_iam_role.ecs_task_role.name}"
    policy_arn = "${aws_iam_policy.policy.arn}"
}

module "job" {
  source = "../../modules/stem-job"

  service_name = "${var.service_name}"
  environment = "${var.environment}"
  ecs_cluster_name = "${data.terraform_remote_state.ecs.ecs_cluster_name}"
  ecs_cluster_arn = "${data.terraform_remote_state.ecs.ecs_cluster_arn}"
  container_definition = "${data.template_file.container_definition.rendered}"

  # IMPORTANT: see comment on depends_on field
  ecs_task_role_arn = "${aws_iam_role.ecs_task_role.arn}"

  # If other policies are added to the task role in separate aws_iam_role_policy
  # resources, add them as dependencies here, so that the module doesn't get
  # instantiated until after the role is populated
  depends_on = ["aws_iam_role_policy_attachment.test-attach"]

  desired_number_of_tasks = "${var.desired_number_of_tasks}"
  min_number_of_tasks = "${var.min_number_of_tasks}"
  max_number_of_tasks = "${var.max_number_of_tasks}"
}
@rvaidya

This comment has been minimized.

Show comment
Hide comment
@rvaidya

rvaidya Feb 7, 2018

I am using modules simply to reduce the amount of boilerplate I need to write to set up an instance template, a managed instance group, etc. for my machine clusters. I want to be able to pass in additional resources like load balancer pools, disk images, etc. but I can't set my module to depend on these passed in resources.

rvaidya commented Feb 7, 2018

I am using modules simply to reduce the amount of boilerplate I need to write to set up an instance template, a managed instance group, etc. for my machine clusters. I want to be able to pass in additional resources like load balancer pools, disk images, etc. but I can't set my module to depend on these passed in resources.

@ozbillwang

This comment has been minimized.

Show comment
Hide comment
@ozbillwang

ozbillwang Feb 8, 2018

@sgendler-stem

In your sample, you can move the codes about iam role, policy to that module stem-job as well.

I did similar without depend_on issue.

ozbillwang commented Feb 8, 2018

@sgendler-stem

In your sample, you can move the codes about iam role, policy to that module stem-job as well.

I did similar without depend_on issue.

@pixelicous

This comment has been minimized.

Show comment
Hide comment
@pixelicous

pixelicous Feb 19, 2018

So....... this is open for a very long time?
I cannot understand why one module cannot depend on another.

I have a module that creates webapps, it needs to receive some items from a different module.. how do i go about doing that??

So basic..

pixelicous commented Feb 19, 2018

So....... this is open for a very long time?
I cannot understand why one module cannot depend on another.

I have a module that creates webapps, it needs to receive some items from a different module.. how do i go about doing that??

So basic..

@agnelantony2

This comment has been minimized.

Show comment
Hide comment
@agnelantony2

agnelantony2 Mar 4, 2018

is this still an open issue ? any feature is being worked on for this ask ?

agnelantony2 commented Mar 4, 2018

is this still an open issue ? any feature is being worked on for this ask ?

@pb0101

This comment has been minimized.

Show comment
Hide comment
@pb0101

pb0101 Mar 16, 2018

This is so basic. I am running into issues with some large stacks. Hopefully this issue gets fixed soon.

pb0101 commented Mar 16, 2018

This is so basic. I am running into issues with some large stacks. Hopefully this issue gets fixed soon.

@hashicorp hashicorp locked and limited conversation to collaborators Mar 16, 2018

@apparentlymart apparentlymart added config and removed core labels Aug 1, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.