Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

depends_on cannot be used in a module #10462

Closed
chrisrlong opened this issue Dec 1, 2016 · 136 comments
Closed

depends_on cannot be used in a module #10462

chrisrlong opened this issue Dec 1, 2016 · 136 comments
Milestone

Comments

@chrisrlong
Copy link

@chrisrlong chrisrlong commented Dec 1, 2016

Hi there,

Terraform Version

0.8.0 rc1+

Affected Resource(s)

module

Terraform Configuration Files

module "legacy_site" {
  source = "../../../../../modules/site"
  name = "foo-site"
  health_check_target = "TCP:443"
  azs = "${var.azs}"
  instance_count = "${var.instance_count}"
  vpc = "apps"
  region = "${var.region}"
  environment = "${var.environment}"
  run_list = "hs_site_foo"

  #rds_complete = "${module.rds.db_instance_id}"
  #elasticache_cache_complete = "${module.elasticache_cache.elasticache_id}"
  #elasticache_sessions_complete = "${module.elasticache_sessions.elasticache_id}"

  depends_on = [
  "module.rds",
  "module.elasticache_sessions"
  ]

}

Debug Output

Error loading Terraform: module root: module legacy_site: depends_on is not a valid parameter
module root: module legacy_site: depends_on is not a valid parameter

Expected Behavior

I am trying to use the new depends_on instead of the above outputs, so I create and provision my app once I know database and caches are built.

Actual Behavior

Nothing as terraform errors out as above.

Steps to Reproduce

  1. terraform apply

References

depends_on can reference modules. This allows a resource or output to depend on everything within a module. (#10076)

@stack72
Copy link
Contributor

@stack72 stack72 commented Dec 1, 2016

Hi @chrisrlong

This is very strange - we introduced depends_on for modules in 0.8.0-beta2

can you run terraform version and post the output here for me?

Paul

@stack72 stack72 added bug core labels Dec 1, 2016
@chrisrlong
Copy link
Author

@chrisrlong chrisrlong commented Dec 1, 2016

Hi @stack72,

I have complied master from version b15b8bd (Terraform v0.8.0-dev (b15b8bd+CHANGES)), which should include beta2 features. (I did this to get 10337 and 10338 fixes)

It seems it would work if you set a resource to depend on a module, but you cannot set a module to depend_on another module......

Thanks for the great tool btw ;)

Chris

@stack72
Copy link
Contributor

@stack72 stack72 commented Dec 1, 2016

Hi @chrisrlong

thanks for getting back to me - you are indeed correct - modules cannot (currently) depend_on other modules but can depend on a resource only

Will change the tag on this from bug to enhancement :)

Keep an eye out for it soon ™️

Thanks

Paul

@mitchellh
Copy link
Member

@mitchellh mitchellh commented Dec 1, 2016

Retagging as enhancement, since this isn't broken functionality, its functionality that doesn't exist yet. :)

@mitchellh mitchellh added enhancement and removed bug labels Dec 1, 2016
@sheerun
Copy link
Contributor

@sheerun sheerun commented Jan 3, 2017

My use case: the slave virtual machine depends on master virtual machine to exist

@shantanugadgil
Copy link

@shantanugadgil shantanugadgil commented Jan 7, 2017

I am seeing the same error:
module root: module example: depends_on is not a valid parameter

My Terraform version is:
Terraform v0.8.2

My use case is that I want a module to depend on a resource.

@arehmandev
Copy link
Contributor

@arehmandev arehmandev commented Jan 9, 2017

Error loading Terraform: module root: module etcdbastion: depends_on is not a valid parameter.
^ Seeing it here too

@cemo
Copy link

@cemo cemo commented Jan 10, 2017

@mitchellh I believe this one can solve unnested modules problem too. There were some issues regarding this #10883 before. This solution would be intuitive and address these issues.

@vladimir-kozyrev
Copy link

@vladimir-kozyrev vladimir-kozyrev commented Feb 14, 2017

Waiting for this feature to be added 👍

@prog893

This comment was marked as off-topic.

1 similar comment
@avatar4d

This comment was marked as off-topic.

@panmanphil

This comment was marked as off-topic.

@shoelessrob

This comment was marked as off-topic.

7 similar comments
@mschenck

This comment was marked as off-topic.

@cpanayides

This comment was marked as off-topic.

@derBroBro

This comment was marked as off-topic.

@waffleshop

This comment was marked as off-topic.

@laiph

This comment was marked as off-topic.

@mqasim1983

This comment was marked as off-topic.

@seyeda

This comment was marked as off-topic.

@AirbornePorcine
Copy link

@AirbornePorcine AirbornePorcine commented Mar 10, 2017

Anyone have a good workaround for this in the meantime?

@jacobwgillespie
Copy link

@jacobwgillespie jacobwgillespie commented Mar 10, 2017

My pseudo workaround is to add a list variable inside the module:

variable "depends_on" { default = [], type = "list" }

Then when using the module, pass it a computed value from the resource I want it to depend on:

module "something" {
  depends_on = ["${aws_instance.instance.private_ip}"]
}
@AirbornePorcine
Copy link

@AirbornePorcine AirbornePorcine commented Mar 10, 2017

Thanks, that doesn't entirely fit what I think this issue is talking about though - making a module able to depend on another one. I don't have any resources that the module should depend on - it should depend on another module being created.

I've tried doing stuff like this:
module 1:

output "wait_for_cluster" { value = "ref to a resource that gets created last by this module" }

module 2:

variable "wait_for_cluster" {}

Main terraform template:

module "1" {

}

module "2" {
 wait_for_cluster = "${module.1.wait_for_cluster}"
}

But this doesn't do anything - my module two is still created at basically the same time as module 1.

@b-dean
Copy link

@b-dean b-dean commented Mar 10, 2017

@AirbornePorcine In your second module you need to actually use the wait_for_cluster variable somewhere that establishes a dependency. Such as in a template or a trigger on a null_resource, etc.

See #1178 (comment) from @kristjanelias

The only change I'd make to his workaround is to maybe make the dummy dependency resource use a trigger so it changes when the instance changes.

resource "null_resource" "dummy_dependency" {
  triggers {
    dependency_id = "${aws_instance.instance.id}"
  }
}
@AirbornePorcine
Copy link

@AirbornePorcine AirbornePorcine commented Mar 10, 2017

Ah, that makes sense! Should have known that Terraform would optimize away my variable if I didn't use it. :) I tried out your suggestion with the trigger and that didn't do it, but setting the depends_id on a tag on some resource in module 2 seems to have done it. Hacky but oh well I guess.

@jspiro
Copy link

@jspiro jspiro commented Jan 17, 2020

Thank you @danieldreier – epic update and the transparency is appreciated! I think we can finally all rest easy that this is coming ^_^

@chrisrlong
Copy link
Author

@chrisrlong chrisrlong commented Jan 20, 2020

Thanks is good news @danieldreier. Time will come to remove all the work arounds soon then...

@jared2501
Copy link

@jared2501 jared2501 commented Jan 22, 2020

Glad to hear! super exciting

@ortelli-oxalide
Copy link

@ortelli-oxalide ortelli-oxalide commented Jan 22, 2020

Thanks @danieldreier we are looking forward to that !

As I cannot see anyone else reporting a specific issue related to module dependencies that I'm encountering, I'll report it here. I hope that the support of "depends_on" on modules will resolve that, but I'm not quite sure. Any idea ?

With some code like this:

module "some_module" {
  ...
  some_parameter = module.another_module.output
  ...
}

# some_module source code
resource "some_resource" "_" {
  count = length(var.send_rds_logs_to_dd_with_lambda_arn)
  ...
}

Terraform raise the following error (at plan or apply):

Error: Invalid count argument
...
in resource "some_resource" "_":
   2:   count = length(var.send_rds_logs_to_dd_with_lambda_arn)

The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the count depends on.
@matti
Copy link

@matti matti commented Jan 23, 2020

I think I managed to create a valid workaround for this: https://github.com/matti/terraform-module-depends_on

It requires modules to have some boilerplate, but otherwise it seems to work fine.

@kuwas
Copy link

@kuwas kuwas commented Jan 24, 2020

@danieldreier @apparentlymart Thanks for the update and putting the emphasis on this, it is greatly appreciated.

@solidDoWant
Copy link

@solidDoWant solidDoWant commented Jan 27, 2020

@danieldreier Great to hear you guys are working on this. Have you guys been able to scope and figure out a timeline for the project yet? Roughly when this is planned to be released is one of the deciding factors for if my team will be using TFC or switching to Pulumi. Can you provide any broad information on when this will be released (week/month/year)?

Edit: Others might still need an ETA but we don't. The lack of communication (both on GitHub and via support email) about any estimates as to when Hashicorp will complete features they've committed to has resulted in my team canceling our TFC subscription. We're now moving to Pulumi.

@danieldreier
Copy link
Contributor

@danieldreier danieldreier commented Feb 1, 2020

@solidDoWant While I appreciate the need for you and others to plan, we are not yet in a position to communicate any timelines. My hope above was to begin communicating what we’re currently focused on more publicly.

I can give a few details on the larger first-class module project that this is a part of to help you with your planning:

  • improving modules is the main feature of the next terraform release: we're expecting to ship make modules first-class in 0.13, which is the next major release. The only other thing that we'll block 0.13.0 on is a set of critical bug fixes that require backward-incompatible behavior changes. We're not planning on holding back 0.13.0 for any other features.
  • depends_on may ship after count and for_each: depends_on is a subset of the larger project of first-class modules. As part of that project, we'll make depends_on, count, and for_each work in modules. I can't tell you, yet, whether we'll ship depends_on, and so count and for_each all at once, or what order they'll ship in - we may ship for_each and count in 0.13.0 and then quickly follow up with a minor release that adds depends_on. We do intend to batch any breaking changes in 0.13.0, so that the later addition of depends_on would be purely additive.
@jspiro
Copy link

@jspiro jspiro commented Feb 1, 2020

That prioritization makes a lot of sense to me. Looking forward to it!

@pixelicous
Copy link

@pixelicous pixelicous commented Feb 5, 2020

Great, 3 years in, looking forward to having this needed feature.

@hakro
Copy link

@hakro hakro commented Mar 2, 2020

This is a really important feature to have in my opinion. It would make the code much more readable and maintainable than by using one of the hack that we find on blog posts.

@emmaLP
Copy link

@emmaLP emmaLP commented Mar 4, 2020

Can anyone please give a time line on when this feature will be released?

@max-l-weaver
Copy link

@max-l-weaver max-l-weaver commented Mar 5, 2020

Wow, over three years and still nothing?!

@matti
Copy link

@matti matti commented Mar 5, 2020

three years is nothing in terraform time

@ketzacoatl
Copy link
Contributor

@ketzacoatl ketzacoatl commented Mar 12, 2020

Wow, over three years and still nothing?!

@max-l-weaver with this comment, you are showing how unappreciative and ignorant you can be.

You can fault the devs for not getting this right with modules from the start, but aside from that, you should have more awareness of reality and what these changes have required before sharing those types of crass comments.

In fact, you should probably not use any open source software, as the type of comment you left above are one of the major reasons why application developers give up on open source.

Please be more kind and appreciative in your comments, or leave them to yourself.

@solidDoWant
Copy link

@solidDoWant solidDoWant commented Mar 12, 2020

Just because the software is open source doesn't mean we aren't paying for it. Many of us (myself previously included) pay a monthly or yearly fee for Terraform/TFC and in term expect support and timely software updates. The fact that Hashicorp rarely if ever shares roadmaps or any kind of even a guess at a timeline quite frankly makes it unusable for business critical services, such as the infrastructure backing software company's services. The lack of communication on one of, if not the most requested feature of the last 3-4 years is not something to appreciate.

@confiq
Copy link

@confiq confiq commented Mar 12, 2020

@solidDoWant , chillrex man... Many people "pay a monthly" and wait patiently for a new feature.
This specific feature requires lot of tech resources and time, no need for such an outcry because it does not help anyone.

They decided to implement it which is great news, crass comments like these do not help anyone.

@kuwas
Copy link

@kuwas kuwas commented Mar 12, 2020

@solidDoWant There are various workarounds for this feature that currently exists, such using templates to generate tf files, so its not really a showstopper. We should be thankful that they even prioritized this at all.

@solidDoWant
Copy link

@solidDoWant solidDoWant commented Mar 12, 2020

To clarify, my issue isn't at all with the lack of the feature - it's with the lack of communication. I would be fine with Hashicorp never developing this feature, but I take issue with Hashicorp not being able to even take a guess as to when it will be completed, if at all. It's great that they've communicated that this is a top priority, as that address the 'if it will happen', but they haven't addressed the 'when'.

This Github issue isn't the only place affected by the underlying company issue either. There are 136 PRs on this repo alone (not to mention the numerous providers) that are open. Out of those 135, only one has been assigned a milestone. Roughly 15 haven't even been commented on. On most of these there is no indication when, or if these PRs will be addressed. Not great from a user's perspective when there's a fix you're waiting on in one of them, or in a developer's perspective when your PR sits open for months or years.

Hashicorp makes some cool tools and (from what I've seen) most of their code is pretty solid. I've no issues with the software developers, but their product strategy and overall communication leaves a lot to be desired.

@camjackson
Copy link

@camjackson camjackson commented Mar 12, 2020

@danieldreier can you please lock this issue to contributors only? I believe at this point just about every constructive comment that could be made, has already been made.

There are likely hundreds of people subscribed to this issue, and we don't all need to get an email every time someone wants to complain. I'll be happy if the next time I hear about this feature is when it's done.

@ketzacoatl
Copy link
Contributor

@ketzacoatl ketzacoatl commented Mar 12, 2020

To clarify, my issue isn't at all with the lack of the feature - it's with the lack of communication.

Thanks for clarifying.

I would be fine with Hashicorp never developing this feature, but I take issue with Hashicorp not being able to even take a guess as to when it will be completed, if at all. It's great that they've communicated that this is a top priority, as that address the 'if it will happen', but they haven't addressed the 'when'.

I can understand that frustration, I agree Hashicorp could improve their communication with us as users of various kinds.

However, WRT this general topic of first-class modules, I believe Hashicorp has been pretty clear, and within the bounds of what's possible (and also avoiding making unnecessary expectations which they may fail to deliver on).

For example, in the 3 years that this has transpired, we've seen the following:

  1. many, rather verbose and elaborate discussions of the use cases and needs and possibilities, the problems and blocks, as well as how to get there.
  2. Hashicorp realizes their fsck up, and acknowledges that TF core needs to be refactored in major ways.
  3. Hashicorp lays down the roadmaps for 0.11 and 0.12, with various breaking/major changes that will facilitate the next round of changes
  4. Hashicorp plows through 0.11 and 0.12 releases, following through as planned.
  5. Hashicorp starts work on 0.13 which is focusing on modules as a first-class citizen

Particularly for those last two, no timelines (with unreasonable expectations) were laid down. Yea, I would have loved to see 0.12 completed sooner, or dev on 0.13 to have started sooner, but I'd rather this than to have a product which is all over the place, poorly developed, poorly designed, lots of major breaking changes, and just generally moving to the whims of the wind as it blows.

@max-l-weaver
Copy link

@max-l-weaver max-l-weaver commented Mar 13, 2020

@ketzacoatl please take a deep breath and calm down.

Imo Hashicorp market themselves as a software provider that provides solutions for businesses. If it was some dude working on his own in his basement then of course i wouldn't post such ignorant comments, however the fact that this feature has been open for over three years with no communication is a little disconcerting. I apologise if you felt somehow attacked by my comment.

@phildier
Copy link

@phildier phildier commented Mar 13, 2020

Please take the complaints to twitter or reddit. I'm subscribed to this thread for news about the feature only.

@hashicorp hashicorp locked as too heated and limited conversation to collaborators Mar 25, 2020
@danieldreier
Copy link
Contributor

@danieldreier danieldreier commented Mar 25, 2020

Based on the feedback here, I've locked this discussion. We're actively working on module expansion. The engineers are making good progress, and I am excited about 0.13. For folks who are really upset, I'm ddreier@hashicorp.com and I'm happy to talk.

@jbardin jbardin added this to the v0.13.0 milestone May 20, 2020
@danieldreier
Copy link
Contributor

@danieldreier danieldreier commented May 21, 2020

I'm very excited to announce that beta 1 of terraform 0.13.0 will be available on June 3rd, and will include module depend_on. I've pinned an issue with more details about the beta program, and posted a discuss thread for folks who want to talk about it more.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

You can’t perform that action at this time.