Terraform output needs an option to exclude the provider block

[jaygorrell]

Since one of the primary purposes of the Terraform output is to fit into an existing TF environment, the provider block is likely to already exist. It would be great if there were a simple flag to --include-provider=[true|false].

It may even make sense to default to false given what I assume to be the more common workflow.

[justinsb]

Great suggestion - I'll see what can be done.

[shamil]

This can be easily done using terrsform itself, not sure putting that logic inside kops will not over complicate things. kops should keep things simple.

Terraform supports overrides nativelly. So you could put an override.tf file in the same directory where kops outputs the terraform confgs. The override.tf file could contain something like:

provider "aws" {
  region = "us-east-1"
  access_key = "XXXXXXXXXXXXXXXXXXXX"
  secret_key = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}

That's it, now when you apply the configs, terraform will know to use your provider instead of kops's one. In my case I do lot's of overrides without affecting kops generated configs whatsoever. Works for me for long time without issues.

Read more about terraform overrides: https://www.terraform.io/docs/configuration/override.html

[jaygorrell]

I'll defer to @justinsb on the ticket itself but that's a really great suggestion, @shamil. It should solve my specific problem... thanks!

[krisnova]

@justinsb - Stretch goal for 1.4.2

[justinsb]

Slight problem ... we need to specify the region. That's why we're outputing the block (there is nothing else in there). Is there a terraform trick I'm missing?

[jaygorrell]

Odds are if someone is using TF output from Kops they're already using TF and have a provider with a region specified. Given that, my thought was that the block itself could be optional because what Kops outputs may not be exactly right for the user (ie. they have access keys being passed to the provider, or an alias).

The block is definitely needed, it's just a question of where it comes from.

[chrislovecnm]

Can you mock up something for us? Even better if you can contribute ;)

[jaygorrell]

I'd love to contribute but I need to get past the intimidation factor of a new unknown project ...oh, and learn Go.

As a mockup, I'm not sure what it'd look like really. The only thing you'd need is an extra flag used with the TF output to indicate if you want the provider or not. Since --target output is it's own flag, it currently doesn't support extra options. Maybe --target-options with some flags for different outputs would work but I don't know if that's heading down a bad path or not. Something like --target-options=noprovider.

Either way, this is unlikely still a P1 since @shamil posted a great workaround. The editor complains about duplicate providers being declared but the override takes precedence.

[chrislovecnm]

We have developer office hours, if you want to swing by and talk through changes. The Friday coming up at 9 am mt

[jaygorrell]

I'd love to make it to some of these but it conflicts with my daily standups - difficult for me to make things like this during business hours.

[krisnova]

@jaygorrell can we set up a one off 1:1? Would love to meet you, and introduce you to the project.. touch base.. etc etc.. Doesn't need to be anything crazy.. just a quick hangout to go over introductions, talk about this issue, and the other topics you are involved in. Feel free to ping me on slack!

[jaygorrell]

Apologies for the delay but thanks for the offer @kris-nova ... I actually just got approval to attend these since we make heavy use of Kubernetes as part of the business. It may not be every time, but I'll do what I can when other pressing things aren't going on.

[justinsb]

Moving to 1.5.1, as the requirement isn't yet clear and 1.5.0 is hopefully imminent.

[erickt]

I had a workaround for this, where I was able to put my provider in a supermodule, and the kops terraform in a submodule, like this:

provider "aws" {
  profile = "some_profile"
  region = "us-west-2"
}

module "k8s" {
  source = "./kubernetes"
}

Terraform would then use the outer provider, not the inner one. Unfortunately it seems that Terraform 0.11's new provider support has broken this workaround, and now will error out with:

...

Error: Error refreshing state: 1 error(s) occurred:

* module.kubernetes.provider.aws: No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

I feel like the easiest solution would be for kops to not generate a provider, and document how to use the generated terraform as a module.

[erickt]

According to hashicorp/terraform#16721, there is now no way to override a provider in a submodule if any parameters are set, like the region. So it looks like we'll need to work around this somehow.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close