Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync terraform EKS module with the reference architecture #11995

Merged
merged 9 commits into from
Aug 18, 2022
Merged

Conversation

nandajavarma
Copy link
Contributor

@nandajavarma nandajavarma commented Aug 9, 2022

Description

This PR adds a single-cluster AWS terraform module, corresponding to our reference architecture for the same cloud provider.

Related Issue(s)

Fixes #11533

How to test

You can follow the README added to this PR to test this terraform module.

Release Notes

NONE

Documentation

Werft options:

  • /werft with-preview

Copy link
Contributor

@lucasvaltl lucasvaltl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not fully review all of the settings and resources used to make sure that they match the reference architecture, but I did already find that there is a deviation in the DB set up and wanted to already mention that :)

--> For those thar review this: please make sure to review that the infrastructure created via this terraform matches the infrastructure described in the reference architecture guide exactly.

install/infra/modules/eks/database.tf Outdated Show resolved Hide resolved
@nandajavarma
Copy link
Contributor Author

@lucasvaltl I have addressed the comments. Sorry on the back and forth on this. I kinda of went easy on the configurable settings considering it can/will be changed. I will keep this in mind and try to do an exact 1:1 config.

@lucasvaltl
Copy link
Contributor

lucasvaltl commented Aug 10, 2022

👍 Thank you! And no worries - maybe the intent here was not clear enough from my side. We want this terraform to effectively create exactly what the guide describes :)

Copy link
Contributor

@adrienthebo adrienthebo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Looks good overall, I see a few hot spots but nothing major.

  1. Security group inline rules vs aws_security_group_rule instances - the docs say that inline rules and separate rule resources may not play well together. Can we check this out and see if we have any risk here? We have a couple of egress rules addressed to 0.0.0.0/0 which deviates from the reference architecture; I'd like to confirm that they're not in use so we're testing the strict policies defined in the reference architecture.
  2. RDS publicly_accessible setting - can we drop this setting?
  3. S3 security policies - can we restrict the policy to the object storage and container registry buckets?

As noted inline I have a handful of opportunistic or style comments; if any of them resonate with you then we can address them but can and should skip if the review process starts running long.

Edit: one more quick topic - it seems like the only way to keep track of AWS resources is with the copious use of labels on every resource. Are we using labels to identify associated resources? (If not, give me a yell and I'll spin up a ticket to add that later.)

install/infra/modules/eks/variables.tf Outdated Show resolved Hide resolved
install/infra/modules/eks/variables.tf Outdated Show resolved Hide resolved
install/infra/modules/eks/database.tf Outdated Show resolved Hide resolved
install/infra/modules/eks/database.tf Outdated Show resolved Hide resolved
install/infra/modules/eks/kubernetes.tf Outdated Show resolved Hide resolved
install/infra/modules/eks/storage.tf Outdated Show resolved Hide resolved
install/infra/modules/eks/storage.tf Outdated Show resolved Hide resolved
install/infra/single-cluster/aws/README.md Show resolved Hide resolved
@@ -0,0 +1,6 @@
terraform {
backend "s3" {
bucket = "gitpod-tf"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: If a user directly uses this module, will this backend definition conflict with their environment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point! I could leave the bucket field empty since I cannot use variables in bucket definition. But I am also thinking, the chances of them having a bucket named gitpod-tf is unlikely, so it will error out saying no such bucket is present. Not sure what the right route is, I might leave this as is.

@lucasvaltl
Copy link
Contributor

lucasvaltl commented Aug 17, 2022

👋 It seems that errors can still creep up during the make plan phase on the EKS side: see screenshot here (internal). If I remember correctly, these were the exact same errors I ran into as well. On my side, it was because I was not starting off of a clean slate. I assume that @mbrevoort was although I am not sure. Can we please take a look at this before we merge this to see if we can get rid of these errors? Alternatively, if this is a case of users doing it wrong, it is likely worth updating the readme :)

Copy link
Contributor

@adrienthebo adrienthebo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied from Slack:

I see some issues that may pose some additional complications if users try to use it, such as the declaration of terraform backends within the modules and the presence of terraform.tfvars files that could be moved to terraform.tfvars.example. Nothing's a blocker but there are some sharp edges that we can remove.

This also represents a significant amount of work; getting it merged will make everyone's lives easier, and holding this back will block related PRs.

Let's move ahead with this PR and chase down some of these issues in follow-up PRs. Great work! 🍻

@nandajavarma
Copy link
Contributor Author

/unhold

@roboquat roboquat merged commit b09a3eb into main Aug 18, 2022
@roboquat roboquat deleted the nvn/tf-doc branch August 18, 2022 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none size/XXL team: delivery Issue belongs to the self-hosted team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sync AWS terraform module with reference architecture
4 participants