Support Hashicorp Terraform for Stage/Prod Environment

[kevin9foong]

Could we explore using Terraform to manage our cloud infrastructure?

Description of feature/enhancement

Automate cloud provisioning with Terraform for both staging and prod env.

Justification

• Terraform allows us to track our infrastructure as code and perform version control.
• It also allows us to provision and de-provision infra in a single command.
• It also ensures replicability and ease of setting up future cloud environments (infrastructure variables etc)
• Hashicorp Vault supports managing of cloud related secrets.
• Terraform is an all-in-one infrastructure management tool. It does not only support GCP, it supports cloud providers (AWS/Azure), observability platforms such as Grafana etc, providing a single source of truth for all infrastructure used.
• It can also support Solr setup Solr for AWS using terraform

We could perhaps improve our infra/cloud management experience using Hashicorp terraform.

• The cons though is that there will be a learning curve to implement this. (Need to learn terraform).

Existing similar features and their shortcomings

• It can be hard to set up the current staging environments. Current solutions require running multiple commands to set up staging environment using the cloud GUI + gradlew google plugin combination. Leading to tougher infra setup.
• There isn't a single source of truth for infra setup currently.
• Lack of version control of infra

[wkurniawan07]

Much as terraform is a powerful tool (it is used extensively where I'm working), I don't think TM is operating in a scale that using terraform is beneficial in a way that outweighs its learning curve and quirks.
I will convert this to discussion first until we are certain we will be doing this.