There are a few tasks you need to perform after cloning the repo to your local machine:
Terraform does not require installation. Just download the binary from https://www.terraform.io/ and copy it to a folder in your path. On Linux and Mac /usr/local/bin
is a good place.
Attention: Make sure to download the latest 0.11.x release. Terraform 0.12, when available, will be a different language and will require an Upgrade processs.
Verify your installation before proceeding:
$ terraform version
Terraform v0.11.11
The first command to run for a new configuration -- or after checking out an existing configuration from version control -- is terraform init
, which initializes various local settings and data that will be used by subsequent commands.
$ terraform init
Initializing modules...
...
The config is prepared to work with multiple environments. There is validation in place to make sure you don't accidentally deploy the wrong config to the wrong environment (dev to production for example). The validation is simple. It will check that your stage
variable has the same value as your current workspace. If that is not the case it will fail to continue.
Before continuing create the environments you want to use: $ terraform workspace new prod Created and switched to workspace "prod"!
You're now on a new, empty workspace. Workspaces isolate their state, so if you run "terraform plan" Terraform will not see any existing state for this configuration.
$ terraform workspace new prod
Created and switched to workspace "prod"!
You're now on a new, empty workspace. Workspaces isolate their state,
so if you run "terraform plan" Terraform will not see any existing state
for this configuration.
$ terraform workspace new dev
Created and switched to workspace "dev"!
You're now on a new, empty workspace. Workspaces isolate their state,
so if you run "terraform plan" Terraform will not see any existing state
for this configuration.
To show the current environment use the subcommand show
:
$ terraform workspace show
dev
To change the environment use the subcommand select
$ terraform workspace select prod
Switched to workspace "prod".
Here is what happens if you try to deploy to the wrong environment:
$ terraform apply
Error: module.alb.null_resource.validate_worspace: : invalid or unknown key:
ERROR: You are trying to run dev stage in the prod workspace.
Create/Select the correct workspace first.
Error: module.elasticsearch.module.cluster.null_resource.validate_worspace: : invalid or unknown key:
ERROR: You are trying to run dev stage in the prod workspace.
Create/Select the correct workspace first.
Error: module.kibana.module.cluster.null_resource.validate_worspace: : invalid or unknown key:
ERROR: You are trying to run dev stage in the prod workspace.
Create/Select the correct workspace first.
Error: module.logstash.module.cluster.null_resource.validate_worspace: : invalid or unknown key:
ERROR: You are trying to run dev stage in the prod workspace.
Create/Select the correct workspace first.
Error: null_resource.validate_worspace: : invalid or unknown key:
ERROR: You are trying to run dev stage in the prod workspace.
Create/Select the correct workspace first.
There are many ways to pass paramenters to terraform. The more convenient is through files. That repo contains to sample files that we can use. Copy dev.tfvars.sample
and prod.tfvars.sample
to dev.tfvars
and prod.tfvars
and adjust the parameters. The included values are reasonable defaults.
Here is the list of variables with the description:
Variable | Description | Default |
---|---|---|
aws_access_key | The AWS Access of the account used to deploy the infra. I suggest creating a dedicated user with only programatic acess. | |
aws_secret_key | The AWS secret key | |
region | AWS region | |
vpc | VPC ID where to deploy the instances. Has to be in the selected region. | |
subnets | At least 2 subnets are required or the load balances will fail to be provisioned. Each subnet should be on a different availability zone. Has to be in the selected vpc. | |
stage | The Stage or Environment you are deploying, e.g. prod , staging , dev , qa |
|
namespace | Namespace, which could be your organization name or abbreviation, e.g. co or company |
|
cluster_name | Elasticsearch Cluster Name. Will be concatenated with namespace and prefix to set Elasticsearch cluster.name variable |
|
ssh_key_pair | SSH key pair name that will be added to the instances. https://console.aws.amazon.com/ec2/v2/home?#KeyPairs | |
elk_version | Elastic Stack version to install (6.X.Y) during the initial provisioning. If blank or latest will install the latest version. |
latest |
es_instance_type | Elasticsearch Node instance type | |
es_node_count | Number of Elasticsearch Nodes in the cluster | 4 |
es_storage_size | Elasticsearch Node data volume size (GB) | 100 |
kibana_instance_type | Kibana instance type | |
kibana_node_count | Number of Kibana Nodes in the cluster | 2 |
logstash_instance_type | Logstash instance type | |
logstash_node_count | Number of Logstash Nodes in the cluster | 2 |
cerebro_instance_type | Cerebro instance type | t3.small |
cerebro_version | Cerebro version to provision (tested with 0.8.1) | 0.8.1 |
*_instance_type: Make sure that the instace types selected are available in the selected region.
Terraform will deploy the file files/logstash.conf
as the initial Logstash pipeline. You can customize it to your needs, but remember to keep the hosts
attributte of the elasticsearch
output like in the file if you want it to use the provisioned elasticsearch cluster.
output {
elasticsearch {
hosts => ['${elasticsearch_lb_host}']
...
}
}
Terraform will replace the string ${elasticsearch_lb_host}
with the correct value.
After adjusting the variables, make sure that you are in the appropriate environment and deploy the infrastructure:
$ terraform workspace select dev
Switched to workspace "dev".
$ terraform apply --var-file=dev.tfvars
...
Terraform will show you a summary of the changes and ask for confirmation. Carefully review the changes and, if you agree type yes
and enter to apply the changes.
After deploying the infrastructure Terraform will show you a set of usefuls output.
If you rather not specify the filename every time, simply rename you most used file to terraform.tfvars. Terraform you automatically load that file if it exists. Files explicitly passed with --var-file
will have precedence.
If you are done with a test environment and wants to clean everything up use the destroy
command:
$ terraform destroy --var-file=dev.tfvars
Terraform will show you the summary and ask for confirmation before continuing.