Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add settings for autoscaling to Bigtable module. #1006

Merged
merged 7 commits into from
Nov 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
88 changes: 75 additions & 13 deletions modules/bigtable-instance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,12 @@ This module allows managing a single BigTable instance, including access configu

- [ ] support bigtable_gc_policy
- [ ] support bigtable_app_profile
- [ ] support cluster replicas
- [ ] support IAM for tables

## Examples

### Simple instance with access configuration
### Instance with access configuration

```hcl

Expand All @@ -32,24 +34,84 @@ module "bigtable-instance" {
}
# tftest modules=1 resources=4
```

### Instance with static number of nodes

If you are not using autoscaling settings, you must set a specific number of nodes with the variable `num_nodes`.

```hcl

module "bigtable-instance" {
source = "./fabric/modules/bigtable-instance"
project_id = "my-project"
name = "instance"
cluster_id = "instance"
zone = "europe-west1-b"
num_nodes = 5
}
# tftest modules=1 resources=1
```

### Instance with autoscaling (based on CPU only)

If you use autoscaling, you should not set the variable `num_nodes`.

```hcl

module "bigtable-instance" {
source = "./fabric/modules/bigtable-instance"
project_id = "my-project"
name = "instance"
cluster_id = "instance"
zone = "europe-southwest1-b"
autoscaling_config = {
min_nodes = 3
max_nodes = 7
cpu_target = 70
}
}
# tftest modules=1 resources=1
```

### Instance with autoscaling (based on CPU and/or storage)

```hcl

module "bigtable-instance" {
source = "./fabric/modules/bigtable-instance"
project_id = "my-project"
name = "instance"
cluster_id = "instance"
zone = "europe-southwest1-a"
storage_type = "SSD"
autoscaling_config = {
min_nodes = 3
max_nodes = 7
cpu_target = 70
storage_target = 4096
}
}
# tftest modules=1 resources=1
```
<!-- BEGIN TFDOC -->

## Variables

| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [name](variables.tf#L45) | The name of the Cloud Bigtable instance. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L56) | Id of the project where datasets will be created. | <code>string</code> | ✓ | |
| [zone](variables.tf#L88) | The zone to create the Cloud Bigtable cluster in. | <code>string</code> | ✓ | |
| [cluster_id](variables.tf#L17) | The ID of the Cloud Bigtable cluster. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
| [deletion_protection](variables.tf#L23) | Whether or not to allow Terraform to destroy the instance. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the instance will fail. | <code></code> | | <code>true</code> |
| [display_name](variables.tf#L28) | The human-readable display name of the Bigtable instance. | <code></code> | | <code>null</code> |
| [iam](variables.tf#L33) | IAM bindings for topic in {ROLE => [MEMBERS]} format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [instance_type](variables.tf#L39) | (deprecated) The instance type to create. One of 'DEVELOPMENT' or 'PRODUCTION'. | <code>string</code> | | <code>null</code> |
| [num_nodes](variables.tf#L50) | The number of nodes in your Cloud Bigtable cluster. | <code>number</code> | | <code>1</code> |
| [storage_type](variables.tf#L61) | The storage type to use. | <code>string</code> | | <code>&#34;SSD&#34;</code> |
| [table_options_defaults](variables.tf#L67) | Default option of tables created in the BigTable instance. | <code title="object&#40;&#123;&#10; split_keys &#61; list&#40;string&#41;&#10; column_family &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; split_keys &#61; &#91;&#93;&#10; column_family &#61; null&#10;&#125;">&#123;&#8230;&#125;</code> |
| [tables](variables.tf#L79) | Tables to be created in the BigTable instance, options can be null. | <code title="map&#40;object&#40;&#123;&#10; split_keys &#61; list&#40;string&#41;&#10; column_family &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [name](variables.tf#L56) | The name of the Cloud Bigtable instance. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L67) | Id of the project where datasets will be created. | <code>string</code> | ✓ | |
| [zone](variables.tf#L99) | The zone to create the Cloud Bigtable cluster in. | <code>string</code> | ✓ | |
| [autoscaling_config](variables.tf#L17) | Settings for autoscaling of the instance. If you set this variable, the variable num_nodes is ignored. | <code title="object&#40;&#123;&#10; min_nodes &#61; number&#10; max_nodes &#61; number&#10; cpu_target &#61; number,&#10; storage_target &#61; optional&#40;number, null&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [cluster_id](variables.tf#L28) | The ID of the Cloud Bigtable cluster. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
| [deletion_protection](variables.tf#L34) | Whether or not to allow Terraform to destroy the instance. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the instance will fail. | <code></code> | | <code>true</code> |
| [display_name](variables.tf#L39) | The human-readable display name of the Bigtable instance. | <code></code> | | <code>null</code> |
| [iam](variables.tf#L44) | IAM bindings for topic in {ROLE => [MEMBERS]} format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [instance_type](variables.tf#L50) | (deprecated) The instance type to create. One of 'DEVELOPMENT' or 'PRODUCTION'. | <code>string</code> | | <code>null</code> |
| [num_nodes](variables.tf#L61) | The number of nodes in your Cloud Bigtable cluster. This value is ignored if you are using autoscaling. | <code>number</code> | | <code>1</code> |
| [storage_type](variables.tf#L72) | The storage type to use. | <code>string</code> | | <code>&#34;SSD&#34;</code> |
| [table_options_defaults](variables.tf#L78) | Default option of tables created in the BigTable instance. | <code title="object&#40;&#123;&#10; split_keys &#61; list&#40;string&#41;&#10; column_family &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; split_keys &#61; &#91;&#93;&#10; column_family &#61; null&#10;&#125;">&#123;&#8230;&#125;</code> |
| [tables](variables.tf#L90) | Tables to be created in the BigTable instance, options can be null. | <code title="map&#40;object&#40;&#123;&#10; split_keys &#61; list&#40;string&#41;&#10; column_family &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |

## Outputs

Expand Down
15 changes: 11 additions & 4 deletions modules/bigtable-instance/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ locals {
tables = {
for k, v in var.tables : k => v != null ? v : var.table_options_defaults
}
num_nodes = var.autoscaling_config == null ? var.num_nodes : null
}

resource "google_bigtable_instance" "default" {
Expand All @@ -27,6 +28,16 @@ resource "google_bigtable_instance" "default" {
cluster_id = var.cluster_id
zone = var.zone
storage_type = var.storage_type
num_nodes = local.num_nodes
dynamic "autoscaling_config" {
for_each = var.autoscaling_config == null ? [] : [""]
content {
min_nodes = var.autoscaling_config.min_nodes
max_nodes = var.autoscaling_config.max_nodes
cpu_target = var.autoscaling_config.cpu_target
storage_target = var.autoscaling_config.storage_target
}
}
}
instance_type = var.instance_type

Expand Down Expand Up @@ -56,8 +67,4 @@ resource "google_bigtable_table" "default" {
family = each.value.column_family
}
}

# lifecycle {
# prevent_destroy = true
# }
}
13 changes: 12 additions & 1 deletion modules/bigtable-instance/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,17 @@
* limitations under the License.
*/

variable "autoscaling_config" {
description = "Settings for autoscaling of the instance. If you set this variable, the variable num_nodes is ignored."
type = object({
min_nodes = number
max_nodes = number
cpu_target = number,
storage_target = optional(number, null)
})
default = null
}

variable "cluster_id" {
description = "The ID of the Cloud Bigtable cluster."
type = string
Expand Down Expand Up @@ -48,7 +59,7 @@ variable "name" {
}

variable "num_nodes" {
description = "The number of nodes in your Cloud Bigtable cluster."
description = "The number of nodes in your Cloud Bigtable cluster. This value is ignored if you are using autoscaling."
type = number
default = 1
}
Expand Down