Skip to content

Commit

Permalink
Add autoscaling to DPMS terraform (#10522) (#724)
Browse files Browse the repository at this point in the history
[upstream:6742c3a25b388ebbd567b462693c4855e5c8c179]

Signed-off-by: Modular Magician <magic-modules@google.com>
  • Loading branch information
modular-magician committed Jun 12, 2024
1 parent 0f7be8b commit fd70dbf
Show file tree
Hide file tree
Showing 16 changed files with 486 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
21 changes: 21 additions & 0 deletions dataproc_metastore_service_autoscaling_max_scaling_factor/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
resource "google_dataproc_metastore_service" "test_resource" {
service_id = "test-service-${local.name_suffix}"
location = "us-central1"

# DPMS 2 requires SPANNER database type, and does not require
# a maintenance window.
database_type = "SPANNER"

hive_metastore_config {
version = "3.1.2"
}

scaling_config {
autoscaling_config {
autoscaling_enabled = true
limit_config {
max_scaling_factor = 1.0
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
===

These examples use real resources that will be billed to the
Google Cloud Platform project you use - so make sure that you
run "terraform destroy" before quitting!

===
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Dataproc Metastore Service Autoscaling Max Scaling Factor - Terraform

## Setup

<walkthrough-author name="rileykarson@google.com" analyticsId="UA-125550242-1" tutorialName="dataproc_metastore_service_autoscaling_max_scaling_factor" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>

Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.

<walkthrough-project-billing-setup></walkthrough-project-billing-setup>

Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.

## Terraforming!

Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
the project name from the environment variable.

```bash
export GOOGLE_CLOUD_PROJECT={{project-id}}
```

After that, let's get Terraform started. Run the following to pull in the providers.

```bash
terraform init
```

With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!

```bash
terraform apply
```

Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.

```bash
yes
```


## Post-Apply

### Editing your config

Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.

```bash
terraform plan
```

So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
run a 'plan' again.

```bash
terraform plan
```

Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
at the 'yes' prompt.

```bash
terraform apply
```

```bash
yes
```

## Cleanup

Run the following to remove the resources Terraform provisioned:

```bash
terraform destroy
```
```bash
yes
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
resource "google_dataproc_metastore_service" "test_resource" {
service_id = "test-service-${local.name_suffix}"
location = "us-central1"

# DPMS 2 requires SPANNER database type, and does not require
# a maintenance window.
database_type = "SPANNER"

hive_metastore_config {
version = "3.1.2"
}

scaling_config {
autoscaling_config {
autoscaling_enabled = true
limit_config {
min_scaling_factor = 0.1
max_scaling_factor = 1.0
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
===

These examples use real resources that will be billed to the
Google Cloud Platform project you use - so make sure that you
run "terraform destroy" before quitting!

===
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Dataproc Metastore Service Autoscaling Min And Max Scaling Factor - Terraform

## Setup

<walkthrough-author name="rileykarson@google.com" analyticsId="UA-125550242-1" tutorialName="dataproc_metastore_service_autoscaling_min_and_max_scaling_factor" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>

Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.

<walkthrough-project-billing-setup></walkthrough-project-billing-setup>

Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.

## Terraforming!

Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
the project name from the environment variable.

```bash
export GOOGLE_CLOUD_PROJECT={{project-id}}
```

After that, let's get Terraform started. Run the following to pull in the providers.

```bash
terraform init
```

With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!

```bash
terraform apply
```

Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.

```bash
yes
```


## Post-Apply

### Editing your config

Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.

```bash
terraform plan
```

So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
run a 'plan' again.

```bash
terraform plan
```

Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
at the 'yes' prompt.

```bash
terraform apply
```

```bash
yes
```

## Cleanup

Run the following to remove the resources Terraform provisioned:

```bash
terraform destroy
```
```bash
yes
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
21 changes: 21 additions & 0 deletions dataproc_metastore_service_autoscaling_min_scaling_factor/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
resource "google_dataproc_metastore_service" "test_resource" {
service_id = "test-service-${local.name_suffix}"
location = "us-central1"

# DPMS 2 requires SPANNER database type, and does not require
# a maintenance window.
database_type = "SPANNER"

hive_metastore_config {
version = "3.1.2"
}

scaling_config {
autoscaling_config {
autoscaling_enabled = true
limit_config {
min_scaling_factor = 0.1
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
===

These examples use real resources that will be billed to the
Google Cloud Platform project you use - so make sure that you
run "terraform destroy" before quitting!

===
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Dataproc Metastore Service Autoscaling Min Scaling Factor - Terraform

## Setup

<walkthrough-author name="rileykarson@google.com" analyticsId="UA-125550242-1" tutorialName="dataproc_metastore_service_autoscaling_min_scaling_factor" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>

Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.

<walkthrough-project-billing-setup></walkthrough-project-billing-setup>

Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.

## Terraforming!

Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
the project name from the environment variable.

```bash
export GOOGLE_CLOUD_PROJECT={{project-id}}
```

After that, let's get Terraform started. Run the following to pull in the providers.

```bash
terraform init
```

With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!

```bash
terraform apply
```

Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.

```bash
yes
```


## Post-Apply

### Editing your config

Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.

```bash
terraform plan
```

So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
run a 'plan' again.

```bash
terraform plan
```

Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
at the 'yes' prompt.

```bash
terraform apply
```

```bash
yes
```

## Cleanup

Run the following to remove the resources Terraform provisioned:

```bash
terraform destroy
```
```bash
yes
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
Loading

0 comments on commit fd70dbf

Please sign in to comment.