Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[testing framework] Add documentation #33454

Merged
merged 9 commits into from
Aug 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions internal/command/jsonformat/state.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ import (

type State struct {
StateFormatVersion string `json:"state_format_version"`
RootModule jsonstate.Module `json:"root,omitempty"`
RootModuleOutputs map[string]jsonstate.Output `json:"root_module_outputs,omitempty"`
RootModule jsonstate.Module `json:"root_module,omitempty"`
RootModuleOutputs map[string]jsonstate.Output `json:"outputs,omitempty"`

ProviderFormatVersion string `json:"provider_format_version"`
ProviderSchemas map[string]*jsonprovider.Provider `json:"provider_schemas,omitempty"`
Expand Down
2 changes: 1 addition & 1 deletion internal/command/views/test_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2734,7 +2734,7 @@ func TestTestJSON_Run(t *testing.T) {
"test_state": map[string]interface{}{
"state_format_version": "1.0",
"provider_format_version": "1.0",
"root": map[string]interface{}{
"root_module": map[string]interface{}{
"resources": []interface{}{
map[string]interface{}{
"address": "test_resource.creating",
Expand Down
14 changes: 9 additions & 5 deletions website/data/cli-nav-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,13 @@
}
]
},
{
"title": "Testing Terraform",
"routes": [
{ "title": "Overview", "path": "test" },
{ "title": "<code>test<code>", "href": "/cli/commands/test"}
]
},
{
"title": "Automating Terraform",
"routes": [
Expand Down Expand Up @@ -320,10 +327,7 @@
"href": "/cli/commands/state/show"
},
{ "title": "<code>taint</code>", "href": "/cli/commands/taint" },
{
"title": "<code>test (deprecated)</code>",
"href": "/cli/commands/test"
},
{ "title": "<code>test</code>", "href": "/cli/commands/test" },
{ "title": "<code>untaint</code>", "href": "/cli/commands/untaint" },
{ "title": "<code>validate</code>", "href": "/cli/commands/validate" },
{ "title": "<code>version</code>", "href": "/cli/commands/version" },
Expand Down Expand Up @@ -406,7 +410,7 @@
]
},
{ "title": "taint", "path": "commands/taint" },
{ "title": "test (deprecated)", "path": "commands/test", "hidden": true },
{ "title": "test", "path": "commands/test" },
{ "title": "untaint", "path": "commands/untaint" },
{ "title": "validate", "path": "commands/validate" },
{ "title": "version", "path": "commands/version" },
Expand Down
14 changes: 4 additions & 10 deletions website/data/language-nav-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,8 @@
"routes": [
{ "title": "Overview", "path": "files" },
{ "title": "Override Files", "path": "files/override" },
{
"title": "Dependency Lock File",
"path": "files/dependency-lock"
}
{ "title": "Dependency Lock File", "path": "files/dependency-lock" },
{ "title": "Test Files", "path": "files/tests" }
]
},
{
Expand Down Expand Up @@ -213,11 +211,6 @@
"path": "modules/develop/refactoring"
}
]
},
{
"title": "Module Testing Experiment",
"path": "modules/testing-experiment",
"hidden": true
}
]
},
Expand Down Expand Up @@ -1059,8 +1052,9 @@
}
]
},
{ "title": "Tests", "path": "tests" },
{
"title": "Upgrading to Terraform v1.5",
"title": "Upgrading to Terraform v1.6",
"path": "upgrade-guides"
},
{
Expand Down
44 changes: 38 additions & 6 deletions website/docs/cli/commands/test.mdx
Original file line number Diff line number Diff line change
@@ -1,13 +1,45 @@
---
page_title: 'Command: test'
description: Part of the ongoing design research for module integration testing.
description: >-
The `terraform test` command loads and executes Terraform testing files.
---

# Command: test

The `terraform test` command is currently serving as part of
[the module integration testing experiment](/terraform/language/modules/testing-experiment).
The `terraform test` command reads in Terraform testing files and executes the tests within.

It's not ready for routine use, but if you'd be interested in trying the
prototype functionality then we'd love to hear your feedback. See the
experiment details page linked above for more information.
The `test` command, and the test file syntax, are particularly helpful for module authors who want to validate and test their shared modules. You can also use the `test` command to validate root modules.

## Usage

Usage: `terraform test [options]`

This command searches the current directory and the specified testing directory (`tests`, by default) for any Terraform testing files, and executes the specified tests. Refer to [Tests](/terraform/language/tests) for more details on test files.

Terraform then executes a series of Terraform plan or apply commands according to the test files' specifications, and also validates the relevant plan and state files according to the test files' specifications.

!> **Warning:** The Terraform test command can create real infrastructure than can cost you money. Refer to the [Terraform Test Cleanup](#terraform-test-cleanup) section for best practices on ensuring created infrastructure is destroyed.

## General Options

The following options apply to the Terraform `test` command:

* `-filter=testfile` - Limits the `terraform test` operation to the specified test files.

* `-json` - Displays machine-readable JSON output for your testing results.

* `-test-directory=<relative directory>` - Overrides the directory that Terraform looks into for test files. Note that Terraform always loads testing files within the main configuration directory. The default testing directory is `tests`.

* `-verbose` - Prints out the plan or state for each `run` block within a test file, based on the `command` attribute of each `run` block.

## State Management

Each Terraform test file will maintain all Terraform state it requires within memory as it executes, starting empty. This state is entirely separate from any existing state for the configuration under test, so you can safely execute Terraform test commands without affecting any live infrastructure.

### Terraform Test Cleanup

The Terraform `test` command creates _real_ infrastructure. Once Terraform fully executes each test file, Terraform attempts to destroy any remaining infrastructure. If it cannot do this, Terraform reports a list of resources it created but could not destroy.

You should monitor the output of the test command closely to ensure Terraform removes the infrastructure it created or perform manual cleanup if not. We recommend creating dedicated testing accounts within the target providers that you can routinely and safely purge to ensure any accidental and costly resources aren't left behind.

Terraform also provides diagnostics explaining why it could now automatically clean up. You should resolve these diagnostics to ensure that future clean-up operations are successful.
60 changes: 60 additions & 0 deletions website/docs/cli/test/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
page_title: Testing Terraform - Terraform CLI
description: >-
Write structured tests and validations for your configuration to ensure
correctness in your infrastructure.
---

# Testing Terraform

Terraform provides numerous testing capabilities to validate your infrastructure.

These testing capabilities fit into two main categories:

1. Validating your configuration and infrastructure as part of your regular Terraform operations.
2. Performing traditional unit and integration testing on your configuration.

Refer to [Custom Conditions](/terraform/language/expressions/custom-conditions) and [Checks](/terraform/language/checks) to learn more about the first testing capability. Terraform's [`test`](/terraform/cli/commands/test) command provides the second capability.

## A brief history

liamcervante marked this conversation as resolved.
Show resolved Hide resolved
The various testing capabilities were introduced in the following versions:

- Terraform v0.13.0 introduced [Input Variable Validation](/terraform/language/expressions/custom-conditions#input-variable-validation).
- Terraform v0.15.0 introduced an experimental Terraform [`test`](/terraform/language/v1.5.x/modules/testing-experiment) command.
- Terraform v1.2.0 introduced [Pre and Post-conditions](/terraform/language/expressions/custom-conditions#preconditions-and-postconditions).
- Terraform v1.5.0 introduced [Checks](/terraform/language/checks).
- Terraform v1.6.0 deprecated the experimental Terraform `test` command, and released an updated and finalized Terraform [`test`](/terraform/cli/commands/test) command.

Note the introduction and deprecation of the experimental `test` command, followed by the introduction of the finalized `test` command. Refer to the [v1.6.x Upgrade Guide](/terraform/language/v1.6.x/upgrade-guides) for a summary of the changes between the experimental and finalized command.

## The `test` command

The Terraform `test` command:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this list!! 🔥


- Locates Terraform testing files within your configuration directory.
- Provisions the infrastructure within your configuration as specified by each testing file.
- Runs the assertions from the test file against the provisioned infrastructure.
- Destroys the provisioned infrastructure at the end of the test.

The `test` command, along with command-line flags and options, is discussed in detail within [Command: test](/terraform/cli/commands/test).

### Write configuration for tests

Terraform test files [have their own configuration syntax](/terraform/language/tests). This test file syntax focuses on customizing Terraform executions for the current configuration and overriding variables and providers to test different behaviors.

## Validations

Validations allow you to verify aspects of your configuration and infrastructure as it is applied and created. Terraform Cloud also supports automated [Continuous Validation](/terraform/cloud-docs/workspaces/health#continuous-validation).

The Terraform `test` command also executes any validations within your configuration as part of the tests it executes. For more information on the available validation, refer to [Checks](/terraform/language/checks) and [Custom Conditions](/terraform/language/expressions/custom-conditions).

## Tests or Validations

You can write many validations as test assertions, but there are specific use cases for both.

Validations are executed during Terraform plan and apply operations, and the Terraform `test` command also runs validations while executing tests. Therefore, use validations to validate aspects of your configuration that should always be true and could impact the valid execution of your infrastructure.

Module authors should note that validations are executed and exposed to module users, so if they fail, ensure the failure messages are understandable and actionable.

In contrast, Terraform only executes tests when you run `terraform test`. Use tests to assert the correctness of any logical operations or specific behavior within your configuration. For example, you can test that Terraform creates conditional resources based on an input by setting the input controlling those resources to a certain value then verifying the resources Terraform creates.