-
-
Notifications
You must be signed in to change notification settings - Fork 619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: run terraform validate
on modules again
#20230
Changes from all commits
686e2b2
809012d
e80e07a
a84690d
3db5912
1569028
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,124 @@ | ||
--- | ||
title: "Terraform Overview" | ||
slug: "terraform-overview" | ||
hidden: false | ||
createdAt: "2023-11-22T17:00:00.000Z" | ||
--- | ||
> 🚧 Terraform support is in alpha stage | ||
> | ||
> Pants is currently building support for developing and deploying Terraform. Simple use cases might be supported, but many options are missing. | ||
> | ||
> Please share feedback for what you need to use Pants with your Terraform modules and deployments by either [opening a GitHub issue](https://github.com/pantsbuild/pants/issues/new/choose) or [joining our Slack](doc:getting-help)! | ||
|
||
Initial setup | ||
============= | ||
|
||
First, activate the relevant backend in `pants.toml`: | ||
|
||
```toml pants.toml | ||
[GLOBAL] | ||
backend_packages = [ | ||
... | ||
"pants.backend.experimental.terraform", | ||
... | ||
] | ||
``` | ||
|
||
The Terraform backend also needs Python to run Pants's analysers. The setting `[python].interpreter_constraints` will need to be set. | ||
|
||
Adding Terraform targets | ||
------------------------ | ||
|
||
The Terraform backend has 2 target types: | ||
- `terraform_module` for Terraform source code | ||
- `terraform_deployment` for deployments that can be deployed with the `experimental-deploy` goal | ||
|
||
### Modules | ||
|
||
The `tailor` goal will automatically generate `terraform_module` targets. Run [`pants tailor ::`](doc:initial-configuration#5-generate-build-files). For example: | ||
|
||
``` | ||
❯ pants tailor :: | ||
Created src/terraform/root/BUILD: | ||
- Add terraform_module target root | ||
``` | ||
|
||
### Deployments | ||
|
||
`terraform_deployments` must be manually created. The deployment points to a `terraform_module` target as its `root_module` field. This module will be the "root" module that Terraform operations will be run on. You can reference vars files with the `var_files` field. You can have multiple deployments reference the same module: | ||
|
||
|
||
``` | ||
terraform_module(name="root") | ||
terraform_deployment(name="prod", root_module=":root", var_files=["prod.tfvars"]) | ||
terraform_deployment(name="test", root_module=":root", var_files=["test.tfvars"]) | ||
``` | ||
|
||
### Lockfiles | ||
|
||
Automatic lockfile management is currently in progress. You can include lockfiles manually as a dependency: | ||
|
||
``` | ||
terraform_deployment(name="prod", root_module=":root", dependencies=[":lockfile"]) | ||
file(name="lockfile", source=".terraform.lock.hcl") | ||
``` | ||
|
||
Basic Operations | ||
---------------- | ||
|
||
### Formatting | ||
|
||
Run `terraform fmt` as part of the `fix`, `fmt`, or `lint` goals. | ||
|
||
``` | ||
pants fix :: | ||
[INFO] Completed: pants.backend.terraform.lint.tffmt.tffmt.tffmt_fmt - terraform-fmt made no changes. | ||
|
||
✓ terraform-fmt made no changes. | ||
``` | ||
|
||
### Validate | ||
|
||
Run `terraform validate` as part of the `check` goal. | ||
|
||
``` | ||
pants check :: | ||
[INFO] Completed: pants.backend.terraform.goals.check.terraform_check - terraform-validate succeeded. | ||
Success! The configuration is valid. | ||
|
||
✓ terraform-validate succeeded. | ||
|
||
``` | ||
|
||
`terraform validate` isn't valid for all Terraform modules. Some child modules, in particular those using aliased providers, need to have their providers provided by a "root" module. You can opt these modules out of `validate` by setting `skip_terraform_validate=True`. For example: | ||
|
||
``` | ||
terraform_module(skip_terraform_validate=True) | ||
``` | ||
|
||
### Deploying | ||
|
||
> 🚧 Terraform deployment support is in alpha stage | ||
> | ||
> Many options and features aren't supported yet. | ||
> Local state backends aren't supported. | ||
|
||
|
||
Run `terraform apply` as part of the `experimental-deploy` goal. The process is run interactively, so you will be prompted for variables and confirmation as usual. | ||
|
||
``` | ||
pants experimental-deploy :: | ||
[INFO] Deploying targets... | ||
--- 8< --- | ||
Do you want to perform these actions? | ||
Terraform will perform the actions described above. | ||
Only 'yes' will be accepted to approve. | ||
|
||
Enter a value: yes | ||
--- 8< --- | ||
Apply complete! Resources: 4 added, 0 changed, 0 destroyed. | ||
|
||
✓ testprojects/src/terraform/root:root deployed | ||
``` | ||
|
||
You can set auto approve by adding `-auto-approve` to the `[download-terraform].args` setting in `pants.toml`. You can also set it for a single pants invocation with `--download-terraform-args='-auto-approve'`, for example `pants experimental-deploy "--download-terraform-args='-auto-approve'"`. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
--- | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe add this documentation page in a separate PR in order to keep this PR on one topic? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @benjyw: thoughts on above question? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm neutral on that question, since the docs are pertinent to the code change, even if wider. |
||
title: "Common subsystem tasks" | ||
slug: "plugins-common-subsystem" | ||
excerpt: "Common tasks for Subsystems" | ||
hidden: false | ||
createdAt: "2023-11-22T17:00:00.000Z" | ||
--- | ||
Skipping individual targets | ||
--------------------------- | ||
|
||
Many subsystems allow skipping specific targets. For example, you might have Python files that you want to not typecheck with mypy. In Pants, this is achieved with a `skip_*` field on the target. This is simple to implement. | ||
|
||
1. Create a field for skipping your tool | ||
|
||
```python | ||
from pants.engine.target import BoolField | ||
|
||
class SkipFortranLintField(BoolField): | ||
alias = "skip_fortran_lint" | ||
default = False | ||
help = "If true, don't run fortran-lint on this target's code." | ||
``` | ||
|
||
2. Register this field on the appropriate targets. | ||
|
||
```python | ||
def rules(): | ||
return [ | ||
FortranSourceTarget.register_plugin_field(SkipFortranLintField), | ||
] | ||
``` | ||
|
||
3. Add this field as part of your subsystems `opt_out` method: | ||
|
||
```python | ||
from dataclasses import dataclass | ||
|
||
from pants.engine.target import FieldSet, Target | ||
|
||
|
||
@dataclass | ||
class FortranLintFieldSet(FieldSet): | ||
required_fields = (FortranSourceField,) | ||
|
||
source: FortranSourceField | ||
|
||
@classmethod | ||
def opt_out(cls, tgt: Target) -> bool: | ||
return tgt.get(SkipFortranLintField).value | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,23 @@ | ||
# Copyright 2021 Pants project contributors (see CONTRIBUTORS.md). | ||
# Licensed under the Apache License, Version 2.0 (see LICENSE). | ||
from dataclasses import dataclass | ||
from typing import Union | ||
|
||
from pants.backend.terraform.dependencies import TerraformInitRequest, TerraformInitResponse | ||
from pants.backend.terraform.target_types import TerraformDeploymentFieldSet | ||
from pants.backend.terraform.target_types import ( | ||
TerraformBackendConfigField, | ||
TerraformDeploymentFieldSet, | ||
TerraformDeploymentTarget, | ||
TerraformFieldSet, | ||
TerraformModuleTarget, | ||
TerraformRootModuleField, | ||
) | ||
from pants.backend.terraform.tool import TerraformProcess | ||
from pants.core.goals.check import CheckRequest, CheckResult, CheckResults | ||
from pants.engine.internals.selectors import Get, MultiGet | ||
from pants.engine.process import FallibleProcessResult | ||
from pants.engine.rules import collect_rules, rule | ||
from pants.engine.target import BoolField, Target | ||
from pants.engine.unions import UnionRule | ||
from pants.option.option_types import SkipOption | ||
from pants.option.subsystem import Subsystem | ||
|
@@ -21,11 +32,41 @@ class TerraformValidateSubsystem(Subsystem): | |
skip = SkipOption("check") | ||
|
||
|
||
class SkipTerraformValidateField(BoolField): | ||
alias = "skip_terraform_validate" | ||
default = False | ||
help = "If true, don't run `terraform validate` on this target's code. If this target is a module, `terraform validate might still be run on a `terraform_deployment that references this module." | ||
|
||
|
||
@dataclass(frozen=True) | ||
class TerraformValidateFieldSet(TerraformFieldSet): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Doesn't this need to extend the parent's There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didn't know the answer, so I dug around. Seems the typical way does not add the skip field to the required fields, but also doesn't inherit from the parent FieldSet. Usually the fieldset has just the sources field as required. I think that's to let them cover all the target types that have those sources (for example, isort needs to cover I think using |
||
@classmethod | ||
def opt_out(cls, tgt: Target) -> bool: | ||
return tgt.get(SkipTerraformValidateField).value | ||
|
||
|
||
class TerraformCheckRequest(CheckRequest): | ||
field_set_type = TerraformDeploymentFieldSet | ||
field_set_type = TerraformValidateFieldSet | ||
tool_name = TerraformValidateSubsystem.options_scope | ||
|
||
|
||
def terraform_fieldset_to_init_request( | ||
terraform_fieldset: Union[TerraformDeploymentFieldSet, TerraformFieldSet] | ||
) -> TerraformInitRequest: | ||
if isinstance(terraform_fieldset, TerraformDeploymentFieldSet): | ||
deployment = terraform_fieldset | ||
return TerraformInitRequest( | ||
deployment.root_module, deployment.backend_config, deployment.dependencies | ||
) | ||
if isinstance(terraform_fieldset, TerraformFieldSet): | ||
module = terraform_fieldset | ||
return TerraformInitRequest( | ||
TerraformRootModuleField(module.address.spec, module.address), | ||
TerraformBackendConfigField(None, module.address), | ||
module.dependencies, | ||
) | ||
|
||
|
||
@rule | ||
async def terraform_check( | ||
request: TerraformCheckRequest, subsystem: TerraformValidateSubsystem | ||
|
@@ -36,9 +77,8 @@ async def terraform_check( | |
initialised_terraforms = await MultiGet( | ||
Get( | ||
TerraformInitResponse, | ||
TerraformInitRequest( | ||
deployment.root_module, deployment.backend_config, deployment.dependencies | ||
), | ||
TerraformInitRequest, | ||
terraform_fieldset_to_init_request(deployment), | ||
) | ||
for deployment in request.field_sets | ||
) | ||
|
@@ -71,5 +111,7 @@ async def terraform_check( | |
def rules(): | ||
return ( | ||
*collect_rules(), | ||
TerraformDeploymentTarget.register_plugin_field(SkipTerraformValidateField), | ||
TerraformModuleTarget.register_plugin_field(SkipTerraformValidateField), | ||
UnionRule(CheckRequest, TerraformCheckRequest), | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding docs!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm hoping that we get some feedback once people can see that it exists haha