Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[datadog_integration_gcp_sts] Add Account tags, ResourceCollectionEnabled and IsSecurityCommandCenterEnabled support to gcp tf module #2134

Merged
merged 5 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
104 changes: 61 additions & 43 deletions datadog/fwprovider/resource_datadog_integration_gcp_sts.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ package fwprovider
import (
"context"

"github.com/hashicorp/terraform-plugin-framework/resource/schema/booldefault"

"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
"github.com/hashicorp/terraform-plugin-framework/diag"
frameworkPath "github.com/hashicorp/terraform-plugin-framework/path"
Expand All @@ -26,12 +28,15 @@ type integrationGcpStsResource struct {
}

type integrationGcpStsModel struct {
ID types.String `tfsdk:"id"`
Automute types.Bool `tfsdk:"automute"`
ClientEmail types.String `tfsdk:"client_email"`
DelegateAccountEmail types.String `tfsdk:"delegate_account_email"`
IsCspmEnabled types.Bool `tfsdk:"is_cspm_enabled"`
HostFilters types.Set `tfsdk:"host_filters"`
ID types.String `tfsdk:"id"`
AccountTags types.Set `tfsdk:"account_tags"`
Automute types.Bool `tfsdk:"automute"`
ClientEmail types.String `tfsdk:"client_email"`
DelegateAccountEmail types.String `tfsdk:"delegate_account_email"`
HostFilters types.Set `tfsdk:"host_filters"`
IsCspmEnabled types.Bool `tfsdk:"is_cspm_enabled"`
IsSecurityCommandCenterEnabled types.Bool `tfsdk:"is_security_command_center_enabled"`
ResourceCollectionEnabled types.Bool `tfsdk:"resource_collection_enabled"`
Copy link

@ash-ddog ash-ddog Mar 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • What is DelegateAccountEmail vs ClientEmail ? My guess is DelegateAccountEmail == client_id and ClientEmail == client_email
    • Update: read the descriptions below, but still, not sure how DelegateAccountEmail can have any affect on the API call we make since it's not a request parameter?
  • We're missing cloud_run_revision_filters here, not sure if that's on purpose or not.

Copy link

@ash-ddog ash-ddog Mar 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I'm sorry, thought this PR was for our V1 API but it's V2.

But still the comments above are valid.

Copy link
Contributor

@smuhit smuhit Mar 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DelegateAccountEmail is the STS delegate. (Basically, the principal that needs the token creator role on the client Email).

In this schema, it's a computed value that this provider returns (and not as input), mainly to allow setup purely through terraform.

Basically:

  • terraform to create GCP service account (including permissions)
  • terraform using this module to configure the datadog gcp sts integration (using output from previous)
  • terraform to give output of integration (second bullet) the token creator role to the service account in first bullet

Let me know if you have questions

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have an upcoming task to add cloud_run_revision_filters.

}

func NewIntegrationGcpStsResource() resource.Resource {
Expand All @@ -52,6 +57,11 @@ func (r *integrationGcpStsResource) Schema(_ context.Context, _ resource.SchemaR
response.Schema = schema.Schema{
Description: "Provides a Datadog Integration GCP Sts resource. This can be used to create and manage Datadog - Google Cloud Platform integration.",
Attributes: map[string]schema.Attribute{
"account_tags": schema.SetAttribute{
Optional: true,
Description: "Tags to be associated with GCP metrics and service checks from your account.",
ElementType: types.StringType,
ash-ddog marked this conversation as resolved.
Show resolved Hide resolved
},
"automute": schema.BoolAttribute{
Optional: true,
Computed: true,
Expand All @@ -71,17 +81,27 @@ func (r *integrationGcpStsResource) Schema(_ context.Context, _ resource.SchemaR
stringplanmodifier.UseStateForUnknown(),
},
},
"host_filters": schema.SetAttribute{
Optional: true,
Description: "Your Host Filters.",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kinda nit: might want to be more specific about what this actually is? maybe Filters to be applied for GCE instance resources or something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just shuffled it (to be ordered alphabetically).. I didn't actually add this description (It exists currently: https://registry.terraform.io/providers/DataDog/datadog/latest/docs/resources/integration_gcp_sts#host_filters)

We can update it, but can tackle in a separate PR

ElementType: types.StringType,
},
"is_cspm_enabled": schema.BoolAttribute{
Optional: true,
Computed: true,
Description: "When enabled, Datadog performs configuration checks across your Google Cloud environment by continuously scanning every resource, which may incur additional charges.",
Description: "Whether Datadog collects cloud security posture management resources from your GCP project. If enabled, requires `resource_collection_enabled` to also be enabled.",
},
"host_filters": schema.SetAttribute{
"is_security_command_center_enabled": schema.BoolAttribute{
Description: "When enabled, Datadog will attempt to collect Security Command Center Findings. Note: This requires additional permissions on the service account.",
Optional: true,
Description: "Your Host Filters.",
ElementType: types.StringType,
Computed: true,
Default: booldefault.StaticBool(false),
},
"id": utils.ResourceIDAttribute(),
"resource_collection_enabled": schema.BoolAttribute{
Description: "When enabled, Datadog scans for all resources in your GCP environment.",
Optional: true,
Computed: true,
}, "id": utils.ResourceIDAttribute(),
},
}
}
Expand Down Expand Up @@ -148,7 +168,10 @@ func (r *integrationGcpStsResource) Create(ctx context.Context, request resource
delegateEmail := delegateResponse.Data.Attributes.GetDelegateAccountEmail()
state.DelegateAccountEmail = types.StringValue(delegateEmail)

body, diags := r.buildIntegrationGcpStsRequestBody(ctx, &state)
attributes, diags := r.buildIntegrationGcpStsRequestBody(ctx, &state)
body := datadogV2.NewGCPSTSServiceAccountCreateRequestWithDefaults()
body.Data = datadogV2.NewGCPSTSServiceAccountDataWithDefaults()
body.Data.SetAttributes(attributes)
response.Diagnostics.Append(diags...)
if response.Diagnostics.HasError() {
return
Expand Down Expand Up @@ -178,7 +201,11 @@ func (r *integrationGcpStsResource) Update(ctx context.Context, request resource

id := state.ID.ValueString()

body, diags := r.buildIntegrationGcpStsUpdateRequestBody(ctx, &state)
attributes, diags := r.buildIntegrationGcpStsRequestBody(ctx, &state)
body := datadogV2.NewGCPSTSServiceAccountUpdateRequestWithDefaults()
body.Data = datadogV2.NewGCPSTSServiceAccountUpdateRequestDataWithDefaults()
body.Data.SetAttributes(attributes)

response.Diagnostics.Append(diags...)
if response.Diagnostics.HasError() {
return
Expand Down Expand Up @@ -222,6 +249,9 @@ func (r *integrationGcpStsResource) updateState(ctx context.Context, state *inte
state.ID = types.StringValue(resp.GetId())

attributes := resp.GetAttributes()
if accountTags, ok := attributes.GetAccountTagsOk(); ok && len(*accountTags) > 0 {
state.AccountTags, _ = types.SetValueFrom(ctx, types.StringType, *accountTags)
}
if automute, ok := attributes.GetAutomuteOk(); ok {
state.Automute = types.BoolValue(*automute)
}
Expand All @@ -234,39 +264,24 @@ func (r *integrationGcpStsResource) updateState(ctx context.Context, state *inte
if isCspmEnabled, ok := attributes.GetIsCspmEnabledOk(); ok {
state.IsCspmEnabled = types.BoolValue(*isCspmEnabled)
}
}

func (r *integrationGcpStsResource) buildIntegrationGcpStsRequestBody(ctx context.Context, state *integrationGcpStsModel) (*datadogV2.GCPSTSServiceAccountCreateRequest, diag.Diagnostics) {
diags := diag.Diagnostics{}
attributes := datadogV2.GCPSTSServiceAccountAttributes{}

if !state.Automute.IsNull() {
attributes.SetAutomute(state.Automute.ValueBool())
}
if !state.ClientEmail.IsNull() {
attributes.SetClientEmail(state.ClientEmail.ValueString())
if isSecurityCommandCenterEnabled, ok := attributes.GetIsSecurityCommandCenterEnabledOk(); ok {
state.IsSecurityCommandCenterEnabled = types.BoolValue(*isSecurityCommandCenterEnabled)
}
if !state.IsCspmEnabled.IsNull() {
attributes.SetIsCspmEnabled(state.IsCspmEnabled.ValueBool())
}

hostFilters := make([]string, 0)
if !state.HostFilters.IsNull() {
diags.Append(state.HostFilters.ElementsAs(ctx, &hostFilters, false)...)
if resourceCollectionEnabled, ok := attributes.GetResourceCollectionEnabledOk(); ok {
state.ResourceCollectionEnabled = types.BoolValue(*resourceCollectionEnabled)
}
attributes.SetHostFilters(hostFilters)

req := datadogV2.NewGCPSTSServiceAccountCreateRequestWithDefaults()
req.Data = datadogV2.NewGCPSTSServiceAccountDataWithDefaults()
req.Data.SetAttributes(attributes)

return req, diags
}

func (r *integrationGcpStsResource) buildIntegrationGcpStsUpdateRequestBody(ctx context.Context, state *integrationGcpStsModel) (*datadogV2.GCPSTSServiceAccountUpdateRequest, diag.Diagnostics) {
func (r *integrationGcpStsResource) buildIntegrationGcpStsRequestBody(ctx context.Context, state *integrationGcpStsModel) (datadogV2.GCPSTSServiceAccountAttributes, diag.Diagnostics) {
diags := diag.Diagnostics{}
attributes := datadogV2.GCPSTSServiceAccountAttributes{}

accountTags := make([]string, 0)
if !state.AccountTags.IsNull() {
diags.Append(state.AccountTags.ElementsAs(ctx, &accountTags, false)...)
}
attributes.SetAccountTags(accountTags)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments apply here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be doing something like the IsUnknown() for AccountTags (like we're doing below) since it's also a new field? Not familiar enough with terraform to know what's right here.


if !state.Automute.IsNull() {
attributes.SetAutomute(state.Automute.ValueBool())
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this stuff is super duplicative. Is it worth refactoring like this?

func (r *integrationGcpStsResource) buildIntegrationGcpStsCreateRequest(ctx context.Context, state *integrationGcpStsModel) (*datadogV2.GCPSTSServiceAccountCreateRequest, diag.Diagnostics) {
	attrs, diags := r.buildIntegrationGcpStsAccountAttributes(ctx, state)
	req := datadogV2.NewGCPSTSServiceAccountCreateRequestWithDefaults()
	req.Data = datadogV2.NewGCPSTSServiceAccountDataWithDefaults()
	req.Data.SetAttributes(*attrs)
	return req, diags
}

func (r *integrationGcpStsResource) buildIntegrationGcpStsUpdateRequest(ctx context.Context, state *integrationGcpStsModel) (*datadogV2.GCPSTSServiceAccountUpdateRequest, diag.Diagnostics) {
	attrs, diags := r.buildIntegrationGcpStsAccountAttributes(ctx, state)
	req := datadogV2.NewGCPSTSServiceAccountUpdateRequestWithDefaults()
	req.Data = datadogV2.NewGCPSTSServiceAccountUpdateRequestDataWithDefaults()
	req.Data.SetAttributes(*attrs)
	return req, diags
}

func (r *integrationGcpStsResource) buildIntegrationGcpStsAccountAttributes(ctx context.Context, state *integrationGcpStsModel) (*datadogV2.GCPSTSServiceAccountAttributes, diag.Diagnostics) {
	diags := diag.Diagnostics{}
	attributes := datadogV2.NewGCPSTSServiceAccountAttributesWithDefaults()

	accountTags := make([]string, 0)
	if !state.AccountTags.IsNull() {
		diags.Append(state.AccountTags.ElementsAs(ctx, &accountTags, false)...)
	}
	attributes.SetAccountTags(accountTags)

	if !state.Automute.IsNull() {
		attributes.SetAutomute(state.Automute.ValueBool())
	}
	if !state.ClientEmail.IsNull() {
		attributes.SetClientEmail(state.ClientEmail.ValueString())
	}
	if !state.IsCspmEnabled.IsNull() {
		attributes.SetIsCspmEnabled(state.IsCspmEnabled.ValueBool())
	}

	hostFilters := make([]string, 0)
	if !state.HostFilters.IsNull() {
		diags.Append(state.HostFilters.ElementsAs(ctx, &hostFilters, false)...)
	}
	attributes.SetHostFilters(hostFilters)

	return attributes, diags
}

Weird thing is that I'm pretty sure that the update API does not support updating ClientEmail, but it was already in the update body before these changes 🥴 .

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO that's pertinent but maybe an a separated PR?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we mostly took this (thanks), but still we have ClientEmail in the update API when that's not supported. Not blocking, but could confuse someone in the future.

Copy link
Contributor

@smuhit smuhit Mar 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.. But didn't want to make functionally different changes (removal of ClientEmail in the Update call) in a PR that is essentially additive changes.

Expand All @@ -283,9 +298,12 @@ func (r *integrationGcpStsResource) buildIntegrationGcpStsUpdateRequestBody(ctx
}
attributes.SetHostFilters(hostFilters)

req := datadogV2.NewGCPSTSServiceAccountUpdateRequestWithDefaults()
req.Data = datadogV2.NewGCPSTSServiceAccountUpdateRequestDataWithDefaults()
req.Data.SetAttributes(attributes)
if !state.IsSecurityCommandCenterEnabled.IsUnknown() {
attributes.SetIsSecurityCommandCenterEnabled(state.IsSecurityCommandCenterEnabled.ValueBool())
}
if !state.ResourceCollectionEnabled.IsUnknown() {
attributes.SetResourceCollectionEnabled(state.ResourceCollectionEnabled.ValueBool())
}

return req, diags
return attributes, diags
}
18 changes: 9 additions & 9 deletions datadog/tests/cassettes/TestAccIntegrationGcpStsBasic.yaml

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ interactions:
remote_addr: ""
request_uri: ""
body: |
{"data":{"attributes":{"automute":true,"client_email":"tf-TestAccIntegrationGcpStsDefault-local-1704918354@test-project.iam.gserviceaccount.com","host_filters":[],"is_cspm_enabled":false},"type":"gcp_service_account"}}
{"data":{"attributes":{"account_tags":[],"automute":true,"client_email":"tf-TestAccIntegrationGcpStsDefault-local-1704918354@test-project.iam.gserviceaccount.com","host_filters":[],"is_cspm_enabled":false,"is_security_command_center_enabled":false},"type":"gcp_service_account"}}
form: {}
headers:
Accept:
Expand Down
21 changes: 21 additions & 0 deletions datadog/tests/resource_datadog_integration_gcp_sts_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,20 @@ func TestAccIntegrationGcpStsBasic(t *testing.T) {
"datadog_integration_gcp_sts.foo", "client_email", fmt.Sprintf("%s@test-project.iam.gserviceaccount.com", uniq)),
resource.TestCheckResourceAttr(
"datadog_integration_gcp_sts.foo", "is_cspm_enabled", "false"),
resource.TestCheckResourceAttr(
"datadog_integration_gcp_sts.foo", "is_security_command_center_enabled", "false"),
resource.TestCheckResourceAttr(
"datadog_integration_gcp_sts.foo", "resource_collection_enabled", "false"),
resource.TestCheckTypeSetElemAttr(
"datadog_integration_gcp_sts.foo", "host_filters.*", "tag:one"),
resource.TestCheckTypeSetElemAttr(
"datadog_integration_gcp_sts.foo", "host_filters.*", "tag:two"),
resource.TestCheckTypeSetElemAttr(
"datadog_integration_gcp_sts.foo", "account_tags.*", "a:tag"),
resource.TestCheckTypeSetElemAttr(
"datadog_integration_gcp_sts.foo", "account_tags.*", "another:one"),
resource.TestCheckTypeSetElemAttr(
"datadog_integration_gcp_sts.foo", "account_tags.*", "and:another"),
),
},
{
Expand All @@ -47,8 +57,14 @@ func TestAccIntegrationGcpStsBasic(t *testing.T) {
"datadog_integration_gcp_sts.foo", "client_email", fmt.Sprintf("%s@test-project.iam.gserviceaccount.com", uniq)),
resource.TestCheckResourceAttr(
"datadog_integration_gcp_sts.foo", "is_cspm_enabled", "true"),
resource.TestCheckResourceAttr(
"datadog_integration_gcp_sts.foo", "is_security_command_center_enabled", "true"),
resource.TestCheckResourceAttr(
"datadog_integration_gcp_sts.foo", "resource_collection_enabled", "true"),
resource.TestCheckNoResourceAttr(
"datadog_integration_gcp_sts.foo", "host_filters"),
resource.TestCheckNoResourceAttr(
"datadog_integration_gcp_sts.foo", "account_tags"),
),
},
},
Expand Down Expand Up @@ -89,6 +105,9 @@ resource "datadog_integration_gcp_sts" "foo" {
client_email = "%s@test-project.iam.gserviceaccount.com"
host_filters = ["tag:one", "tag:two"]
is_cspm_enabled = "false"
resource_collection_enabled = "false"
is_security_command_center_enabled = "false"
account_tags = ["a:tag", "another:one", "and:another"]
}`, uniq)
}

Expand All @@ -98,6 +117,8 @@ resource "datadog_integration_gcp_sts" "foo" {
automute = "true"
client_email = "%s@test-project.iam.gserviceaccount.com"
is_cspm_enabled = "true"
resource_collection_enabled = "true"
is_security_command_center_enabled = "true"
}`, uniq)
}

Expand Down
5 changes: 4 additions & 1 deletion docs/resources/integration_gcp_sts.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,12 @@ resource "datadog_integration_gcp_sts" "foo" {

### Optional

- `account_tags` (Set of String) Tags to be associated with GCP metrics and service checks from your account.
- `automute` (Boolean) Silence monitors for expected GCE instance shutdowns.
- `host_filters` (Set of String) Your Host Filters.
- `is_cspm_enabled` (Boolean) When enabled, Datadog performs configuration checks across your Google Cloud environment by continuously scanning every resource, which may incur additional charges.
- `is_cspm_enabled` (Boolean) Whether Datadog collects cloud security posture management resources from your GCP project. If enabled, requires `resource_collection_enabled` to also be enabled.
- `is_security_command_center_enabled` (Boolean) When enabled, Datadog will attempt to collect Security Command Center Findings. Note: This requires additional permissions on the service account. Defaults to `false`.
- `resource_collection_enabled` (Boolean) When enabled, Datadog scans for all resources in your GCP environment.

### Read-Only

Expand Down