Added support for Datastream service & for ConnectionProfile resource #6479

melinath · 2022-08-29T19:00:48Z

First part of hashicorp/terraform-provider-google#10810

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

google_datastream_connection_profile

modular-magician · 2022-08-29T19:12:38Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 2099 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 8 files changed, 2099 insertions(+), 2 deletions(-))
TF Validator: Diff ( 4 files changed, 511 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 229 insertions(+))

modular-magician · 2022-08-29T20:17:34Z

Tests analytics

Total tests: 2152
Passed tests 1906
Skipped tests: 233
Failed tests: 13

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccDatastreamConnectionProfile_update|TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample|TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample|TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample|TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample|TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccCloudFunctions2Function_fullUpdate|TestAccSqlDatabaseInstance_SqlServerAuditConfig|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange

modular-magician · 2022-08-29T20:33:26Z

Tests passed during RECORDING mode:
TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample[view]
TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample[view]
TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample[view]
TestAccCloudFunctions2Function_fullUpdate[view]
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view]

Tests failed during RECORDING mode:
TestAccDatastreamConnectionProfile_update[view]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample[view]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample[view]
TestAccComputeInstance_soleTenantNodeAffinities[view]
TestAccComputeGlobalForwardingRule_internalLoadBalancing[view]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[view]
TestAccSqlDatabaseInstance_SqlServerAuditConfig[view]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange[view]

Please fix these to complete your PR
View the build log or the debug log for each test

mmv1/products/datastream/api.yaml

slevenick · 2022-08-29T20:52:19Z

mmv1/products/datastream/api.yaml

+      A set of reusable connection configurations to be used as a source or destination for a stream.
+    properties:
+      - !ruby/object:Api::Type::String
+        name: 'name'


I tend to prefer renaming connection_profile_id -> name and omitting the output-only name variable because it should be identical to the resource's id field, but there is precedence for both ways so I won't block on that

In the absence of a right way - I'd rather leave it called connection_profile_id because that matches the API field. would it make sense to just remove support for the name field since it duplicates id?

mmv1/products/datastream/terraform.yaml

mmv1/third_party/terraform/tests/resource_datastream_connection_profile_test.go

modular-magician · 2022-08-30T17:34:09Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 2178 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 8 files changed, 2178 insertions(+), 2 deletions(-))
TF Validator: Diff ( 4 files changed, 511 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 229 insertions(+))

modular-magician · 2022-08-30T18:23:49Z

Tests analytics

Total tests: 2154
Passed tests 1912
Skipped tests: 234
Failed tests: 8

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccSqlDatabaseInstance_SqlServerAuditConfig|TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample|TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample|TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccCloudRunService_cloudRunServiceStaticOutboundExample

modular-magician · 2022-08-30T18:31:27Z

Tests failed during RECORDING mode:
TestAccSqlDatabaseInstance_SqlServerAuditConfig[Error message] [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange[Error message] [Debug log]
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[Error message] [Debug log]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample[Error message] [Debug log]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample[Error message] [Debug log]
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_internalLoadBalancing[Error message] [Debug log]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

modular-magician · 2022-08-30T22:56:32Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 2152 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 8 files changed, 2152 insertions(+), 2 deletions(-))
TF Validator: Diff ( 4 files changed, 511 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 229 insertions(+))

modular-magician · 2022-08-30T23:50:10Z

Tests analytics

Total tests: 2154
Passed tests 1911
Skipped tests: 234
Failed tests: 9

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccSqlDatabaseInstance_SqlServerAuditConfig|TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample|TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange|TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccBigQueryDataTable_bigtable|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccComputeInstance_soleTenantNodeAffinities

modular-magician · 2022-08-30T23:57:58Z

Tests passed during RECORDING mode:
TestAccBigQueryDataTable_bigtable[Debug log]

Tests failed during RECORDING mode:
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[Error message] [Debug log]
TestAccSqlDatabaseInstance_SqlServerAuditConfig[Error message] [Debug log]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample[Error message] [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange[Error message] [Debug log]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_internalLoadBalancing[Error message] [Debug log]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[Error message] [Debug log]
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

trentkgUjet · 2022-08-31T15:54:59Z

Would it be possible to add the private connection resource to this PR as well? Otherwise, in order to use this feature Ujet would need to expose our cloudsql instance with a public IP, which compliance won't let us do.

I'm sure other businesses are in the same boat!

melinath · 2022-08-31T16:30:21Z

@trentkgUjet I'm going to get to PrivateConnection and Stream as well. However, to keep review simple, there will only be one resource per PR.

melinath · 2022-08-31T16:30:31Z

cc @iperetz-goo

modular-magician · 2022-08-31T17:05:23Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 2244 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 8 files changed, 2244 insertions(+), 2 deletions(-))
TF Validator: Diff ( 4 files changed, 511 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 229 insertions(+))

modular-magician · 2022-08-31T18:07:04Z

Tests analytics

Total tests: 2154
Passed tests 1911
Skipped tests: 234
Failed tests: 9

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample|TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample|TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccCGCSnippet_eventarcWorkflowsExample|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccSqlDatabaseInstance_SqlServerAuditConfig|TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange

modular-magician · 2022-08-31T18:21:12Z

Tests passed during RECORDING mode:
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[Debug log]

Tests failed during RECORDING mode:
TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample[Error message] [Debug log]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample[Error message] [Debug log]
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_internalLoadBalancing[Error message] [Debug log]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[Error message] [Debug log]
TestAccCGCSnippet_eventarcWorkflowsExample[Error message] [Debug log]
TestAccSqlDatabaseInstance_SqlServerAuditConfig[Error message] [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

slevenick · 2022-09-01T19:53:47Z

/gcbrun enabled the API

modular-magician · 2022-09-01T20:06:35Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 2244 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 8 files changed, 2244 insertions(+), 2 deletions(-))
TF Validator: Diff ( 4 files changed, 511 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 229 insertions(+))

Async definitions are used to figure out whether to call async code even if the async code isn't autogenerated

modular-magician · 2022-09-01T20:51:59Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 2280 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 8 files changed, 2280 insertions(+), 2 deletions(-))
TF Validator: Diff ( 4 files changed, 511 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 229 insertions(+))

modular-magician · 2022-09-01T21:16:44Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 2280 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 8 files changed, 2280 insertions(+), 2 deletions(-))
TF Validator: Diff ( 4 files changed, 511 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 229 insertions(+))

modular-magician · 2022-09-01T22:11:00Z

Tests analytics

Total tests: 2159
Passed tests 1915
Skipped tests: 235
Failed tests: 9

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange|TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample|TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample|TestAccSqlDatabaseInstance_SqlServerAuditConfig|TestAccFirebaserulesRelease_BasicRelease|TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccCloudRunService_cloudRunServiceStaticOutboundExample

modular-magician · 2022-09-01T22:25:48Z

Tests passed during RECORDING mode:
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[Debug log]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileFullExample[Debug log]
TestAccDatastreamConnectionProfile_datastreamConnectionProfileBasicExample[Debug log]
TestAccFirebaserulesRelease_BasicRelease[Debug log]

Tests failed during RECORDING mode:
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange[Error message] [Debug log]
TestAccSqlDatabaseInstance_SqlServerAuditConfig[Error message] [Debug log]
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_internalLoadBalancing[Error message] [Debug log]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

…GoogleCloudPlatform#6479) * Added datastream connection profile * Updated resource and tests to get to passing basic/full tests * Added update test * Corrected license dates * Moved url params into parameters array * Removed long timeouts * Ensured that update test actually tests update * Made update test move from gcs bucket to postgresql database instance * Switched to handwritten operation due to unexpected error structure * Added authorized networks to sql database instance * fixed whitespace * Re-added async definitions Async definitions are used to figure out whether to call async code even if the async code isn't autogenerated * Added postgresql password to importstateverifyignore

melinath added 3 commits August 26, 2022 13:51

Added datastream connection profile

5685a85

Updated resource and tests to get to passing basic/full tests

59ae10b

Added update test

d5b0dbf

melinath requested review from a team and ScottSuarez and removed request for a team August 29, 2022 19:03

melinath requested review from a team and slevenick and removed request for ScottSuarez and a team August 29, 2022 20:35

slevenick reviewed Aug 29, 2022

View reviewed changes

melinath added 5 commits August 30, 2022 09:14

Corrected license dates

d74bf48

Moved url params into parameters array

5266ca2

Removed long timeouts

d7cdfd3

Ensured that update test actually tests update

b5a58a6

Made update test move from gcs bucket to postgresql database instance

065f680

Switched to handwritten operation due to unexpected error structure

1163d50

Added authorized networks to sql database instance

b90ea5e

fixed whitespace

0c0883b

Re-added async definitions

799ad33

Async definitions are used to figure out whether to call async code even if the async code isn't autogenerated

Added postgresql password to importstateverifyignore

8ace307

melinath requested a review from slevenick September 1, 2022 22:41

slevenick approved these changes Sep 2, 2022

View reviewed changes

melinath merged commit ad14bfc into main Sep 6, 2022

melinath deleted the b-233804820 branch September 6, 2022 16:37

melinath mentioned this pull request Sep 16, 2022

Add new Resource Cryptokeyversions #6553

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support for Datastream service & for ConnectionProfile resource #6479

Added support for Datastream service & for ConnectionProfile resource #6479

melinath commented Aug 29, 2022

modular-magician commented Aug 29, 2022

modular-magician commented Aug 29, 2022

modular-magician commented Aug 29, 2022

slevenick Aug 29, 2022

melinath Aug 30, 2022

modular-magician commented Aug 30, 2022

modular-magician commented Aug 30, 2022

modular-magician commented Aug 30, 2022

modular-magician commented Aug 30, 2022

modular-magician commented Aug 30, 2022

modular-magician commented Aug 30, 2022

trentkgUjet commented Aug 31, 2022

melinath commented Aug 31, 2022

melinath commented Aug 31, 2022

modular-magician commented Aug 31, 2022

modular-magician commented Aug 31, 2022

modular-magician commented Aug 31, 2022

slevenick commented Sep 1, 2022

modular-magician commented Sep 1, 2022

modular-magician commented Sep 1, 2022

modular-magician commented Sep 1, 2022

modular-magician commented Sep 1, 2022

modular-magician commented Sep 1, 2022

Added support for Datastream service & for ConnectionProfile resource #6479

Added support for Datastream service & for ConnectionProfile resource #6479

Conversation

melinath commented Aug 29, 2022

modular-magician commented Aug 29, 2022

Diff report:

modular-magician commented Aug 29, 2022

Tests analytics

Action taken

modular-magician commented Aug 29, 2022

slevenick Aug 29, 2022

Choose a reason for hiding this comment

melinath Aug 30, 2022

Choose a reason for hiding this comment

modular-magician commented Aug 30, 2022

Diff report:

modular-magician commented Aug 30, 2022

Tests analytics

Action taken

modular-magician commented Aug 30, 2022

modular-magician commented Aug 30, 2022

Diff report:

modular-magician commented Aug 30, 2022

Tests analytics

Action taken

modular-magician commented Aug 30, 2022

trentkgUjet commented Aug 31, 2022

melinath commented Aug 31, 2022

melinath commented Aug 31, 2022

modular-magician commented Aug 31, 2022

Diff report:

modular-magician commented Aug 31, 2022

Tests analytics

Action taken

modular-magician commented Aug 31, 2022

slevenick commented Sep 1, 2022

modular-magician commented Sep 1, 2022

Diff report:

modular-magician commented Sep 1, 2022

Diff report:

modular-magician commented Sep 1, 2022

Diff report:

modular-magician commented Sep 1, 2022

Tests analytics

Action taken

modular-magician commented Sep 1, 2022