Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync main feature branch provider functions #10291

Conversation

SarahFrench
Copy link
Collaborator

Release Note Template for Downstream PRs (will be copied)


kumadee and others added 30 commits February 20, 2024 19:53
* fixes: permadiff issue if event trigger region is not specified

- fixes hashicorp/terraform-provider-google#17161

* chore: Updated cloudfunction2 test with no explicit trigger_region
* Fix GKE front matter, bad field docs

* Update container_cluster.html.markdown
…_nat" (#9712)

* added default value for statis allocation

* setting the default from API
* add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress

* make build number optional

* regroup regex, cleaner comparison of versions

* correction
* resourceManagerTags added to Cluster Node Config schema

* update beta tag

* add cluster and node proto tests

* add expand and flatten proto

* removed beta tag

* added to documentation

* added resource manager tags to auto pilot

* migrating resourceManagerTags tests

* migrating node_pools test

* migrating additional tests

* minor fixes

* fixing tests

* add in-place update support

* fixed tests

* fixed annotations

* validated clusters and node pools tests. Isolated node pool auto config

* isolated resource manager tags from docs

* fixed permission issue

* fixed spaces

* fixed non determinism on tag keys

* removed auto_pilot rmts

* fixed time_sleep

* add depends_on to IAM policies
* Initial replication commit

* Cleanup work

- Renamed a lot of files to make clear which resource the belong to
- Updated documentation for resource fields
- Renamed a few resource fields and changed some types
- Disabled the custom code for now. Needs to be discussed first

* Update example file

* Updated example file

* Major updates

- Reorganisation of block
- Reorganisation of fields to match API documentation
- Updated example parameters
- Added missing API fields
- Improved descriptions
-

* For replication deletion, stop replication first

* Add support for deleting destination volume on replication delete

* Make volumes deletable in presence of snapshots.

This change will be PRed for volume resource independently. Adding it here while it is not in main.

* Improving debug error message

* yaml check and format fix

* Add wait for mirror to initialize.

Required to run destroy shortly after create.

* Wait on destroy, not on create

* Make deleting a replication more robust

- doc improvements
- started to implement stop/resume. More work required.
- renamed a few files to better reflect what they are good for

* adding support for stop/resume

* yamlformat and lint

* Add force delete to delete volumes with nested snapshots

* resource test first version

* More changes to make tests solid

- Introduced new parameter to wait for mirror_status==MIRRORED
- more mirror state reconciliation

* Test updates

* few cleanups

* Make virtual field verifies happy

* Minor test improvements

* More fine tuning

- Remove merge conflict in volume.yaml
- make generated test work
- make output field work
- ignore_read for virtual fields

* Resource name change as suggested by @slevenick

* Remove snapshot code block and fix typo

* Detect manual stop/resume actions

* Remove ignore_read for deletion_policy

* - Made destinationVolumeParameters immutable. It still requires ignore_read.
- removed ignore_read from virtual_fields

* destinationVolumeParameters are only evaluated at create. Make the immutable.

* Name cleanups and comment improvements

* removed comment

Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com>

* tabs to spaces in resource block

Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com>

* Updates to address review comments

- make wait_for_mirror also work for stop/resume, additionally to create
- convert tabs in test resource blocks to spaces
- fix typos

* Rewording of comments

Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com>

---------

Co-authored-by: G-NamanGupta <gnaman@netapp.com>
Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com>
* Ensured that beta runs in TeamCity use only beta paths

* Added tests for sweeper package path
* add description

* remove tail space
* Add location field in DNS authorization resource

* Fixed the provider version of the updated config

---------

Co-authored-by: Hamza Hassan <hamzahassan@google.com>
* block upgrade to composer 3

* make isComposer3 more generic, correct imageVersionChangeValidationFunc

* added validation for Composer 2/3 specific fields

* add tests for validation

* add checks in flattenComposerEnvironmentConfig

* Update attributes of fields not used in Composer 3

* make customizeDiff functions beta only

* remove Computed from gke_cluster

* remove Optional instead of Computed

* add envCfg.PrivateEnvironmentConfig is nil check

* modify isComposer3 to take string

* minor correction to avoid merge conflicts
…Proxy resource (#10011)

Co-authored-by: Hamza Hassan <hamzahassan@google.com>
Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied
* Add IAM resources for Cloud Deploy Target

* update

* add back import_format
* Add dataGovernanceType and remoteFunctionOptions to bigquery_routine

* add function-sources.zip to biguquery fixtures

* fix resource names in TestAccBigQueryRoutine

* add bigquery routine remote function example
* I'm on vaction :)

* ScottSuarez not vaction anymore

* ScottSuarez not vaction anymore
melinath and others added 15 commits March 22, 2024 13:00
* bq table - add geojson support

* bq table - add geojson support

* CR comments
pipelines for YAML lints are failing due to:
```
Traceback (most recent call last):
  File "/home/runner/.local/bin/yamllint", line 5, in <module>
    from yamllint.cli import run
  File "/home/runner/.local/lib/python3.10/site-packages/yamllint/cli.py", line 24, in <module>
    from yamllint.config import YamlLintConfig, YamlLintConfigError
  File "/home/runner/.local/lib/python3.10/site-packages/yamllint/config.py", line 19, in <module>
    import pathspec
ModuleNotFoundError: No module named 'pathspec'
```

A fresh install in a virtual environment using the following command:
```bash
$ pip install yamllint==1.32.0 pyyaml==6.0.1
$ pip freeze
pathspec==0.12.1
PyYAML==6.0.1
yamllint==1.32.0
```

Regression introduced by 60917c8
* remove pattern field

* fix lint yaml files
* add support for dataproc metastore scheduled backups

* CR comments

* CR comments

* CR comments

* CR comments

* CR comments

* Update mmv1/third_party/terraform/services/dataprocmetastore/resource_dataproc_metastore_service_test.go.erb

Co-authored-by: Zhenhua Li <zhenhuali@google.com>

* Update mmv1/third_party/terraform/services/dataprocmetastore/resource_dataproc_metastore_service_test.go.erb

Co-authored-by: Zhenhua Li <zhenhuali@google.com>

* CR comments

---------

Co-authored-by: Zhenhua Li <zhenhuali@google.com>
* Update default TF version used in TeamCity to be 1.8.0-rc1

This will be updated to 1.8.0 after the GA release

* Update pom.xml to pull in a hardcoded server-api dependency version : 2024.07-SNAPSHOT
Co-authored-by: Krishnan Gopal <krishgopal@google.com>
* fix test TestAccContainerCluster_withInvalidAutoscalingProfile

* fix test TestAccContainer*_withInvalid*
* Upgrade DCL to 1.64.0

* Fixed test randomness
Copy link

google-cla bot commented Mar 26, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 11 files changed, 384 insertions(+), 10 deletions(-))
google-beta provider: Diff ( 12 files changed, 391 insertions(+), 14 deletions(-))

@SarahFrench
Copy link
Collaborator Author

Waiting for VCR tests to complete before merging into the feature branch
I'll then, hopefully, not see new merge conflicts from the feature branch to main. After the VCR tests complete on that PR I'll merge to main.

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3541
Passed tests: 3098
Skipped tests: 366
Affected tests: 77

Click here to see the affected service packages
all service packages are affected

Action taken

Found 77 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccAlloydbCluster_continuousBackup_CMEKIsUpdatable|TestAccApphubWorkload_apphubWorkloadUpdate|TestAccArtifactRegistryRepository_artifactRegistryRepositoryCmekExample|TestAccBigQueryDataset_cmek|TestAccBigQueryTable_Kms|TestAccBinaryAuthorizationAttestor_kms|TestAccComposerEnvironment_withEncryptionConfigComposer1|TestAccComposerEnvironment_withEncryptionConfigComposer2|TestAccComputeDisk_encryptionKMS|TestAccComputeDisk_pdHyperDiskEnableConfidentialCompute|TestAccComputeImage_imageEncryptionKey|TestAccComputeInstanceTemplate_EncryptKMS|TestAccComputeInstanceTemplate_sourceImageEncryptionKey|TestAccComputeInstanceTemplate_sourceSnapshotEncryptionKey|TestAccComputeInstance_confidentialHyperDiskBootDisk|TestAccComputeInstance_kmsDiskEncryption|TestAccComputeRegionInstanceTemplate_sourceImageEncryptionKey|TestAccComputeRegionInstanceTemplate_sourceSnapshotEncryptionKey|TestAccContainerAnalysisOccurrence_basic|TestAccContainerAnalysisOccurrence_multipleSignatures|TestAccContainerCluster_nodeAutoprovisioningDefaultsBootDiskKmsKey|TestAccContainerCluster_withBootDiskKmsKey|TestAccContainerCluster_withConfidentialBootDisk|TestAccContainerCluster_withConfidentialBootDiskNodeConfig|TestAccContainerCluster_withDatabaseEncryption|TestAccContainerNodePool_withConfidentialBootDisk|TestAccDataKmsSecretCiphertext_basic|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_imageTransformationsUpdate|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformationsUpdate|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformations_primitiveTransformations|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformations_primitiveTransformations_bucketingConfig|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformations_primitiveTransformations_dateShiftConfig|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformations_primitiveTransformations_fixedSizeBucketingConfig|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformationsUpdate|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_cryptoDeterministicConfig|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_cryptoHashConfig|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_cryptoReplaceFfxFpeConfig|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_dateShiftConfig|TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_update|TestAccDataSourceGoogleKmsCryptoKeyVersion_basic|TestAccDataSourceGoogleKmsCryptoKey_basic|TestAccDataSourceGoogleKmsKeyRing_basic|TestAccDataSourceGoogleServiceAccountIdToken_impersonation|TestAccDataprocCluster_KMS|TestAccDataprocCluster_withKerberos|TestAccEventarcChannel_cryptoKeyUpdate|TestAccEventarcGoogleChannelConfig_cryptoKeyUpdate|TestAccEventarcTrigger_channel|TestAccFirebaseAppCheckServiceConfig_firebaseAppCheckServiceConfigUpdate|TestAccKmsSecretCiphertext_basic|TestAccLoggingFolderSettings_loggingFolderSettingsAllExample|TestAccLoggingFolderSettings_update|TestAccLoggingOrganizationSettings_update|TestAccLookerInstance_lookerInstanceEnterpriseFullTestExample|TestAccNotebooksInstance_notebookInstanceFullExample|TestAccPubsubTopic_cmek|TestAccSecretManagerSecret_automaticCmekUpdate|TestAccSecretManagerSecret_cmek|TestAccSecretManagerSecret_secretWithAutomaticCmekExample|TestAccSecretManagerSecret_userManagedCmekUpdate|TestAccStorageObjectKms|TestAccVertexAIEndpoint_vertexAiEndpointNetwork|TestAccVertexAIFeaturestoreEntitytypeIamBindingGenerated|TestAccVertexAIFeaturestoreEntitytypeIamMemberGenerated|TestAccVertexAIFeaturestoreEntitytypeIamPolicyGenerated|TestAccVertexAIFeaturestoreEntitytype_vertexAiFeaturestoreEntitytypeExample|TestAccVertexAIFeaturestoreEntitytype_vertexAiFeaturestoreEntitytypeWithBetaFieldsExample|TestAccVertexAIFeaturestoreIamBindingGenerated|TestAccVertexAIFeaturestoreIamMemberGenerated|TestAccVertexAIFeaturestoreIamPolicyGenerated|TestAccVertexAIFeaturestore_vertexAiFeaturestoreExample|TestAccVertexAIFeaturestore_vertexAiFeaturestoreScalingExample|TestAccVertexAIFeaturestore_vertexAiFeaturestoreWithBetaFieldsExample|TestAccVertexAIMetadataStore_vertexAiMetadataStoreExample|TestAccVertexAITensorboard_vertexAiTensorboardFullExample|TestAccWorkbenchInstance_workbenchInstanceFullExample|TestAccWorkflowsWorkflow_CMEK

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccApphubWorkload_apphubWorkloadUpdate[Debug log]
TestAccDataKmsSecretCiphertext_basic[Debug log]
TestAccDataSourceGoogleKmsCryptoKeyVersion_basic[Debug log]
TestAccDataSourceGoogleKmsCryptoKey_basic[Debug log]
TestAccDataSourceGoogleKmsKeyRing_basic[Debug log]
TestAccDataSourceGoogleServiceAccountIdToken_impersonation[Debug log]
TestAccFirebaseAppCheckServiceConfig_firebaseAppCheckServiceConfigUpdate[Debug log]
TestAccKmsSecretCiphertext_basic[Debug log]
TestAccLoggingFolderSettings_loggingFolderSettingsAllExample[Debug log]
TestAccLoggingFolderSettings_update[Debug log]
TestAccLookerInstance_lookerInstanceEnterpriseFullTestExample[Debug log]
TestAccNotebooksInstance_notebookInstanceFullExample[Debug log]
TestAccPubsubTopic_cmek[Debug log]
TestAccSecretManagerSecret_automaticCmekUpdate[Debug log]
TestAccSecretManagerSecret_cmek[Debug log]
TestAccSecretManagerSecret_secretWithAutomaticCmekExample[Debug log]
TestAccSecretManagerSecret_userManagedCmekUpdate[Debug log]
TestAccStorageObjectKms[Debug log]
TestAccVertexAIFeaturestoreEntitytypeIamBindingGenerated[Debug log]
TestAccVertexAIFeaturestoreEntitytypeIamMemberGenerated[Debug log]
TestAccVertexAIFeaturestoreEntitytypeIamPolicyGenerated[Debug log]
TestAccVertexAIFeaturestoreEntitytype_vertexAiFeaturestoreEntitytypeExample[Debug log]
TestAccVertexAIFeaturestoreEntitytype_vertexAiFeaturestoreEntitytypeWithBetaFieldsExample[Debug log]
TestAccVertexAIFeaturestoreIamBindingGenerated[Debug log]
TestAccVertexAIFeaturestoreIamMemberGenerated[Debug log]
TestAccVertexAIFeaturestoreIamPolicyGenerated[Debug log]
TestAccVertexAIFeaturestore_vertexAiFeaturestoreExample[Debug log]
TestAccVertexAIFeaturestore_vertexAiFeaturestoreWithBetaFieldsExample[Debug log]
TestAccVertexAIMetadataStore_vertexAiMetadataStoreExample[Debug log]
TestAccVertexAITensorboard_vertexAiTensorboardFullExample[Debug log]
TestAccWorkbenchInstance_workbenchInstanceFullExample[Debug log]
TestAccWorkflowsWorkflow_CMEK[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbCluster_continuousBackup_CMEKIsUpdatable[Error message] [Debug log]
TestAccArtifactRegistryRepository_artifactRegistryRepositoryCmekExample[Error message] [Debug log]
TestAccBigQueryDataset_cmek[Error message] [Debug log]
TestAccBigQueryTable_Kms[Error message] [Debug log]
TestAccBinaryAuthorizationAttestor_kms[Error message] [Debug log]
TestAccComposerEnvironment_withEncryptionConfigComposer1[Error message] [Debug log]
TestAccComposerEnvironment_withEncryptionConfigComposer2[Error message] [Debug log]
TestAccComputeDisk_encryptionKMS[Error message] [Debug log]
TestAccComputeDisk_pdHyperDiskEnableConfidentialCompute[Error message] [Debug log]
TestAccComputeImage_imageEncryptionKey[Error message] [Debug log]
TestAccComputeInstanceTemplate_EncryptKMS[Error message] [Debug log]
TestAccComputeInstanceTemplate_sourceImageEncryptionKey[Error message] [Debug log]
TestAccComputeInstanceTemplate_sourceSnapshotEncryptionKey[Error message] [Debug log]
TestAccComputeInstance_confidentialHyperDiskBootDisk[Error message] [Debug log]
TestAccComputeInstance_kmsDiskEncryption[Error message] [Debug log]
TestAccComputeRegionInstanceTemplate_sourceImageEncryptionKey[Error message] [Debug log]
TestAccComputeRegionInstanceTemplate_sourceSnapshotEncryptionKey[Error message] [Debug log]
TestAccContainerAnalysisOccurrence_basic[Error message] [Debug log]
TestAccContainerAnalysisOccurrence_multipleSignatures[Error message] [Debug log]
TestAccContainerCluster_nodeAutoprovisioningDefaultsBootDiskKmsKey[Error message] [Debug log]
TestAccContainerCluster_withBootDiskKmsKey[Error message] [Debug log]
TestAccContainerCluster_withConfidentialBootDisk[Error message] [Debug log]
TestAccContainerCluster_withConfidentialBootDiskNodeConfig[Error message] [Debug log]
TestAccContainerCluster_withDatabaseEncryption[Error message] [Debug log]
TestAccContainerNodePool_withConfidentialBootDisk[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_imageTransformationsUpdate[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformationsUpdate[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformations_primitiveTransformations[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformations_primitiveTransformations_bucketingConfig[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformations_primitiveTransformations_dateShiftConfig[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_infoTypeTransformations_primitiveTransformations_fixedSizeBucketingConfig[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformationsUpdate[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_cryptoDeterministicConfig[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_cryptoHashConfig[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_cryptoReplaceFfxFpeConfig[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_dateShiftConfig[Error message] [Debug log]
TestAccDataLossPreventionDeidentifyTemplate_dlpDeidentifyTemplate_recordTransformations_with_infoTypeTransformations_update[Error message] [Debug log]
TestAccDataprocCluster_KMS[Error message] [Debug log]
TestAccDataprocCluster_withKerberos[Error message] [Debug log]
TestAccEventarcChannel_cryptoKeyUpdate[Error message] [Debug log]
TestAccEventarcGoogleChannelConfig_cryptoKeyUpdate[Error message] [Debug log]
TestAccEventarcTrigger_channel[Error message] [Debug log]
TestAccLoggingOrganizationSettings_update[Error message] [Debug log]
TestAccVertexAIEndpoint_vertexAiEndpointNetwork[Error message] [Debug log]
TestAccVertexAIFeaturestore_vertexAiFeaturestoreScalingExample[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

@SarahFrench
Copy link
Collaborator Author

After discussion with @shuya I'm going ahead with merge. The failures above aren't due to code changes in this PR/the feature branch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment