Internal RFC: Acceptance test generation

[ewbankkit]

Description

In theory we can generate acceptance tests based on the CloudAPI schema definition in order to fully validate the behavior of the generated provider and protect against both local and upstream regression.

The terraform-plugin-framework does not currently provide testing support, however as the existing SDK relies on binary testing it should be possible to use the terraform-plugin-sdk to enable these tests to be run.

However, given the generated nature of the provider, and that the generation code is really the subject under test we may choose to rely mainly on unit tests. The detailed per-attribute, full lifecycle tests that we do for the existing provider, may not be appropriate given the vast range of services on offer from AWS and that the provider code is meant to be generic across services.

Definition of Done

• We understand whether the terraform-plugin-sdk will allow us to test this provider.
  That we have in Internal RFC (with team alignment) which specifies:
• Our testing strategy for the generated provider
• If we choose to pursue generated acceptance tests, an agreed scope for 0.1, 0.2 and 1.0 release.

[ewbankkit]

Following the merge of #79. We should now be able to generate a _basic and _disappears test for every resource type that has No required attributes.
Detecting this should be possible during Terraform schema code generation.

For those resource types that have required attributes the generated _basic tests is:

func TestAccAWSAppMeshVirtualService_basic(t *testing.T) {
	data := acctest.NewTestData(t, "AWS::AppMesh::VirtualService", "aws_appmesh_virtual_service", "test")
	r := virtualServiceTest{}

	data.ResourceTest(t, []resource.TestStep{
		{
			Config: r.basic(data),

			ExpectError: regexp.MustCompile(`Missing required argument`),
		},
	})
}

[breathingdust]

Internal RFC and review complete.