protoc-gen-go-aip-test

Generate test suites for protobuf services implementing standard AIP methods.

The generated test suites are based on guidance for standard methods, and experience from implementing these methods in practice. See Suites for a list of the generated tests.

Experimental: This plugin is experimental, and breaking changes with regard to the generated tests suites should be expected.

Usage

Step 1: Declare a service with AIP standard methods

service FreightService {
  // Get a shipper.
  // See: https://google.aip.dev/131 (Standard methods: Get).
  rpc GetShipper(GetShipperRequest) returns (Shipper) {
    option (google.api.http) = {
      get: "/v1/{name=shippers/*}"
    };
    option (google.api.method_signature) = "name";
  }

  // ...
}

Step 2: Install the generator

Either install using go install:

go install github.com/einride/protoc-gen-go-aip-test@latest

Or download a prebuilt binary from releases and put it in your PATH.

The generator can also be built from source using Go.

Step 3: Generate test suites

Include the plugin in protoc invocation

protoc
  --go-aip-test_out=[OUTPUT DIR] \
  --go-aip-test_opt=module=[OUTPUT MODULE] \
  [.proto files ...]

This can also be done via a buf generate template. See buf.gen.yaml for an example.

Step 4: Run tests

There are two alternative ways of bootstrapping the tests.

Alternative 1:

Instantiate the generated test suites and call the methods you want to test.

package example

func Test_FreightService(t *testing.T) {
	t.Skip("this is just an example, the service is not implemented.")
	// setup server before test
	server := examplefreightv1.UnimplementedFreightServiceServer{}
	// setup test suite
	suite := examplefreightv1.FreightServiceTestSuite{
		T:      t,
		Server: server,
	}

	// run tests for each resource in the service
	ctx := context.Background()
	suite.TestShipper(ctx, examplefreightv1.ShipperTestSuiteConfig{
		// Create should return a resource which is valid to create, i.e.
		// all required fields set.
		Create: func() *examplefreightv1.Shipper {
			return &examplefreightv1.Shipper{
				DisplayName:    "Example shipper",
				BillingAccount: "billingAccounts/12345",
			}
		},
		// Update should return a resource which is valid to update, i.e.
		// all required fields set.
		Update: func() *examplefreightv1.Shipper {
			return &examplefreightv1.Shipper{
				DisplayName:    "Updated example shipper",
				BillingAccount: "billingAccounts/54321",
			}
		},
	})
}

Alternative 2:

Implement the generated configure provider interface (FreightServiceTestSuiteConfigProvider) and pass the implementation to TestServices to start the tests.

A benefit of using TestServices (over alternative 1) is that as new services or resources are added to the API the test code won't compile until the required inputs are also added (or explicitly ignored). This makes it harder to forget to add the test implementations for new services/resources.

package example

import "testing"

func Test_FreightService(t *testing.T) {
	// Even though no implementation exists, the tests will pass but be skipped.
	examplefreightv1.TestServices(t, &aipTests{})
}

type aipTests struct{}

var _ examplefreightv1.FreightServiceTestSuiteConfigProvider = &aipTests{}

func (a aipTests) FreightServiceShipper(_ *testing.T) *examplefreightv1.FreightServiceShipperTestSuiteConfig {
	// Returns nil to indicate that it's not ready to be tested.
	return nil
}

func (a aipTests) FreightServiceSite(_ *testing.T) *examplefreightv1.FreightServiceSiteTestSuiteConfig {
	// Returns nil to indicate that it's not ready to be tested.
	return nil
}

Skipping tests

There may be multiple reasons for an API to deviate from the guidance for standard methods (for examples see AIP-200). This plugin supports skipping individual or groups of tests using the Skip field generated for each test suite config.

Each test are compared, using strings.Contains, against a list of skipped test patterns. The full name of each test will follow the format [resource]/[method type]/[test_name].

Sample skips:

• "Get/invalid_name" skips the "invalid name" test for Get standard method.
• "Get" skips all tests for a Get standard method.

Suites

Create

Name	Description	Generated only if all are true:
missing parent	Method should fail with InvalidArgument if no parent is provided.	has Create method resource has a parent
invalid parent	Method should fail with InvalidArgument if provided parent is invalid.	has Create method resource has a parent
create time	Field create_time should be populated when the resource is created.	Create method does not return long-running operation has Create method has field 'create_time'
persisted	The created resource should be persisted and reachable with Get.	Create method does not return long-running operation has Create method has Get method
user settable id	If method support user settable IDs, when set the resource should be returned with the provided ID.	Create method does not return long-running operation has Create method has user settable ID
invalid user settable id	Method should fail with InvalidArgument if the user settable id doesn't conform to RFC-1034, see doc.	Create method does not return long-running operation has Create method has user settable ID
already exists	If method support user settable IDs and the same ID is reused the method should return AlreadyExists.	Create method does not return long-running operation has Create method has user settable ID
required fields	The method should fail with InvalidArgument if the resource has any required fields and they are not provided.	has Create method resource has any required fields
resource references	The method should fail with InvalidArgument if the resource has any resource references and they are invalid.	has Create method resource has any mutable resource references
etag populated	Field etag should be populated when the resource is created.	Create method does not return long-running operation has Create method has field 'etag'

Get

Name	Description	Generated only if all are true:
missing name	Method should fail with InvalidArgument if no name is provided.	has Get method
invalid name	Method should fail with InvalidArgument if the provided name is not valid.	has Get method
exists	Resource should be returned without errors if it exists.	has Get method
not found	Method should fail with NotFound if the resource does not exist.	has Get method
only wildcards	Method should fail with InvalidArgument if the provided name only contains wildcards ('-')	has Get method
soft-deleted	A soft-deleted resource should be returned without errors.	has Delete method has Get method has field 'delete_time'

BatchGet

Name	Description	Generated only if all are true:
invalid parent	Method should fail with InvalidArgument if provided parent is invalid.	has BatchGet method is not alternative batch request message resource has a parent
names missing	Method should fail with InvalidArgument if no names are provided.	has BatchGet method is not alternative batch request message
invalid names	Method should fail with InvalidArgument if a provided name is not valid.	has BatchGet method is not alternative batch request message
wildcard name	Method should fail with InvalidArgument if a provided name only contains wildcards (-)	has BatchGet method is not alternative batch request message
all exists	Resources should be returned without errors if they exist.	has BatchGet method is not alternative batch request message
atomic	The method must be atomic; it must fail for all resources or succeed for all resources (no partial success).	has BatchGet method is not alternative batch request message
parent mismatch	If a caller sets the "parent", and the parent collection in the name of any resource being retrieved does not match, the request must fail.	has BatchGet method is not alternative batch request message resource has a parent
ordered	The order of resources in the response must be the same as the names in the request.	has BatchGet method is not alternative batch request message
duplicate names	If a caller provides duplicate names, the service should return duplicate resources.	has BatchGet method is not alternative batch request message

Update

Name	Description	Generated only if all are true:
missing name	Method should fail with InvalidArgument if no name is provided.	has Update method
invalid name	Method should fail with InvalidArgument if provided name is not valid.	has Update method
update time	Field update_time should be updated when the resource is updated.	Create method does not return long-running operation Update method does not return long-running operation has Create method has Update method has field 'update_time'
persisted	The updated resource should be persisted and reachable with Get.	Update method does not return long-running operation has Get method has Update method
preserve create_time	The field create_time should be preserved when a '*'-update mask is used.	Update method does not return long-running operation has Update method has field 'create_time' resource has any required fields
etag mismatch	Method should fail with Aborted if the supplied etag doesnt match the current etag value.	has Update method has field 'etag'
etag updated	Field etag should have a new value when the resource is successfully updated.	has Update method has field 'etag'
not found	Method should fail with NotFound if the resource does not exist.	has Update method
invalid update mask	The method should fail with InvalidArgument if the update_mask is invalid.	Update method has update_mask has Update method
required fields	Method should fail with InvalidArgument if any required field is missing when called with '*' update_mask.	has Update method resource has any required fields

List

Name	Description	Generated only if all are true:
invalid parent	Method should fail with InvalidArgument if provided parent is invalid.	has List method resource has a parent
invalid page token	Method should fail with InvalidArgument is provided page token is not valid.	has List method
negative page size	Method should fail with InvalidArgument is provided page size is negative.	has List method
isolation	If parent is provided the method must only return resources under that parent.	has List method resource has a parent
last page	If there are no more resources, next_page_token should not be set.	has List method resource has a parent
more pages	If there are more resources, next_page_token should be set.	has List method resource has a parent
one by one	Listing resource one by one should eventually return all resources.	has List method resource has a parent
deleted	Method should not return deleted resources.	has Delete method has List method resource has a parent
page size zero	Listing resource with page size zero should eventually return all resources.	has List method resource has a parent

Search

Name	Description	Generated only if all are true:
invalid parent	Method should fail with InvalidArgument if provided parent is invalid.	has Search method resource has a parent
invalid page token	Method should fail with InvalidArgument is provided page token is not valid.	has Search method
negative page size	Method should fail with InvalidArgument is provided page size is negative.	has Search method
isolation	If parent is provided the method must only return resources under that parent.	has Search method resource has a parent
last page	If there are no more resources, next_page_token should not be set.	has Search method resource has a parent
more pages	If there are more resources, next_page_token should be set.	has Search method resource has a parent
one by one	Searching resource one by one should eventually return all resources.	has Search method resource has a parent
deleted	Method should not return deleted resources.	has Delete method has Search method resource has a parent

Delete

Name	Description	Generated only if all are true:
missing name	Method should fail with InvalidArgument if no name is provided.	has Delete method
invalid name	Method should fail with InvalidArgument if the provided name is not valid.	has Delete method
exists	Resource should be deleted without errors if it exists.	has Delete method
not found	Method should fail with NotFound if the resource does not exist.	has Delete method
already deleted	Method should fail with NotFound if the resource was already deleted. This also applies to soft-deletion.	has Delete method
only wildcards	Method should fail with InvalidArgument if the provided name only contains wildcards ('-')	has Delete method
etag mismatch	Method should fail with Aborted if the supplied etag doesnt match the current etag value.	has Delete method has field 'etag' request has etag field