From aa0fea4efa4db453a327240fb0764d7a85fe5e86 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Fri, 23 Feb 2024 15:40:38 -0500 Subject: [PATCH 01/14] Init --- docs/testing_servers.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 docs/testing_servers.md diff --git a/docs/testing_servers.md b/docs/testing_servers.md new file mode 100644 index 00000000..7c4676b6 --- /dev/null +++ b/docs/testing_servers.md @@ -0,0 +1,15 @@ +# Testing Servers + +Lorem ipsum dolor sit amet convectateur + + +Create a separate markdown page for "Testing a server" , to describe the process of implementing a server-under-test. + +The page does not need to regurgitate the details that are in the protos. Instead, it should outline the protocol: +- reading from stdin, writing results to stdout, writing other log messages to stderr), provide some pseudo-code for how to handle each of the methods of the conformance service, and point to the relevant doc comments via linking to the generated docs in the BSR. + +The server will likely be the shorter doc (shorter than a client guide), since it has less interaction with the test runner and only handles a single configuration per process. + +It should link to the "Configuring Conformance Tests" page, about how to configure the features that the server supports and how to define known failing and known flaky test cases. + +It could also link to examples in the connect-es repo. From 481166df240d118e00a3a61bfad6185f942daa3d Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Mon, 26 Feb 2024 13:42:22 -0500 Subject: [PATCH 02/14] Docs --- docs/testing_servers.md | 93 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 86 insertions(+), 7 deletions(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index 7c4676b6..c8395e4e 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -1,15 +1,94 @@ # Testing Servers -Lorem ipsum dolor sit amet convectateur +Testing servers with the conformance runner involves the following steps: +* Implementing the ConformanceService endpoints. +* Making an executable file that will read messages from `stdin` and write messages to `stdout`. +* Defining any configuration for what your server supports. For more information on how to do this, see the [Configuring and Running Tests documentation](./configuring_and_running_tests.md). -Create a separate markdown page for "Testing a server" , to describe the process of implementing a server-under-test. +## Process -The page does not need to regurgitate the details that are in the protos. Instead, it should outline the protocol: -- reading from stdin, writing results to stdout, writing other log messages to stderr), provide some pseudo-code for how to handle each of the methods of the conformance service, and point to the relevant doc comments via linking to the generated docs in the BSR. +The basic process for working with servers in the conformance suite will work as follows: -The server will likely be the shorter doc (shorter than a client guide), since it has less interaction with the test runner and only handles a single configuration per process. +* The conformance runner will pass a `ServerCompatRequest` message via `stdin`. This message will contain all the details necessary + for your implementation to start its server-under-test. Note that the request does not specify a port to listen on. + Implementations should instead pick an available ephemeral port according to their OS and return that value in the response. -It should link to the "Configuring Conformance Tests" page, about how to configure the features that the server supports and how to define known failing and known flaky test cases. + `protocol` specifies what protocol will be used to run the tests. + `http_version` specifies which HTTP version will be used. + `use_tls` specifies whether your server should generated a self-signed certificate. Clients will be configured to trust + this certificate when connecting. If `true`, the generated certificate should be returned in the `pem_cert` field of the + `ServerCompatResponse`. If this is set to false, the server should not use TLS and instead use a plaintext/unencrypted socket. + `client_tls_cert` represents a PEM-encoded certificate that, if provided, clients will use to authenticate themselves. + If specified, servers should require that clients provide certificates and they should validate the presented certificate is valid. + Note that this will always be empty if `use_tls` is `false`. + `message_receive_limit` specifies the maximum size in bytes for a message. If this value is non-zero, servers should reject + any message from a client that is larger than the size indicated. -It could also link to examples in the connect-es repo. +* Once the server is started, your implementation should then write a `ServerCompatResponse` message to `stdout`. This will + provide the conformance runner with details about your running server. + + `host` should be set with the host where your server is running. + `port` should be set with the port number where your server is listening. + `pem_cert` should contain the self-signed certificate generated by your server if `use_tls` was set to `true`. Clients + will verify this certificate when connecting via TLS. If `use_tls` was set to `false`, this should always be empty. + +## Implementing the ConformanceService + +The ConformanceService defines a series of endpoints that are meant to exercise all types of RPCs. The basic +approach for all six is that the server accepts the request(s), reads the details, and then reacts accordingly. The details +in the requests will specify various actions that server should take such as response headers and trailers to return, any errors +to return, and any response data to return. Servers mustl also echo back the received request data in their responses. +The method for doing so will vary depending on the type of RPC. + +In all, there are six total endpoints you will need to implement. Below is a brief description of each with helpful pseudocode +for your implementation. + +### Unary + +The `Unary` endpoint is a unary operation that accepts a single request of type `UnaryRequest` and returns a single response of type `UnaryResponse`. +The `UnaryRequest` contains a `response_definition` that tells the server how to respond. This definition contains a `oneof` which specifies +whether the server should return valid response data or return an error. + +```text +capture all request info sent including: +* any request headers +* the actual request body +* any timeout sent + +if response definition specifies valid response data then + build a ConformancePayload object and set the request info as well as the specified response data + +else + build an Error object with the specified error and set the request info into the error details + +set any response headers or trailers indicated in the response definition + +sleep for any specified response delay + +send the response +``` + +For the full documentation on implementing the `Unary` endpoint, click [here][unary]. + + +### IdempotentUnary + +### Unimplemented + +### ClientStream + +### ServerStream + +### BidiStream + +## Examples + +For examples, check out the following: + +* [Connect-ES conformance tests][connect-es-conformance] - This shows the entire process described above in TypeScript/JavaScript. +* [Connect-Go reference implementation][server-reference-impl] - For an example of implementing a server in Go, take a look at the reference server implementation used as part of the conformance runner. + +[connect-es-conformance]: https://github.com/connectrpc/connect-es/tree/main/packages/connect-conformance +[server-reference-impl]: https://github.com/connectrpc/conformance/blob/main/internal/app/referenceserver/impl.go +[unary]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.Unary From 97fe8ee7a168564cd85a330322da28d159ee960d Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Mon, 26 Feb 2024 13:44:03 -0500 Subject: [PATCH 03/14] Docs --- proto/connectrpc/conformance/v1/service.proto | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proto/connectrpc/conformance/v1/service.proto b/proto/connectrpc/conformance/v1/service.proto index 78a02530..ded93a53 100644 --- a/proto/connectrpc/conformance/v1/service.proto +++ b/proto/connectrpc/conformance/v1/service.proto @@ -31,7 +31,7 @@ service ConformanceService { // Response message data is specified as bytes. The service should echo back // request properties in the ConformancePayload and then include the message // data in the data field. - + // // If the response_delay_ms duration is specified, the server should wait the // given duration after reading the request before sending the corresponding // response. From 03918b1e82ad8b8b08f995cf6b8d066ecf864a1e Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Mon, 26 Feb 2024 13:45:55 -0500 Subject: [PATCH 04/14] Docs --- docs/testing_servers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index c8395e4e..ccfbdb7f 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -4,7 +4,7 @@ Testing servers with the conformance runner involves the following steps: * Implementing the ConformanceService endpoints. * Making an executable file that will read messages from `stdin` and write messages to `stdout`. -* Defining any configuration for what your server supports. For more information on how to do this, see the [Configuring and Running Tests documentation](./configuring_and_running_tests.md). +* Defining any configuration for what your server supports. For more information on how to do this, see the [Configuration Files](./configuring_and_running_tests.md#configuration-files) section in the [Configuring and Running Tests documentation](./configuring_and_running_tests.md). ## Process From 0857e186bccc96c72fe23cbb5f7e5b4c4b457dfb Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Mon, 26 Feb 2024 13:47:03 -0500 Subject: [PATCH 05/14] Docs --- docs/testing_servers.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index ccfbdb7f..a82b5e63 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -14,24 +14,24 @@ The basic process for working with servers in the conformance suite will work as for your implementation to start its server-under-test. Note that the request does not specify a port to listen on. Implementations should instead pick an available ephemeral port according to their OS and return that value in the response. - `protocol` specifies what protocol will be used to run the tests. - `http_version` specifies which HTTP version will be used. - `use_tls` specifies whether your server should generated a self-signed certificate. Clients will be configured to trust - this certificate when connecting. If `true`, the generated certificate should be returned in the `pem_cert` field of the - `ServerCompatResponse`. If this is set to false, the server should not use TLS and instead use a plaintext/unencrypted socket. - `client_tls_cert` represents a PEM-encoded certificate that, if provided, clients will use to authenticate themselves. - If specified, servers should require that clients provide certificates and they should validate the presented certificate is valid. - Note that this will always be empty if `use_tls` is `false`. - `message_receive_limit` specifies the maximum size in bytes for a message. If this value is non-zero, servers should reject - any message from a client that is larger than the size indicated. + * `protocol` specifies what protocol will be used to run the tests. + * `http_version` specifies which HTTP version will be used. + * `use_tls` specifies whether your server should generated a self-signed certificate. Clients will be configured to trust + this certificate when connecting. If `true`, the generated certificate should be returned in the `pem_cert` field of the + `ServerCompatResponse`. If this is set to false, the server should not use TLS and instead use a plaintext/unencrypted socket. + * `client_tls_cert` represents a PEM-encoded certificate that, if provided, clients will use to authenticate themselves. + If specified, servers should require that clients provide certificates and they should validate the presented certificate is valid. + Note that this will always be empty if `use_tls` is `false`. + * `message_receive_limit` specifies the maximum size in bytes for a message. If this value is non-zero, servers should reject + any message from a client that is larger than the size indicated. * Once the server is started, your implementation should then write a `ServerCompatResponse` message to `stdout`. This will provide the conformance runner with details about your running server. - `host` should be set with the host where your server is running. - `port` should be set with the port number where your server is listening. - `pem_cert` should contain the self-signed certificate generated by your server if `use_tls` was set to `true`. Clients - will verify this certificate when connecting via TLS. If `use_tls` was set to `false`, this should always be empty. + * `host` should be set with the host where your server is running. + * `port` should be set with the port number where your server is listening. + * `pem_cert` should contain the self-signed certificate generated by your server if `use_tls` was set to `true`. Clients + will verify this certificate when connecting via TLS. If `use_tls` was set to `false`, this should always be empty. ## Implementing the ConformanceService From edc9a21f6cdfc8b91af3d1a376f4c4537e5a12dc Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Tue, 27 Feb 2024 14:30:18 -0500 Subject: [PATCH 06/14] Testing servers --- docs/testing_servers.md | 168 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 165 insertions(+), 3 deletions(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index a82b5e63..40f85b6a 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -51,13 +51,16 @@ The `UnaryRequest` contains a `response_definition` that tells the server how to whether the server should return valid response data or return an error. ```text +read the request + capture all request info sent including: * any request headers * the actual request body -* any timeout sent if response definition specifies valid response data then - build a ConformancePayload object and set the request info as well as the specified response data + build a ConformancePayload object + + set the request info as well as the specified response data else build an Error object with the specified error and set the request info into the error details @@ -66,7 +69,7 @@ set any response headers or trailers indicated in the response definition sleep for any specified response delay -send the response +return the response or the error depending on which was specified ``` For the full documentation on implementing the `Unary` endpoint, click [here][unary]. @@ -74,14 +77,168 @@ For the full documentation on implementing the `Unary` endpoint, click [here][un ### IdempotentUnary +The `IdempotentUnary` endpoint is also a unary operation. It accepts a request of type `IdempotentUnaryRequest` and returns +a single response of type `IdempotentUnaryResponse`. It should be handled in mostly the same way as `Unary`. However, the +only major difference is that this endpoint should be invoked via an HTTP `GET`. As a result, there is no request body +so the endpoint should read any query params and set them accordingly in the `connect_get_info` field of the +returned `ConformancePayload`. + +For the full documentation on implementing the `IdempotentUnary` endpoint, click [here][idempotentunary]. + ### Unimplemented +The `Unimplemented` endpoint is also a unary operation, but contrary to the above unary endpoints, the implementation +of `Unimplemented` should simply return an `unimplemented` error. It is not necessary to echo back any request information or +conformance payload in the error details. + +For the full documentation on handling the `Unimplemented` endpoint, click [here][unimplemented]. + + ### ClientStream +The `ClientStream` endpoint is a client-streaming operation. It accepts one-to-many requests of type `ClientStreamRequest` +and returns a single response of type `ClientStreamResponse`. Since a client-streaming operation returns a single response, +its process is similar to `Unary`. + +With client-streaming, the response definition specifying the desired response will only be specified in the first request +on the stream. + +```text +while requests are being sent do the following + read a request from the stream + + capture the request body + + if this is the first message being received then + save the response definition + +when requests are complete then + if the response definition specified valid response data then + build a ConformancePayload object + set the following into the conformance payload: + * all requests received + * any specified response data + * any request headers from the stream + + else + build an Error object with the specified error + set the following into the error details + * all requests received + * any specified response data + * any request headers from the stream + +set any response headers or trailers indicated in the response definition + +sleep for any specified response delay + +return the response or the error depending on which was specified +``` + +For the full documentation on handling the `ClientStream` endpoint, click [here][clientstream]. + ### ServerStream +The `ServerStream` endpoint is a server-streaming operation. It accepts a single request of type `ServerStreamRequest` and +returns one-to-many response of type `ServerStreamResponse`. When echoing request information back, the `ServerStream` +implementation should only set this information in the first response sent. + +```text +read the request + +capture all request info sent including: +* any request headers +* the actual request body +* the response definition + +if a response definition was specified then + set any response headers or trailers on the response stream + + immediately send the headers/trailers on the stream so that they can be read by the client + + loop over any response data specified + build a ConformancePayload object + + set the response data into the conformance payload + + if this is the first response being sent then + set the request info into the conformance payload + + sleep for any specified response delay + + send the response + + if an error was specified in the response definition then + if no responses have been sent yet + build an Error object with the specified error + + set the following into the error details + * the received request + * any request headers +``` + +For the full documentation on handling the `ServerStream` endpoint, click [here][serverstream]. + ### BidiStream +The `BidiStream` endpoint is a bidirectional-streaming operation. It accepts one-to-many requests of type `BidiStreamRequest` and +returns one-to-many responses of type `BidiStreamResponse`. The `BidiStream` operation implementation should be capable +of handling full-duplex streaming as well as half-duplex streaming. + +```text +while requests are being sent do the following + read a request from the stream + + capture the request body + + if this is the first message being received then + save the response definition + save whether this is full duplex or half duplex + + if a response definition was specified then + set any response headers or trailers on the response stream + + if full duplex then + immediately send the headers/trailers on the stream so that they can be read by the client + + if full duplex then + if response data was specified then + build a ConformancePayload object + + set the response data into the conformance payload + + if this is the first response being sent then + set the request info into the conformance payload + + sleep for any specified response delay + + send the response + + if half duplex and if response data was specified then + immediately send the headers/trailers on the stream so that they can be read by the client + + loop over any response data specified + build a ConformancePayload object + + set the response data into the conformance payload + + if this is the first response being sent then + set the request info into the conformance payload + + sleep for any specified response delay + + send the response + + if an error was specified in the response definition then + if no responses have been sent yet + build an Error object with the specified error + + set the following into the error details + * the received request + * any request headers +``` + +For the full documentation on handling the `BidiStream` endpoint, click [here][bidistream]. + ## Examples For examples, check out the following: @@ -92,3 +249,8 @@ For examples, check out the following: [connect-es-conformance]: https://github.com/connectrpc/connect-es/tree/main/packages/connect-conformance [server-reference-impl]: https://github.com/connectrpc/conformance/blob/main/internal/app/referenceserver/impl.go [unary]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.Unary +[idempotentunary]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.IdempotentUnary +[unimplemented]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.Unimplemented +[clientstream]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.ClientStream +[serverstream]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.ServerStream +[bidistream]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.BidiStream From cb107bbf112f7c91166ffa50f982e258e3d673c6 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Tue, 27 Feb 2024 15:23:57 -0500 Subject: [PATCH 07/14] Docs --- docs/testing_servers.md | 222 +++++++++++++++++++++++++++------------- 1 file changed, 153 insertions(+), 69 deletions(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index 40f85b6a..e882e05a 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -14,13 +14,15 @@ The basic process for working with servers in the conformance suite will work as for your implementation to start its server-under-test. Note that the request does not specify a port to listen on. Implementations should instead pick an available ephemeral port according to their OS and return that value in the response. + Fields in the request are: + * `protocol` specifies what protocol will be used to run the tests. * `http_version` specifies which HTTP version will be used. - * `use_tls` specifies whether your server should generated a self-signed certificate. Clients will be configured to trust + * `use_tls` specifies whether your server should generate a self-signed certificate. Clients will be configured to trust this certificate when connecting. If `true`, the generated certificate should be returned in the `pem_cert` field of the - `ServerCompatResponse`. If this is set to false, the server should not use TLS and instead use a plaintext/unencrypted socket. + `ServerCompatResponse`. If this is set to `false`, the server should not use TLS and instead use a plaintext/unencrypted socket. * `client_tls_cert` represents a PEM-encoded certificate that, if provided, clients will use to authenticate themselves. - If specified, servers should require that clients provide certificates and they should validate the presented certificate is valid. + If specified, servers should require that clients provide certificates and they should ensure the presented certificate is valid. Note that this will always be empty if `use_tls` is `false`. * `message_receive_limit` specifies the maximum size in bytes for a message. If this value is non-zero, servers should reject any message from a client that is larger than the size indicated. @@ -35,41 +37,63 @@ The basic process for working with servers in the conformance suite will work as ## Implementing the ConformanceService -The ConformanceService defines a series of endpoints that are meant to exercise all types of RPCs. The basic -approach for all six is that the server accepts the request(s), reads the details, and then reacts accordingly. The details -in the requests will specify various actions that server should take such as response headers and trailers to return, any errors -to return, and any response data to return. Servers mustl also echo back the received request data in their responses. -The method for doing so will vary depending on the type of RPC. +The ConformanceService defines a series of endpoints that are meant to exercise all types of RPCs. The details +in the requests will specify various attributes that the server should handle, such as determining response headers and +trailers to return, any errors to throw, and any data to respond with. All request types contain a response definition +which is used to instruct the server how to respond to the request. This response definition can also be unset entirely +and servers should respond in a fashion specific to their RPC type. The various approaches are outlined below. + +Servers must also echo back the received request data in their responses and the method for doing so will vary depending +on the type of RPC. The `ConformancePayload` message is a field on all RPC response types. Servers will use this field +to echo back these details. This will include: + +* Any response data +* Information observed from the request such as: + * Request headers + * Any timeout specified + * All request bodies received + * Any query parameters observed (only applicable for GET operations such as `IdempotentUnary`) In all, there are six total endpoints you will need to implement. Below is a brief description of each with helpful pseudocode for your implementation. ### Unary -The `Unary` endpoint is a unary operation that accepts a single request of type `UnaryRequest` and returns a single response of type `UnaryResponse`. -The `UnaryRequest` contains a `response_definition` that tells the server how to respond. This definition contains a `oneof` which specifies -whether the server should return valid response data or return an error. +The `Unary` endpoint is a unary operation that accepts a single request of type `UnaryRequest` and returns a single +response of type `UnaryResponse`. + +The `UnaryRequest` contains a `response_definition` that tells the server how to respond. This definition contains a +`oneof` which specifies whether the server should return valid response data or return an error. Servers should also +allow this response definition to be unset. In which case, they should set no response headers or trailers and return +no response data. The returned conformance payload should only contain the observed request information. + +**Pseudocode** ```text read the request -capture all request info sent including: -* any request headers -* the actual request body - -if response definition specifies valid response data then - build a ConformancePayload object +capture any request headers and the actual request body as request info + +if a response definition was specified then + if response definition specifies valid response data then + + build a conformance payload + + set the request info and response data into the conformance payload + + else + build an error with the specified error and set the request info into the error details - set the request info as well as the specified response data + set any response headers or trailers indicated in the response definition else - build an Error object with the specified error and set the request info into the error details + build a conformance payload -set any response headers or trailers indicated in the response definition + set the request info into the conformance payload sleep for any specified response delay -return the response or the error depending on which was specified +return the response with conformance payload or the error depending on which was specified ``` For the full documentation on implementing the `Unary` endpoint, click [here][unary]. @@ -78,10 +102,11 @@ For the full documentation on implementing the `Unary` endpoint, click [here][un ### IdempotentUnary The `IdempotentUnary` endpoint is also a unary operation. It accepts a request of type `IdempotentUnaryRequest` and returns -a single response of type `IdempotentUnaryResponse`. It should be handled in mostly the same way as `Unary`. However, the -only major difference is that this endpoint should be invoked via an HTTP `GET`. As a result, there is no request body +a single response of type `IdempotentUnaryResponse`. It should be handled in mostly the same way as `Unary`. + +The only major difference is that this endpoint should be invoked via an HTTP `GET`. As a result, there is no request body so the endpoint should read any query params and set them accordingly in the `connect_get_info` field of the -returned `ConformancePayload`. +returned `ConformancePayload`. This RPC is the only one that sets the `connect_get_info` field. For the full documentation on implementing the `IdempotentUnary` endpoint, click [here][idempotentunary]. @@ -93,15 +118,18 @@ conformance payload in the error details. For the full documentation on handling the `Unimplemented` endpoint, click [here][unimplemented]. - ### ClientStream The `ClientStream` endpoint is a client-streaming operation. It accepts one-to-many requests of type `ClientStreamRequest` and returns a single response of type `ClientStreamResponse`. Since a client-streaming operation returns a single response, its process is similar to `Unary`. -With client-streaming, the response definition specifying the desired response will only be specified in the first request -on the stream. +With client-streaming, the response definition will only be specified in the first request on the stream and should be +ignored in all subsequent requests. As with `Unary`, servers should also allow this response definition to be unset. In +which case, they should set no response headers or trailers and return no response data. The returned conformance +payload should only contain the observed request information. + +**Pseudocode** ```text while requests are being sent do the following @@ -111,23 +139,24 @@ while requests are being sent do the following if this is the first message being received then save the response definition - + when requests are complete then - if the response definition specified valid response data then - build a ConformancePayload object - set the following into the conformance payload: - * all requests received - * any specified response data - * any request headers from the stream - - else - build an Error object with the specified error - set the following into the error details - * all requests received - * any specified response data - * any request headers from the stream - -set any response headers or trailers indicated in the response definition + if a response definition was specified then + if response definition specifies valid response data then + + build a conformance payload + + set the request info and response data into the conformance payload + + else + build an error with the specified error and set the request info into the error details + + set any response headers or trailers indicated in the response definition + + else + build a conformance payload + + set the request info into the conformance payload sleep for any specified response delay @@ -142,13 +171,23 @@ The `ServerStream` endpoint is a server-streaming operation. It accepts a single returns one-to-many response of type `ServerStreamResponse`. When echoing request information back, the `ServerStream` implementation should only set this information in the first response sent. +Servers should immediately send response headers on the stream before sleeping +for any specified response delay and/or sending the first message so that +clients can be unblocked reading response headers. + +If a response definition is not specified OR is specified, but response data +is empty, the server should skip sending anything on the stream. When there +are no responses to send, servers should throw an error if one is provided +and return without error if one is not. Stream headers and trailers should +still be set on the stream if provided regardless of whether a response is +sent or an error is thrown. + +**Pseudocode** + ```text read the request -capture all request info sent including: -* any request headers -* the actual request body -* the response definition +capture any request headers, the response definition and the actual request body as request info if a response definition was specified then set any response headers or trailers on the response stream @@ -156,7 +195,7 @@ if a response definition was specified then immediately send the headers/trailers on the stream so that they can be read by the client loop over any response data specified - build a ConformancePayload object + build a conformance payload set the response data into the conformance payload @@ -169,7 +208,7 @@ if a response definition was specified then if an error was specified in the response definition then if no responses have been sent yet - build an Error object with the specified error + build an error with the specified error set the following into the error details * the received request @@ -180,9 +219,54 @@ For the full documentation on handling the `ServerStream` endpoint, click [here] ### BidiStream -The `BidiStream` endpoint is a bidirectional-streaming operation. It accepts one-to-many requests of type `BidiStreamRequest` and -returns one-to-many responses of type `BidiStreamResponse`. The `BidiStream` operation implementation should be capable -of handling full-duplex streaming as well as half-duplex streaming. +The `BidiStream` endpoint is a bidirectional-streaming operation. It accepts +one-to-many requests of type `BidiStreamRequest` and returns one-to-many +responses of type `BidiStreamResponse`. + +Similar to `ServerStream`, servers should immediately send response headers on +the stream before sleeping for any specified response delay and/or sending the +first message so that clients can be unblocked reading response headers. + +If a response definition is not specified OR is specified, but response data +is empty, the server should skip sending anything on the stream. When there +are no responses to send, servers should throw an error if one is provided +and return without error if one is not. Stream headers and trailers should +still be set on the stream if provided regardless of whether a response is +sent or an error is thrown. + +The `BidiStreamRequest` type specifies whether the operation should be full duplex +or half duplex via the `full_duplex` field. + +If the `full_duplex` field is true: + +* the handler should read one request and then send back one response, and + then alternate, reading another request and then sending back another response, etc. + +* if the server receives a request and has no responses to send, it + should throw the error specified in the request. + +* the service should echo back all request properties in the first response + including the last received request. Subsequent responses should only + echo back the last received request. + +* if the `response_delay_ms` duration is specified, the server should wait the given + duration after reading the request before sending the corresponding + response. + +If the `full_duplex` field is false: + +* the handler should read all requests until the client is done sending. + Once all requests are read, the server should then send back any responses + specified in the response definition. + +* the server should echo back all request properties, including all request + messages in the order they were received, in the first response. Subsequent + responses should only include the message data in the data field. + +* if the `response_delay_ms` duration is specified, the server should wait that + long in between sending each response message. + +**Pseudocode** ```text while requests are being sent do the following @@ -202,7 +286,7 @@ while requests are being sent do the following if full duplex then if response data was specified then - build a ConformancePayload object + build a conformance payload set the response data into the conformance payload @@ -211,30 +295,30 @@ while requests are being sent do the following sleep for any specified response delay - send the response + send a response - if half duplex and if response data was specified then - immediately send the headers/trailers on the stream so that they can be read by the client + if half duplex and if response data was specified then + immediately send the headers/trailers on the stream so that they can be read by the client - loop over any response data specified - build a ConformancePayload object + loop over any response data specified + build a conformance payload - set the response data into the conformance payload + set the response data into the conformance payload - if this is the first response being sent then - set the request info into the conformance payload + if this is the first response being sent then + set the request info into the conformance payload - sleep for any specified response delay + sleep for any specified response delay - send the response + send the response - if an error was specified in the response definition then - if no responses have been sent yet - build an Error object with the specified error + if an error was specified in the response definition then + if no responses have been sent yet + build an Error object with the specified error - set the following into the error details - * the received request - * any request headers + set the following into the error details + * the received request + * any request headers ``` For the full documentation on handling the `BidiStream` endpoint, click [here][bidistream]. From d503b269f58a9b8f3d540e62afd81e297c6ac569 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Tue, 27 Feb 2024 17:21:34 -0500 Subject: [PATCH 08/14] Generate --- .../v1/conformancev1connect/service.connect.go | 14 ++++++++++++++ .../connectrpc/conformance/v1/service_grpc.pb.go | 14 ++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/internal/gen/proto/go/connectrpc/conformance/v1/conformancev1connect/service.connect.go b/internal/gen/proto/go/connectrpc/conformance/v1/conformancev1connect/service.connect.go index 83132037..a286244f 100644 --- a/internal/gen/proto/go/connectrpc/conformance/v1/conformancev1connect/service.connect.go +++ b/internal/gen/proto/go/connectrpc/conformance/v1/conformancev1connect/service.connect.go @@ -81,6 +81,13 @@ var ( // ConformanceServiceClient is a client for the connectrpc.conformance.v1.ConformanceService // service. type ConformanceServiceClient interface { + // A unary operation. The request indicates the response headers and trailers + // and also indicates either a response message or an error to send back. + // + // Response message data is specified as bytes. The service should echo back + // request properties in the ConformancePayload and then include the message + // data in the data field. + // // If the response_delay_ms duration is specified, the server should wait the // given duration after reading the request before sending the corresponding // response. @@ -280,6 +287,13 @@ func (c *conformanceServiceClient) IdempotentUnary(ctx context.Context, req *con // ConformanceServiceHandler is an implementation of the // connectrpc.conformance.v1.ConformanceService service. type ConformanceServiceHandler interface { + // A unary operation. The request indicates the response headers and trailers + // and also indicates either a response message or an error to send back. + // + // Response message data is specified as bytes. The service should echo back + // request properties in the ConformancePayload and then include the message + // data in the data field. + // // If the response_delay_ms duration is specified, the server should wait the // given duration after reading the request before sending the corresponding // response. diff --git a/internal/gen/proto/go/connectrpc/conformance/v1/service_grpc.pb.go b/internal/gen/proto/go/connectrpc/conformance/v1/service_grpc.pb.go index 9e43e223..7043e39b 100644 --- a/internal/gen/proto/go/connectrpc/conformance/v1/service_grpc.pb.go +++ b/internal/gen/proto/go/connectrpc/conformance/v1/service_grpc.pb.go @@ -45,6 +45,13 @@ const ( // // For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream. type ConformanceServiceClient interface { + // A unary operation. The request indicates the response headers and trailers + // and also indicates either a response message or an error to send back. + // + // Response message data is specified as bytes. The service should echo back + // request properties in the ConformancePayload and then include the message + // data in the data field. + // // If the response_delay_ms duration is specified, the server should wait the // given duration after reading the request before sending the corresponding // response. @@ -286,6 +293,13 @@ func (c *conformanceServiceClient) IdempotentUnary(ctx context.Context, in *Idem // All implementations must embed UnimplementedConformanceServiceServer // for forward compatibility type ConformanceServiceServer interface { + // A unary operation. The request indicates the response headers and trailers + // and also indicates either a response message or an error to send back. + // + // Response message data is specified as bytes. The service should echo back + // request properties in the ConformancePayload and then include the message + // data in the data field. + // // If the response_delay_ms duration is specified, the server should wait the // given duration after reading the request before sending the corresponding // response. From c55acc6ef4b8e8d78931dee152ae32243fc71d71 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Thu, 29 Feb 2024 11:16:22 -0500 Subject: [PATCH 09/14] Update docs/testing_servers.md Co-authored-by: Joshua Humphries <2035234+jhump@users.noreply.github.com> --- docs/testing_servers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index e882e05a..8acff49c 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -4,7 +4,7 @@ Testing servers with the conformance runner involves the following steps: * Implementing the ConformanceService endpoints. * Making an executable file that will read messages from `stdin` and write messages to `stdout`. -* Defining any configuration for what your server supports. For more information on how to do this, see the [Configuration Files](./configuring_and_running_tests.md#configuration-files) section in the [Configuring and Running Tests documentation](./configuring_and_running_tests.md). +* Defining any configuration for what your server supports. For more information on how to do this, see the docs for [configuring and running tests](./configuring_and_running_tests.md#configuration-files). ## Process From 87a83b64c1149692ded7e77d118a5292572ee6be Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Thu, 29 Feb 2024 11:16:52 -0500 Subject: [PATCH 10/14] Update docs/testing_servers.md Co-authored-by: Joshua Humphries <2035234+jhump@users.noreply.github.com> --- docs/testing_servers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index 8acff49c..5e670f07 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -93,7 +93,7 @@ else sleep for any specified response delay -return the response with conformance payload or the error depending on which was specified +return the response with conformance payload or raise the error, depending on which was specified ``` For the full documentation on implementing the `Unary` endpoint, click [here][unary]. From bb9eabccd7a1b44f3d81fddd1616bfc54a55677d Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Thu, 29 Feb 2024 15:26:00 -0500 Subject: [PATCH 11/14] Generate --- docs/testing_servers.md | 345 +++++++++++------- .../conformance/v1/server_compat.pb.go | 27 +- .../conformance/v1/server_compat.proto | 27 +- 3 files changed, 260 insertions(+), 139 deletions(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index 5e670f07..93e73b1c 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -1,70 +1,125 @@ # Testing Servers -Testing servers with the conformance runner involves the following steps: +The conformance suite provides the ability to run conformance tests against a server implementation. Testing servers +involves the following steps: + +1. Defining any configuration for what your server supports. For more information on how to do this, see the docs for [configuring and running tests](./configuring_and_running_tests.md#configuration-files). +2. Writing an executable file that can read `ServerCompatRequest` messages from `stdin` (and write to `stdout` -- see Step 4). +3. Starting your server according to the values in the request message. +4. Writing a `ServerCompatResponse` about the running server to `stdout`. +5. Implementing the [ConformanceService][conformanceservice] endpoints to handle requests from the reference client. + +## Starting your server + +When the conformance runner is executed for a server-under-test, the runner will analyze the configuration you've specified +and will use that information to build a `ServerCompatRequest`. This request is serialized to bytes and written to `stdin`. +It is then up to the executable file you created as part of Step 2 to read this message and start your server. + +The messages written to `stdin` are size-delimited. This means that first you will need to read a fixed four-byte +preface, which returns a network-byte-order (i.e. big-endian) 32-bit integer. This integer represents the size of the +actual message. After this value is read, you should then read the number of bytes it specifies and then unmarshal those +bytes into a `ServerCompatRequest`. + +This message will contain all the details necessary for your implementation to start its server-under-test. Note that +the request does not specify a port to listen on. Implementations should instead pick an available ephemeral port +according to their OS and return that value in the response. + +Fields in the request are: + +* `protocol` which signals to the server that it must support at least this protocol. Note that it is fine to support others. + For example if `PROTOCOL_CONNECT` is specified, the server _must_ support at least Connect, but _may_ also support + gRPC or gRPC-web. +* `http_version` which signals to the server the minimum HTTP version to support. As with `protocol`, it is fine to support + other versions. +* `use_tls` which specifies whether your server should generate a certificate that clients will be configured to trust when + connecting. The certificate can be any TLS certificate where the subject matches the value sent back in + the `host` field of the `ServerCompatResponse`. Self-signed certificates (and `localhost` as the subject) are allowed. + If `true`, the generated certificate should be returned in the `pem_cert` field of the + `ServerCompatResponse`. If this is set to `false`, the server should not use TLS and instead use a plaintext/unencrypted socket. +* `client_tls_cert` which represents a PEM-encoded certificate that, if provided, clients will use to authenticate themselves. + If specified, servers should require that clients provide certificates and they should ensure the presented certificate is valid. + Note that this will always be empty if `use_tls` is `false`. +* `message_receive_limit` which specifies the maximum size in bytes for a message. If this value is non-zero, servers should reject + any message from a client that is larger than the size indicated. + +Using the values in the request, you can then start your server implementation. Once started, your implementation should +build a `ServerCompatResponse` message. This will provide the conformance runner with details about your running server. + +Once built, you should then write the response message to `stdout` using the same size-delimited algorithm described above. First, +write a network-encoded 32-bit integer indicating the size of the `ServerCompatResponse` message. Then, serialize the +response to bytes and write that to `stdout`. + +Fields in the response are: + +* `host` which should be set with the host where your server is running. This should usually be `127.0.0.1`, unless your + program actually starts a remote server to which the client should connect. +* `port` which should be set with the port number where your server is listening. +* `pem_cert` which should contain the TLS certificate, in PEM format, generated by your server if `use_tls` was set + to `true`. Clients will verify this certificate when connecting via TLS. If `use_tls` was set to `false`, this + should always be empty. -* Implementing the ConformanceService endpoints. -* Making an executable file that will read messages from `stdin` and write messages to `stdout`. -* Defining any configuration for what your server supports. For more information on how to do this, see the docs for [configuring and running tests](./configuring_and_running_tests.md#configuration-files). +## Implementing the ConformanceService -## Process +When verifying a server-under-test, the conformance runner will use a reference client +implementation written in Connect-Go and will use this client to issue requests to the server. The reference client +will read the server's responses and return them to the conformance runner. So, all you need to do from a server +implementation standpoint is handle the requests accordingly. -The basic process for working with servers in the conformance suite will work as follows: +The [ConformanceService][conformanceservice] defines a series of endpoints that are meant to exercise all types of RPCs. The details +in the requests will specify various attributes that the server should handle, such as determining response headers and +trailers to return, any errors to throw, and any data to respond with. All request types contain a response definition +which is used to instruct the server how to respond to the request. This response definition can also be unset entirely +and servers should respond in a fashion specific to their RPC type. -* The conformance runner will pass a `ServerCompatRequest` message via `stdin`. This message will contain all the details necessary - for your implementation to start its server-under-test. Note that the request does not specify a port to listen on. - Implementations should instead pick an available ephemeral port according to their OS and return that value in the response. +In addition to any specified response data, servers must also echo back the received request data in their responses. +This will be done via one of two ways -- either by setting the information into a `ConformancePayload` message in their +response or by setting the information into the details of an `Error` message. - Fields in the request are: +### ConformancePayload - * `protocol` specifies what protocol will be used to run the tests. - * `http_version` specifies which HTTP version will be used. - * `use_tls` specifies whether your server should generate a self-signed certificate. Clients will be configured to trust - this certificate when connecting. If `true`, the generated certificate should be returned in the `pem_cert` field of the - `ServerCompatResponse`. If this is set to `false`, the server should not use TLS and instead use a plaintext/unencrypted socket. - * `client_tls_cert` represents a PEM-encoded certificate that, if provided, clients will use to authenticate themselves. - If specified, servers should require that clients provide certificates and they should ensure the presented certificate is valid. - Note that this will always be empty if `use_tls` is `false`. - * `message_receive_limit` specifies the maximum size in bytes for a message. If this value is non-zero, servers should reject - any message from a client that is larger than the size indicated. +The [`ConformancePayload`][conformancepayload] message is a field on most all response types which contains the following fields: -* Once the server is started, your implementation should then write a `ServerCompatResponse` message to `stdout`. This will - provide the conformance runner with details about your running server. +* `data` which should be set with any response data specified in the response definition. +* `request_info` which is a nested message of type [`RequestInfo`][requestinfo] structured as: + * `request_headers` which represents any observed request headers. + * `timeout_ms` which indicates any timeout included in the request. + * `requests` which is a `repeated` field of [`google.protobuf.Any`][any] types. This is used to echo back all the requests + received. For unary and server-streaming requests, this should always contain a single request. For client-streaming + and half-duplex bidi-streaming, this should contain all client requests in the order received and should be present + in each response. For full-duplex bidi-streaming, this should contain all requests in the order they were received + since the last sent response. + * `connect_get_info` which is only applicable for GET operations such as [`IdempotentUnary`](#idempotentunary). It should contain any + observed query parameters in the request URL. - * `host` should be set with the host where your server is running. - * `port` should be set with the port number where your server is listening. - * `pem_cert` should contain the self-signed certificate generated by your server if `use_tls` was set to `true`. Clients - will verify this certificate when connecting via TLS. If `use_tls` was set to `false`, this should always be empty. + > [!NOTE] + > The response type for the [`Unimplemented`](#unimplemented) endpoint does not contain a conformance payload as + > implementations are not meant to echo back any information. -## Implementing the ConformanceService +### Error -The ConformanceService defines a series of endpoints that are meant to exercise all types of RPCs. The details -in the requests will specify various attributes that the server should handle, such as determining response headers and -trailers to return, any errors to throw, and any data to respond with. All request types contain a response definition -which is used to instruct the server how to respond to the request. This response definition can also be unset entirely -and servers should respond in a fashion specific to their RPC type. The various approaches are outlined below. +The [`Error`][error] message is used to return errors back to the client. It is structured as follows: -Servers must also echo back the received request data in their responses and the method for doing so will vary depending -on the type of RPC. The `ConformancePayload` message is a field on all RPC response types. Servers will use this field -to echo back these details. This will include: +* `code` which represents the error code. +* `message` which is an optional field indicating the error message. +* `details` which is a `repeated` field of [`google.protobuf.Any`][any] types. This is used to attach arbitrary messages and + in the conformance suite is used to echo back request information in circumstances where a conformance payload was + unable to be returned -* Any response data -* Information observed from the request such as: - * Request headers - * Any timeout specified - * All request bodies received - * Any query parameters observed (only applicable for GET operations such as `IdempotentUnary`) +## Endpoints -In all, there are six total endpoints you will need to implement. Below is a brief description of each with helpful pseudocode -for your implementation. +In all, there are six total endpoints in the conformance service definition. Below is a brief description of each with helpful +pseudocode. ### Unary The `Unary` endpoint is a unary operation that accepts a single request of type `UnaryRequest` and returns a single response of type `UnaryResponse`. -The `UnaryRequest` contains a `response_definition` that tells the server how to respond. This definition contains a -`oneof` which specifies whether the server should return valid response data or return an error. Servers should also -allow this response definition to be unset. In which case, they should set no response headers or trailers and return +The `UnaryRequest` contains a `response_definition` field that contains a +`oneof` which specifies whether the server should return valid response data or return an error. If an error is specified, +servers should set request information into the error details. + +Servers should also allow this response definition to be unset. In which case, they should set no response headers or trailers and return no response data. The returned conformance payload should only contain the observed request information. **Pseudocode** @@ -74,22 +129,20 @@ read the request capture any request headers and the actual request body as request info -if a response definition was specified then - if response definition specifies valid response data then - - build a conformance payload - - set the request info and response data into the conformance payload - - else - build an error with the specified error and set the request info into the error details +if response definition specifies an error { + build an error with the specified error and set the request info into the error details set any response headers or trailers indicated in the response definition -else - build a conformance payload +} else { + build a conformance payload with the request info - set the request info into the conformance payload + if a response definition exists { + set response data into the conformance payload + + set any response headers or trailers indicated in the response definition + } +} sleep for any specified response delay @@ -105,16 +158,19 @@ The `IdempotentUnary` endpoint is also a unary operation. It accepts a request o a single response of type `IdempotentUnaryResponse`. It should be handled in mostly the same way as `Unary`. The only major difference is that this endpoint should be invoked via an HTTP `GET`. As a result, there is no request body -so the endpoint should read any query params and set them accordingly in the `connect_get_info` field of the +so the endpoint should read any query parameters and set them accordingly in the `connect_get_info` field of the returned `ConformancePayload`. This RPC is the only one that sets the `connect_get_info` field. +Note that this endpoint is only applicable for Connect implementations that support `GET` requests. All others do not +need to implement it. + For the full documentation on implementing the `IdempotentUnary` endpoint, click [here][idempotentunary]. ### Unimplemented -The `Unimplemented` endpoint is also a unary operation, but contrary to the above unary endpoints, the implementation -of `Unimplemented` should simply return an `unimplemented` error. It is not necessary to echo back any request information or -conformance payload in the error details. +The `Unimplemented` endpoint is also a unary operation, but contrary to the above unary endpoints, this endpoint should +not be implemented at all. The server framework should instead return an `unimplemented` error. It is not necessary to +echo back any request information or conformance payload in the error details. For the full documentation on handling the `Unimplemented` endpoint, click [here][unimplemented]. @@ -125,42 +181,45 @@ and returns a single response of type `ClientStreamResponse`. Since a client-str its process is similar to `Unary`. With client-streaming, the response definition will only be specified in the first request on the stream and should be -ignored in all subsequent requests. As with `Unary`, servers should also allow this response definition to be unset. In -which case, they should set no response headers or trailers and return no response data. The returned conformance -payload should only contain the observed request information. +ignored in all subsequent requests. As with `Unary`, if an error is specified, servers should set request information into +the error details. + +Servers should also allow this response definition to be unset. In which case, they should set no response headers or +trailers and return no response data. The returned conformance payload should only contain the observed request information. **Pseudocode** ```text -while requests are being sent do the following +while requests are being sent { read a request from the stream capture the request body - if this is the first message being received then - save the response definition - -when requests are complete then - if a response definition was specified then - if response definition specifies valid response data then - - build a conformance payload - - set the request info and response data into the conformance payload + if this is the first message being received { + capture the response definition + } +} - else - build an error with the specified error and set the request info into the error details +when requests are complete { + if response definition specifies an error { + build an error with the specified error and set the request info into the error details set any response headers or trailers indicated in the response definition - else - build a conformance payload + } else { + build a conformance payload with the request info + + if a response definition exists { + set response data into the conformance payload - set the request info into the conformance payload + set any response headers or trailers indicated in the response definition + } + } +} sleep for any specified response delay -return the response or the error depending on which was specified +return the response with conformance payload or raise the error, depending on which was specified ``` For the full documentation on handling the `ClientStream` endpoint, click [here][clientstream]. @@ -189,30 +248,34 @@ read the request capture any request headers, the response definition and the actual request body as request info -if a response definition was specified then +if a response definition was specified { set any response headers or trailers on the response stream immediately send the headers/trailers on the stream so that they can be read by the client - loop over any response data specified + loop over any response data specified { build a conformance payload set the response data into the conformance payload - if this is the first response being sent then + if this is the first response being sent { set the request info into the conformance payload + } sleep for any specified response delay send the response + } - if an error was specified in the response definition then - if no responses have been sent yet - build an error with the specified error + if an error was specified in the response definition { + build an error with the specified error - set the following into the error details - * the received request - * any request headers + if no responses have been sent yet { + set the request info into the error details + } + raise the error + } +} ``` For the full documentation on handling the `ServerStream` endpoint, click [here][serverstream]. @@ -266,59 +329,90 @@ If the `full_duplex` field is false: * if the `response_delay_ms` duration is specified, the server should wait that long in between sending each response message. -**Pseudocode** +**Pseudocode for handling full duplex streams** ```text -while requests are being sent do the following +while requests are being sent { read a request from the stream capture the request body - if this is the first message being received then - save the response definition - save whether this is full duplex or half duplex + if this is the first message being received { + capture the response definition + capture full_duplex value (`true`) - if a response definition was specified then - set any response headers or trailers on the response stream + if a response definition was specified { + set any response headers or trailers on the response stream - if full duplex then - immediately send the headers/trailers on the stream so that they can be read by the client + immediately send the headers/trailers on the stream so that they can be read by the client + } + } - if full duplex then - if response data was specified then - build a conformance payload + if response data was specified { + build a conformance payload - set the response data into the conformance payload + set the response data into the conformance payload - if this is the first response being sent then - set the request info into the conformance payload - - sleep for any specified response delay + if this is the first response being sent { + set the request info into the conformance payload + } + + sleep for any specified response delay - send a response + send a response + } +} - if half duplex and if response data was specified then - immediately send the headers/trailers on the stream so that they can be read by the client +if an error was specified in the response definition { + build an error with the specified error - loop over any response data specified - build a conformance payload + if no responses have been sent yet { + set the request info into the error details + } + raise the error +} +``` - set the response data into the conformance payload +**Pseudocode for handling half duplex streams** - if this is the first response being sent then +```text +while requests are being sent { + read a request from the stream + + capture the request body + + if this is the first message being received { + capture the response definition + capture full_duplex value (`false`) + } +} + +if response data was specified { + immediately send the headers/trailers on the stream so that they can be read by the client + + loop over any response data specified { + build a conformance payload + + set the response data into the conformance payload + + if this is the first response being sent { set the request info into the conformance payload + } - sleep for any specified response delay + sleep for any specified response delay - send the response + send the response + } +} - if an error was specified in the response definition then - if no responses have been sent yet - build an Error object with the specified error +if an error was specified in the response definition { + build an error with the specified error - set the following into the error details - * the received request - * any request headers + if no responses have been sent yet { + set the request info into the error details + } + raise the error +} ``` For the full documentation on handling the `BidiStream` endpoint, click [here][bidistream]. @@ -332,6 +426,11 @@ For examples, check out the following: [connect-es-conformance]: https://github.com/connectrpc/connect-es/tree/main/packages/connect-conformance [server-reference-impl]: https://github.com/connectrpc/conformance/blob/main/internal/app/referenceserver/impl.go +[conformanceservice]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService +[conformancepayload]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformancePayload +[requestinfo]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformancePayload.RequestInfo +[error]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.Error +[any]: https://buf.build/protocolbuffers/wellknowntypes/docs/main:google.protobuf#google.protobuf.Any [unary]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.Unary [idempotentunary]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.IdempotentUnary [unimplemented]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService.Unimplemented diff --git a/internal/gen/proto/go/connectrpc/conformance/v1/server_compat.pb.go b/internal/gen/proto/go/connectrpc/conformance/v1/server_compat.pb.go index d6f76510..71ef025c 100644 --- a/internal/gen/proto/go/connectrpc/conformance/v1/server_compat.pb.go +++ b/internal/gen/proto/go/connectrpc/conformance/v1/server_compat.pb.go @@ -53,13 +53,21 @@ type ServerCompatRequest struct { sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields - // The protocol that will be used. + // Signals to the server that it must support at least this protocol. Note + // that it is fine to support others. + // For example if `PROTOCOL_CONNECT` is specified, the server _must_ support + // at least Connect, but _may_ also support gRPC or gRPC-web. Protocol Protocol `protobuf:"varint,1,opt,name=protocol,proto3,enum=connectrpc.conformance.v1.Protocol" json:"protocol,omitempty"` - // The HTTP version that will be used. + // Signals to the server the minimum HTTP version to support. As with + // `protocol`, it is fine to support other versions. For example, if + // `HTTP_VERSION_2` is specified, the server _must_ support HTTP/2, but _may_ also + // support HTTP/1.1 or HTTP/3. HttpVersion HTTPVersion `protobuf:"varint,2,opt,name=http_version,json=httpVersion,proto3,enum=connectrpc.conformance.v1.HTTPVersion" json:"http_version,omitempty"` - // If true, generate a self-signed cert and include it in the - // ServerCompatResponse along with the actual port. Clients - // will be configured to trust this cert when connecting. + // If true, generate a certificate that clients will be configured to trust + // when connecting and return it in the `pem_cert` field of the `ServerCompatResponse`. + // The certificate can be any TLS certificate where the subject matches the + // value sent back in the `host` field of the `ServerCompatResponse`. + // Self-signed certificates (and `localhost` as the subject) are allowed. // If false, the server should not use TLS and instead use // a plain-text/unencrypted socket. UseTls bool `protobuf:"varint,4,opt,name=use_tls,json=useTls,proto3" json:"use_tls,omitempty"` @@ -149,12 +157,15 @@ type ServerCompatResponse struct { sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields - // The host where the server is running. + // The host where the server is running. This should usually be `127.0.0.1`, + // unless your program actually starts a remote server to which the client + // should connect. Host string `protobuf:"bytes,1,opt,name=host,proto3" json:"host,omitempty"` // The port where the server is listening. Port uint32 `protobuf:"varint,2,opt,name=port,proto3" json:"port,omitempty"` - // The server's PEM-encoded certificate, so the - // client can verify it when connecting via TLS. + // The TLS certificate, in PEM format, if `use_tls` was set + // to `true`. Clients will verify this certificate when connecting via TLS. + // If `use_tls` was set to `false`, this should always be empty. PemCert []byte `protobuf:"bytes,3,opt,name=pem_cert,json=pemCert,proto3" json:"pem_cert,omitempty"` } diff --git a/proto/connectrpc/conformance/v1/server_compat.proto b/proto/connectrpc/conformance/v1/server_compat.proto index 8d5b06e6..9e81b831 100644 --- a/proto/connectrpc/conformance/v1/server_compat.proto +++ b/proto/connectrpc/conformance/v1/server_compat.proto @@ -33,13 +33,21 @@ import "connectrpc/conformance/v1/config.proto"; // When testing multiple configurations, multiple test processes // will be started, each with different properties. message ServerCompatRequest { - // The protocol that will be used. + // Signals to the server that it must support at least this protocol. Note + // that it is fine to support others. + // For example if `PROTOCOL_CONNECT` is specified, the server _must_ support + // at least Connect, but _may_ also support gRPC or gRPC-web. Protocol protocol = 1; - // The HTTP version that will be used. + // Signals to the server the minimum HTTP version to support. As with + // `protocol`, it is fine to support other versions. For example, if + // `HTTP_VERSION_2` is specified, the server _must_ support HTTP/2, but _may_ also + // support HTTP/1.1 or HTTP/3. HTTPVersion http_version = 2; - // If true, generate a self-signed cert and include it in the - // ServerCompatResponse along with the actual port. Clients - // will be configured to trust this cert when connecting. + // If true, generate a certificate that clients will be configured to trust + // when connecting and return it in the `pem_cert` field of the `ServerCompatResponse`. + // The certificate can be any TLS certificate where the subject matches the + // value sent back in the `host` field of the `ServerCompatResponse`. + // Self-signed certificates (and `localhost` as the subject) are allowed. // If false, the server should not use TLS and instead use // a plain-text/unencrypted socket. bool use_tls = 4; @@ -58,11 +66,14 @@ message ServerCompatRequest { // The outcome of one ServerCompatRequest. message ServerCompatResponse { - // The host where the server is running. + // The host where the server is running. This should usually be `127.0.0.1`, + // unless your program actually starts a remote server to which the client + // should connect. string host = 1; // The port where the server is listening. uint32 port = 2; - // The server's PEM-encoded certificate, so the - // client can verify it when connecting via TLS. + // The TLS certificate, in PEM format, if `use_tls` was set + // to `true`. Clients will verify this certificate when connecting via TLS. + // If `use_tls` was set to `false`, this should always be empty. bytes pem_cert = 3; } From c8b2c44a2fc33f53382a73a546a08eb9fb4654e8 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Thu, 29 Feb 2024 15:30:37 -0500 Subject: [PATCH 12/14] Links --- docs/testing_servers.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index 93e73b1c..9e40808d 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -4,21 +4,21 @@ The conformance suite provides the ability to run conformance tests against a se involves the following steps: 1. Defining any configuration for what your server supports. For more information on how to do this, see the docs for [configuring and running tests](./configuring_and_running_tests.md#configuration-files). -2. Writing an executable file that can read `ServerCompatRequest` messages from `stdin` (and write to `stdout` -- see Step 4). +2. Writing an executable file that can read [`ServerCompatRequest`][servercompatrequest] messages from `stdin` (and write to `stdout` -- see Step 4). 3. Starting your server according to the values in the request message. -4. Writing a `ServerCompatResponse` about the running server to `stdout`. +4. Writing a [`ServerCompatResponse`][servercompatresponse] about the running server to `stdout`. 5. Implementing the [ConformanceService][conformanceservice] endpoints to handle requests from the reference client. ## Starting your server When the conformance runner is executed for a server-under-test, the runner will analyze the configuration you've specified -and will use that information to build a `ServerCompatRequest`. This request is serialized to bytes and written to `stdin`. +and will use that information to build a [`ServerCompatRequest`][servercompatrequest]. This request is serialized to bytes and written to `stdin`. It is then up to the executable file you created as part of Step 2 to read this message and start your server. The messages written to `stdin` are size-delimited. This means that first you will need to read a fixed four-byte preface, which returns a network-byte-order (i.e. big-endian) 32-bit integer. This integer represents the size of the actual message. After this value is read, you should then read the number of bytes it specifies and then unmarshal those -bytes into a `ServerCompatRequest`. +bytes into a [`ServerCompatRequest`][servercompatrequest]. This message will contain all the details necessary for your implementation to start its server-under-test. Note that the request does not specify a port to listen on. Implementations should instead pick an available ephemeral port @@ -33,9 +33,9 @@ Fields in the request are: other versions. * `use_tls` which specifies whether your server should generate a certificate that clients will be configured to trust when connecting. The certificate can be any TLS certificate where the subject matches the value sent back in - the `host` field of the `ServerCompatResponse`. Self-signed certificates (and `localhost` as the subject) are allowed. + the `host` field of the [`ServerCompatResponse`][servercompatresponse]. Self-signed certificates (and `localhost` as the subject) are allowed. If `true`, the generated certificate should be returned in the `pem_cert` field of the - `ServerCompatResponse`. If this is set to `false`, the server should not use TLS and instead use a plaintext/unencrypted socket. + [`ServerCompatResponse`][servercompatresponse]. If this is set to `false`, the server should not use TLS and instead use a plaintext/unencrypted socket. * `client_tls_cert` which represents a PEM-encoded certificate that, if provided, clients will use to authenticate themselves. If specified, servers should require that clients provide certificates and they should ensure the presented certificate is valid. Note that this will always be empty if `use_tls` is `false`. @@ -43,10 +43,10 @@ Fields in the request are: any message from a client that is larger than the size indicated. Using the values in the request, you can then start your server implementation. Once started, your implementation should -build a `ServerCompatResponse` message. This will provide the conformance runner with details about your running server. +build a [`ServerCompatResponse`][servercompatresponse] message. This will provide the conformance runner with details about your running server. Once built, you should then write the response message to `stdout` using the same size-delimited algorithm described above. First, -write a network-encoded 32-bit integer indicating the size of the `ServerCompatResponse` message. Then, serialize the +write a network-encoded 32-bit integer indicating the size of the [`ServerCompatResponse`][servercompatresponse] message. Then, serialize the response to bytes and write that to `stdout`. Fields in the response are: @@ -428,6 +428,8 @@ For examples, check out the following: [server-reference-impl]: https://github.com/connectrpc/conformance/blob/main/internal/app/referenceserver/impl.go [conformanceservice]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformanceService [conformancepayload]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformancePayload +[servercompatrequest]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ServerCompatRequest +[servercompatresponse]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ServerCompatResponse [requestinfo]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ConformancePayload.RequestInfo [error]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.Error [any]: https://buf.build/protocolbuffers/wellknowntypes/docs/main:google.protobuf#google.protobuf.Any From a2d05389eb77534e3d52b8c003b9a5903cdbc754 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Thu, 29 Feb 2024 15:32:14 -0500 Subject: [PATCH 13/14] Links --- docs/testing_servers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/testing_servers.md b/docs/testing_servers.md index 9e40808d..2f2a8f29 100644 --- a/docs/testing_servers.md +++ b/docs/testing_servers.md @@ -7,7 +7,7 @@ involves the following steps: 2. Writing an executable file that can read [`ServerCompatRequest`][servercompatrequest] messages from `stdin` (and write to `stdout` -- see Step 4). 3. Starting your server according to the values in the request message. 4. Writing a [`ServerCompatResponse`][servercompatresponse] about the running server to `stdout`. -5. Implementing the [ConformanceService][conformanceservice] endpoints to handle requests from the reference client. +5. Implementing the [`ConformanceService`][conformanceservice] endpoints to handle requests from the reference client. ## Starting your server @@ -65,7 +65,7 @@ implementation written in Connect-Go and will use this client to issue requests will read the server's responses and return them to the conformance runner. So, all you need to do from a server implementation standpoint is handle the requests accordingly. -The [ConformanceService][conformanceservice] defines a series of endpoints that are meant to exercise all types of RPCs. The details +The [`ConformanceService`][conformanceservice] defines a series of endpoints that are meant to exercise all types of RPCs. The details in the requests will specify various attributes that the server should handle, such as determining response headers and trailers to return, any errors to throw, and any data to respond with. All request types contain a response definition which is used to instruct the server how to respond to the request. This response definition can also be unset entirely From 1df7c23fd5a5c69154da8463bd679faac060a114 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Thu, 29 Feb 2024 15:40:28 -0500 Subject: [PATCH 14/14] Links --- .../conformance/v1/server_compat.proto | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/proto/connectrpc/conformance/v1/server_compat.proto b/proto/connectrpc/conformance/v1/server_compat.proto index 9e81b831..5c2836c8 100644 --- a/proto/connectrpc/conformance/v1/server_compat.proto +++ b/proto/connectrpc/conformance/v1/server_compat.proto @@ -33,19 +33,19 @@ import "connectrpc/conformance/v1/config.proto"; // When testing multiple configurations, multiple test processes // will be started, each with different properties. message ServerCompatRequest { - // Signals to the server that it must support at least this protocol. Note + // Signals to the server that it must support at least this protocol. Note // that it is fine to support others. - // For example if `PROTOCOL_CONNECT` is specified, the server _must_ support + // For example if `PROTOCOL_CONNECT` is specified, the server _must_ support // at least Connect, but _may_ also support gRPC or gRPC-web. Protocol protocol = 1; - // Signals to the server the minimum HTTP version to support. As with - // `protocol`, it is fine to support other versions. For example, if + // Signals to the server the minimum HTTP version to support. As with + // `protocol`, it is fine to support other versions. For example, if // `HTTP_VERSION_2` is specified, the server _must_ support HTTP/2, but _may_ also // support HTTP/1.1 or HTTP/3. HTTPVersion http_version = 2; - // If true, generate a certificate that clients will be configured to trust - // when connecting and return it in the `pem_cert` field of the `ServerCompatResponse`. - // The certificate can be any TLS certificate where the subject matches the + // If true, generate a certificate that clients will be configured to trust + // when connecting and return it in the `pem_cert` field of the `ServerCompatResponse`. + // The certificate can be any TLS certificate where the subject matches the // value sent back in the `host` field of the `ServerCompatResponse`. // Self-signed certificates (and `localhost` as the subject) are allowed. // If false, the server should not use TLS and instead use @@ -66,14 +66,14 @@ message ServerCompatRequest { // The outcome of one ServerCompatRequest. message ServerCompatResponse { - // The host where the server is running. This should usually be `127.0.0.1`, - // unless your program actually starts a remote server to which the client + // The host where the server is running. This should usually be `127.0.0.1`, + // unless your program actually starts a remote server to which the client // should connect. string host = 1; // The port where the server is listening. uint32 port = 2; - // The TLS certificate, in PEM format, if `use_tls` was set - // to `true`. Clients will verify this certificate when connecting via TLS. + // The TLS certificate, in PEM format, if `use_tls` was set + // to `true`. Clients will verify this certificate when connecting via TLS. // If `use_tls` was set to `false`, this should always be empty. bytes pem_cert = 3; }