From cf57e5069c2faf35eb96f982af223171328cd675 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Fri, 9 Feb 2024 13:50:21 -0500
Subject: [PATCH 01/22] Docs

---
 docs/authoring_test_cases.md                  | 37 +++++++++++++++++++
 .../testsuites/data/client_message_size.yaml  |  2 +
 2 files changed, 39 insertions(+)
 create mode 100644 docs/authoring_test_cases.md

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
new file mode 100644
index 00000000..394e0ceb
--- /dev/null
+++ b/docs/authoring_test_cases.md
@@ -0,0 +1,37 @@
+## Authoring test cases
+
+Test cases for the conformance runner are configured in YAML files and are located [here](testsuites). The basic structure is 
+that each file represents a single suite of tests. A suite can be configured with various directives which will apply to all
+tests within that suite.
+
+For example, you can limit a suite to only run via the Connect and gRPC-web protocols using:
+
+```yaml
+relevantProtocols:
+  - PROTOCOL_CONNECT
+  - PROTOCOL_GRPC_WEB
+```
+
+or you can limit a suite only to the JSON codec:
+
+```yaml
+relevantCodecs:
+  - CODEC_JSON
+```
+
+For a full list of the available configurations for a test suite, see the [Protobuf definition](suite-proto) in the BSR.
+
+Authoring new test cases involves adding either a new file if a new suite is required or added an additional test case to an 
+existing file/suite. Suites are loosely defined and are grouped based on various criteria. When deciding whether to create a new
+suite or add to an existing, the first factor is usually whether the tests apply to a single protocol or all of them. Tests that apply
+to a single protocol are located in files with the protocol as the prefix. For example, `connect_idempotency.yaml` contains tests that
+test GET requests for the Connect protocol only. The `grpc_web_client.yaml` contains client tests for the gRPC-web protocol. Files that
+are not prefixed with a protocol name apply to all protocols.
+
+
+This should describe the test suites folder and link to the YAML config schema by way of the relevant message in the BSR generated docs.
+
+This should describe how the expected responses is auto-generated based on the request details. It usually only needs to be explicitly provided for exception test cases.
+
+[testsuites]: https://github.com/connectrpc/conformance/blob/main/internal/app/connectconformance/testsuites/data
+[suite-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/suite.proto
diff --git a/internal/app/connectconformance/testsuites/data/client_message_size.yaml b/internal/app/connectconformance/testsuites/data/client_message_size.yaml
index 4d073352..f67435bb 100644
--- a/internal/app/connectconformance/testsuites/data/client_message_size.yaml
+++ b/internal/app/connectconformance/testsuites/data/client_message_size.yaml
@@ -1,6 +1,8 @@
 name: Client Message Size
 mode: TEST_MODE_CLIENT
 reliesOnMessageReceiveLimit: true
+# We only test client message size interactions over the proto codec because
+# the test runner bases request size based on the binary size.
 relevantCodecs:
   - CODEC_PROTO
 testCases:

From c9d98888fb07d6ab7aee2a05d7a20f2b00dda9f9 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Mon, 12 Feb 2024 12:44:33 -0500
Subject: [PATCH 02/22] Docs

---
 docs/authoring_test_cases.md | 41 ++++++++++++++++++++++++++++++++++--
 1 file changed, 39 insertions(+), 2 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 394e0ceb..8b9a3ae7 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -28,10 +28,47 @@ to a single protocol are located in files with the protocol as the prefix. For e
 test GET requests for the Connect protocol only. The `grpc_web_client.yaml` contains client tests for the gRPC-web protocol. Files that
 are not prefixed with a protocol name apply to all protocols.
 
+### Naming conventions
 
-This should describe the test suites folder and link to the YAML config schema by way of the relevant message in the BSR generated docs.
+The suites and tests within follow a loose naming convention:
 
-This should describe how the expected responses is auto-generated based on the request details. It usually only needs to be explicitly provided for exception test cases.
+#### Test files
+
+Test files should be named according to the suite inside and the general functionality being tested. In addition:
+
+* If a suite applies only to a certain protocol, the file name should be prefixed with that protocol. If the suite applies
+  to all protocols, this can be omitted.
+* If a suite only contains client or server tests, the file name prefix should include `client` or `server`. If the suite is
+  for both client and server, this can be omitted.
+
+
+For example: `connect_idempotency.yaml` contains a suite for testing idempotency (`GET` support) for the Connect protocol only.
+The `client_message_size.yaml` file contains a suite of client tests only across all protocols. The `connect_client_code_to_http_code.yaml` 
+file is comprised of client tests for the Connect protocol only.
+
+
+#### Tests
+
+Tests should be named according to the following convention:
+
+`{stream type}/{test_description}`
+
+In the case of Bidi tests, you should also add `full_duplex` or `half_duplex` to the test name. For example:
+
+`unary/`
+`server-stream/`
+`bidi-stream/full-duplex/`
+
+The above conventions allow for a more granular control over running tests via the conformance runner. For example, you can run
+only unary tests within a file xxxxx, you can only run suites that apply to the Connect protocol, or you can specify known failing tests for xxxx.
+
+### Expected responses
+
+The expected response for a test is auto-generated based on the request details. The conformance runner will determine what the response 
+should be by the values specified in the test suite and individual test cases. However, you have the ability to explicitly specify your 
+own expected response directly in the test itself. To do so, simply define an `expectedResponse` block for your test case and this will
+override the auto-generated expected response in the test runner. This typically only needs done for exception test cases. For an example,
+take a look at xxxxxxxxxxxxxxxx.
 
 [testsuites]: https://github.com/connectrpc/conformance/blob/main/internal/app/connectconformance/testsuites/data
 [suite-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/suite.proto

From 20d31ff93901743e46386bdcec07ddeba7b79cd0 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Tue, 13 Feb 2024 13:36:44 -0500
Subject: [PATCH 03/22] Docs

---
 docs/authoring_test_cases.md | 84 ++++++++++++++++++------------------
 1 file changed, 42 insertions(+), 42 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 8b9a3ae7..81488616 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -1,38 +1,16 @@
 ## Authoring test cases
 
-Test cases for the conformance runner are configured in YAML files and are located [here](testsuites). The basic structure is 
-that each file represents a single suite of tests. A suite can be configured with various directives which will apply to all
-tests within that suite.
-
-For example, you can limit a suite to only run via the Connect and gRPC-web protocols using:
-
-```yaml
-relevantProtocols:
-  - PROTOCOL_CONNECT
-  - PROTOCOL_GRPC_WEB
-```
-
-or you can limit a suite only to the JSON codec:
-
-```yaml
-relevantCodecs:
-  - CODEC_JSON
-```
-
-For a full list of the available configurations for a test suite, see the [Protobuf definition](suite-proto) in the BSR.
+Test cases for the conformance runner are configured in YAML files and are located [here](testsuites). Each file 
+represents a single suite of tests. 
 
 Authoring new test cases involves adding either a new file if a new suite is required or added an additional test case to an 
-existing file/suite. Suites are loosely defined and are grouped based on various criteria. When deciding whether to create a new
-suite or add to an existing, the first factor is usually whether the tests apply to a single protocol or all of them. Tests that apply
-to a single protocol are located in files with the protocol as the prefix. For example, `connect_idempotency.yaml` contains tests that
-test GET requests for the Connect protocol only. The `grpc_web_client.yaml` contains client tests for the gRPC-web protocol. Files that
-are not prefixed with a protocol name apply to all protocols.
+existing file/suite. Suites are loosely defined and are grouped based on various criteria. 
 
 ### Naming conventions
 
-The suites and tests within follow a loose naming convention:
+Test suites and their tests within follow a loose naming convention. 
 
-#### Test files
+#### Test files (Suites)
 
 Test files should be named according to the suite inside and the general functionality being tested. In addition:
 
@@ -41,34 +19,56 @@ Test files should be named according to the suite inside and the general functio
 * If a suite only contains client or server tests, the file name prefix should include `client` or `server`. If the suite is
   for both client and server, this can be omitted.
 
+For example:
+* `connect_with_get.yaml` contains tests that test GET requests for the Connect protocol only.
+* `grpc_web_client.yaml` contains client tests for the gRPC-web protocol. 
+* `client_message_size.yaml` file contains a suite of client tests only across all protocols. 
+
+#### Tests
 
-For example: `connect_idempotency.yaml` contains a suite for testing idempotency (`GET` support) for the Connect protocol only.
-The `client_message_size.yaml` file contains a suite of client tests only across all protocols. The `connect_client_code_to_http_code.yaml` 
-file is comprised of client tests for the Connect protocol only.
+Test names should be hyphen-delimited. If a suite contains tests of multiple stream types, the test name should be 
+prefixed with the stream type and a backslash (`/`).
 
+ > In the case of Bidi tests, you should also add `full_duplex` or `half_duplex` to the test name. 
 
-#### Tests
+ For example:
+
+`unary/error`
+`server-stream/error-with-responses`
+`bidi-stream/full-duplex/stream-error-returns-success-http-code`
 
-Tests should be named according to the following convention:
+If a stream contains tests for just a single stream type, the stream type can be omitted from the test name.
 
-`{stream type}/{test_description}`
+These conventions allow for more granular control over running tests via the conformance runner, such as only running tests
+for a specific protocol or only running the unary tests within a suite.
 
-In the case of Bidi tests, you should also add `full_duplex` or `half_duplex` to the test name. For example:
+### Configuring test suites
 
-`unary/`
-`server-stream/`
-`bidi-stream/full-duplex/`
+A suite can be configured with various directives which will apply to all tests within that suite. For example, you 
+can limit a suite to only run via the Connect and gRPC-web protocols using:
+
+```yaml
+relevantProtocols:
+  - PROTOCOL_CONNECT
+  - PROTOCOL_GRPC_WEB
+```
 
-The above conventions allow for a more granular control over running tests via the conformance runner. For example, you can run
-only unary tests within a file xxxxx, you can only run suites that apply to the Connect protocol, or you can specify known failing tests for xxxx.
+or you can limit a suite only to the JSON codec:
+
+```yaml
+relevantCodecs:
+  - CODEC_JSON
+```
+
+For a full list of the available configurations for a test suite, see the [Protobuf definition](suite-proto) in the BSR.
 
 ### Expected responses
 
 The expected response for a test is auto-generated based on the request details. The conformance runner will determine what the response 
 should be by the values specified in the test suite and individual test cases. However, you have the ability to explicitly specify your 
-own expected response directly in the test itself. To do so, simply define an `expectedResponse` block for your test case and this will
-override the auto-generated expected response in the test runner. This typically only needs done for exception test cases. For an example,
-take a look at xxxxxxxxxxxxxxxx.
+own expected response directly in the test definition itself. To do so, simply define an `expectedResponse` block for your test case and this will
+override the auto-generated expected response in the test runner. This typically only needs done for exception test cases. For examples,
+search the [test suites](testsuites) directory for `expectedResponse`.
 
 [testsuites]: https://github.com/connectrpc/conformance/blob/main/internal/app/connectconformance/testsuites/data
 [suite-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/suite.proto

From 445679e8645ac5542706ffd7ba80859522191d60 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Tue, 13 Feb 2024 13:39:05 -0500
Subject: [PATCH 04/22] Formatting

---
 docs/authoring_test_cases.md | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 81488616..8ef742c6 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -1,16 +1,16 @@
-## Authoring test cases
+# Authoring test cases
 
 Test cases for the conformance runner are configured in YAML files and are located [here](testsuites). Each file 
 represents a single suite of tests. 
 
 Authoring new test cases involves adding either a new file if a new suite is required or added an additional test case to an 
-existing file/suite. Suites are loosely defined and are grouped based on various criteria. 
+existing file/suite. 
 
-### Naming conventions
+## Naming conventions
 
 Test suites and their tests within follow a loose naming convention. 
 
-#### Test files (Suites)
+### Test files (Suites)
 
 Test files should be named according to the suite inside and the general functionality being tested. In addition:
 
@@ -24,7 +24,7 @@ For example:
 * `grpc_web_client.yaml` contains client tests for the gRPC-web protocol. 
 * `client_message_size.yaml` file contains a suite of client tests only across all protocols. 
 
-#### Tests
+### Tests
 
 Test names should be hyphen-delimited. If a suite contains tests of multiple stream types, the test name should be 
 prefixed with the stream type and a backslash (`/`).
@@ -42,7 +42,7 @@ If a stream contains tests for just a single stream type, the stream type can be
 These conventions allow for more granular control over running tests via the conformance runner, such as only running tests
 for a specific protocol or only running the unary tests within a suite.
 
-### Configuring test suites
+## Configuring test suites
 
 A suite can be configured with various directives which will apply to all tests within that suite. For example, you 
 can limit a suite to only run via the Connect and gRPC-web protocols using:
@@ -62,7 +62,7 @@ relevantCodecs:
 
 For a full list of the available configurations for a test suite, see the [Protobuf definition](suite-proto) in the BSR.
 
-### Expected responses
+## Expected responses
 
 The expected response for a test is auto-generated based on the request details. The conformance runner will determine what the response 
 should be by the values specified in the test suite and individual test cases. However, you have the ability to explicitly specify your 
@@ -70,5 +70,5 @@ own expected response directly in the test definition itself. To do so, simply d
 override the auto-generated expected response in the test runner. This typically only needs done for exception test cases. For examples,
 search the [test suites](testsuites) directory for `expectedResponse`.
 
-[testsuites]: https://github.com/connectrpc/conformance/blob/main/internal/app/connectconformance/testsuites/data
+[testsuites]: https://github.com/connectrpc/conformance/tree/main/internal/app/connectconformance/testsuites/data
 [suite-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/suite.proto

From ab2b497b7948ed346ab038007afe274c50c6660c Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Tue, 13 Feb 2024 13:41:32 -0500
Subject: [PATCH 05/22] Formatting

---
 docs/authoring_test_cases.md | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 8ef742c6..d93216e7 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -29,13 +29,14 @@ For example:
 Test names should be hyphen-delimited. If a suite contains tests of multiple stream types, the test name should be 
 prefixed with the stream type and a backslash (`/`).
 
+ > [!INFO]  
  > In the case of Bidi tests, you should also add `full_duplex` or `half_duplex` to the test name. 
 
  For example:
 
-`unary/error`
-`server-stream/error-with-responses`
-`bidi-stream/full-duplex/stream-error-returns-success-http-code`
+* `unary/error`
+* `server-stream/error-with-responses`
+* `bidi-stream/full-duplex/stream-error-returns-success-http-code`
 
 If a stream contains tests for just a single stream type, the stream type can be omitted from the test name.
 

From 6bd87314949501171743bec37fe2a219a417aa84 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Tue, 13 Feb 2024 13:44:12 -0500
Subject: [PATCH 06/22] Formatting

---
 docs/authoring_test_cases.md | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index d93216e7..d0d831af 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -29,7 +29,7 @@ For example:
 Test names should be hyphen-delimited. If a suite contains tests of multiple stream types, the test name should be 
 prefixed with the stream type and a backslash (`/`).
 
- > [!INFO]  
+ > [!NOTE]  
  > In the case of Bidi tests, you should also add `full_duplex` or `half_duplex` to the test name. 
 
  For example:
@@ -38,7 +38,7 @@ prefixed with the stream type and a backslash (`/`).
 * `server-stream/error-with-responses`
 * `bidi-stream/full-duplex/stream-error-returns-success-http-code`
 
-If a stream contains tests for just a single stream type, the stream type can be omitted from the test name.
+If a suite contains tests for just a single stream type, the stream type can be omitted from the test name.
 
 These conventions allow for more granular control over running tests via the conformance runner, such as only running tests
 for a specific protocol or only running the unary tests within a suite.
@@ -67,9 +67,12 @@ For a full list of the available configurations for a test suite, see the [Proto
 
 The expected response for a test is auto-generated based on the request details. The conformance runner will determine what the response 
 should be by the values specified in the test suite and individual test cases. However, you have the ability to explicitly specify your 
-own expected response directly in the test definition itself. To do so, simply define an `expectedResponse` block for your test case and this will
-override the auto-generated expected response in the test runner. This typically only needs done for exception test cases. For examples,
-search the [test suites](testsuites) directory for `expectedResponse`.
+own expected response directly in the test definition itself. 
+
+To do so, simply define an `expectedResponse` block for your test case and this will
+override the auto-generated expected response in the test runner. This typically only needs done for exception test cases. 
+
+For examples, search the [test suites](testsuites) directory for the word `expectedResponse`.
 
 [testsuites]: https://github.com/connectrpc/conformance/tree/main/internal/app/connectconformance/testsuites/data
 [suite-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/suite.proto

From 7979237eaac6fae229141711641bfa7e8cf3a508 Mon Sep 17 00:00:00 2001
From: Steve Ayers <smaye81@gmail.com>
Date: Tue, 13 Feb 2024 16:33:57 -0500
Subject: [PATCH 07/22] Update docs/authoring_test_cases.md

Co-authored-by: Joshua Humphries <2035234+jhump@users.noreply.github.com>
---
 docs/authoring_test_cases.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index d0d831af..f708a2e5 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -3,7 +3,7 @@
 Test cases for the conformance runner are configured in YAML files and are located [here](testsuites). Each file 
 represents a single suite of tests. 
 
-Authoring new test cases involves adding either a new file if a new suite is required or added an additional test case to an 
+Authoring new test cases involves either a new file, if a new suite is warranted, or an additional test case in an 
 existing file/suite. 
 
 ## Naming conventions

From 1898110f6c96589d0b03c5b7a57f0b1654ce0b4f Mon Sep 17 00:00:00 2001
From: Steve Ayers <smaye81@gmail.com>
Date: Tue, 13 Feb 2024 16:34:08 -0500
Subject: [PATCH 08/22] Update docs/authoring_test_cases.md

Co-authored-by: Joshua Humphries <2035234+jhump@users.noreply.github.com>
---
 docs/authoring_test_cases.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index f708a2e5..8260b54a 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -30,7 +30,8 @@ Test names should be hyphen-delimited. If a suite contains tests of multiple str
 prefixed with the stream type and a backslash (`/`).
 
  > [!NOTE]  
- > In the case of Bidi tests, you should also add `full_duplex` or `half_duplex` to the test name. 
+ > In the case of Bidi tests with separate cases for half-duplex operation vs. full-duplex
+ > operation, you should also add `full_duplex` or `half_duplex` to the test name. 
 
  For example:
 

From cc906976a34e2badc90552aafbae3331ae5e4543 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 13:56:26 -0500
Subject: [PATCH 09/22] Docs

---
 docs/authoring_test_cases.md                  | 137 +++++++++++++++---
 internal/app/connectconformance/results.go    |   2 +
 .../connectconformance/test_case_library.go   |  17 ++-
 3 files changed, 131 insertions(+), 25 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 8260b54a..fd17137f 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -1,11 +1,81 @@
 # Authoring test cases
 
-Test cases for the conformance runner are configured in YAML files and are located [here](testsuites). Each file 
+Test cases for the conformance runner are configured in YAML files and are located [here][testsuites]. Each file 
 represents a single suite of tests. 
 
 Authoring new test cases involves either a new file, if a new suite is warranted, or an additional test case in an 
 existing file/suite. 
 
+For the Protobuf definitions for the conformance runner, see the [connectrpc repository][connectrpc-repo] in the BSR.
+
+## Test suites
+
+A suite represents a series of tests that all test the same general functionality (cancellation, timeouts, etc.). When
+defining a test suite in a YAML file, the only values that are required are the suite name (which should be unique across all suites)
+and at least one test case.
+
+Each suite can be configured with various directives which will apply to all tests within that suite. These directives
+can be used to constrain the tests run by the conformance runner or to signal various configurations to the runner that
+may be needed to execute the tests. The runner will use these directives to expand the tests in the suite into multiple
+permutations. This means that a single test defined in a suite will be run several times across various permutations.
+
+A rundown of the available directives:
+
+The below directives are used to constrain tests within a suite:
+
+* `mode` can be used to specify a suite as applying only to a specific mode (i.e. client or server). For example,
+  if you are writing a suite to target only clients under test, you would specify `mode: TEST_MODE_CLIENT` and the
+  tests in the suite will be run only when the `mode` specified to the runner on the command line is `client`. If not 
+  specified, tests are run regardless of the `mode` set on the command line.
+
+* `relevantProtocols` is used to limit tests only to a specific protocol, such as Connect, gRPC, or gRPC-web. If not
+  specified, tests are run for all protocols.
+
+* `relevantHttpVersions` is used to limit tests to certain HTTP versions, such as HTTP 1.1, HTTP/2, or HTTP/3.
+
+* `relevantCodecs` is used to limit tests to certain codec formats, such as JSON or binary.
+
+* `relevantCompressions` is used to limit tests to certain compression algorithms, such as **gzip**, **brotli**, or **snappy**.
+
+* `connectVersionMode` allows you to either require or ignore validation of the Connect version header or query param. This
+  should be left unspecified if the suite is agnostic to this validation behavior.
+
+The below `reliesOn` directives are used to signal to the test runner how the reference client or server should be
+configured when running tests:
+
+* `reliesOnTls` specifies that a suite relies on TLS. 
+
+* `reliesOnTlsClientCerts` specifies that the suite relies on the _client_ using TLS certificates to authenticate with 
+  the server. Note that if this is set to `true`, `reliesOnTls` must also be `true`.
+
+* `reliesOnConnectGet` specifies that the suite relies on the Connect GET protocol.
+
+* `reliesOnMessageReceiveLimit` specifies that the suite relies on support for limiting the size of received messages.
+  When `true`, the `mode` property must be set to indicate whether the client or server should support the limit.  
+
+## Test cases
+
+Test cases are specified in the `testCases` property of the suite. Each test case starts with the `request` property 
+which defines the request which will be sent to a client during the test run. Each `request` must specify the following
+fields:
+
+* `testName` - For naming conventions, see [below](#naming-conventions).
+* `service` - This will most likely be the fully-qualified path to the `ConformanceService`.
+* `method` - This is a string specifying the method on `service` that will be called.
+* `streamType` - One of `STREAM_TYPE_UNARY`, `STREAM_TYPE_CLIENT_STREAM`, `STREAM_TYPE_SERVER_STREAM`, 
+  `STREAM_TYPE_HALF_DUPLEX_BIDI_STREAM`, or `STREAM_TYPE_FULL_DUPLEX_BIDI_STREAM`.
+
+Once the above are specified, you can then define your request. For a full list of fields to specify in the request,
+see the [`ClientCompatRequest`][client-compat-proto] message in the Conformance Protobuf definitions.
+
+ > [!IMPORTANT]  
+ > The `ClientCompatRequest` message contains some fields that should _not_ be specified in test cases.
+ > These fields include:
+ > * Fields 1 through 8 in the message definition. These fields are automatically populated by the test runner.
+ >   If a test is specific to one of these values, it should instead be indicated in the directives for the test suite
+ >   itself.
+ > * Field 20 (`raw_request`). This field is only used by the reference client for sending anomalous requests.
+
 ## Naming conventions
 
 Test suites and their tests within follow a loose naming convention. 
@@ -44,36 +114,57 @@ If a suite contains tests for just a single stream type, the stream type can be
 These conventions allow for more granular control over running tests via the conformance runner, such as only running tests
 for a specific protocol or only running the unary tests within a suite.
 
-## Configuring test suites
+## Expected responses
 
-A suite can be configured with various directives which will apply to all tests within that suite. For example, you 
-can limit a suite to only run via the Connect and gRPC-web protocols using:
+The expected response for a test is auto-generated based on the request details. The conformance runner will determine 
+what the response should be according to the values specified in the test suite and individual test cases. 
 
-```yaml
-relevantProtocols:
-  - PROTOCOL_CONNECT
-  - PROTOCOL_GRPC_WEB
-```
+You also have the ability to explicitly specify your own expected response directly in the test definition itself. However, 
+this is typically only needed for exception test cases. If the expected response is mostly re-stating the response definition
+that appears in the requests, you should rely on the auto-generation if possible. Otherwise, specifying an expected response 
+can make the test YAML overly verbose and harder to read, write, and maintain. 
 
-or you can limit a suite only to the JSON codec:
+If the test induces behavior that prevents the server from sending or client from receiving the full response definition, it 
+will be necessary to define the expected response explicitly. Timeouts, cancellations, and exceeding message size limits are 
+good examples of this.
 
-```yaml
-relevantCodecs:
-  - CODEC_JSON
-```
+If you do need to specify an explicit response, simply define an `expectedResponse` block for your test case and this will
+override the auto-generated expected response in the test runner. 
 
-For a full list of the available configurations for a test suite, see the [Protobuf definition](suite-proto) in the BSR.
+To tests denoting an explicit response, search the [test suites](testsuites) directory for the word `expectedResponse`.
 
-## Expected responses
+## Example 
 
-The expected response for a test is auto-generated based on the request details. The conformance runner will determine what the response 
-should be by the values specified in the test suite and individual test cases. However, you have the ability to explicitly specify your 
-own expected response directly in the test definition itself. 
+Taking all of the above into account, here is an example test suite:
 
-To do so, simply define an `expectedResponse` block for your test case and this will
-override the auto-generated expected response in the test runner. This typically only needs done for exception test cases. 
-
-For examples, search the [test suites](testsuites) directory for the word `expectedResponse`.
+```yaml
+name: TLS Client Certs
+# This just does the basics with a client-cert, instead of running every test case with them.
+# TODO - Add unary and other stream type tests here also
+testCases:
+  - request:
+      testName: client-stream
+      service: connectrpc.conformance.v1.ConformanceService
+      method: ClientStream
+      streamType: STREAM_TYPE_CLIENT_STREAM
+      requestHeaders:
+        - name: X-Conformance-Test
+          value: ["Value1","Value2"]
+      requestMessages:
+        - "@type": type.googleapis.com/connectrpc.conformance.v1.ClientStreamRequest
+          responseDefinition:
+            responseHeaders:
+              - name: x-custom-header
+                value: ["foo"]
+            responseData: "dGVzdCByZXNwb25zZQ=="
+            responseTrailers:
+              - name: x-custom-trailer
+                value: ["bing"]
+        - "@type": type.googleapis.com/connectrpc.conformance.v1.ClientStreamRequest
+          requestData: "dGVzdCByZXNwb25zZQ=="
+```
 
 [testsuites]: https://github.com/connectrpc/conformance/tree/main/internal/app/connectconformance/testsuites/data
 [suite-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/suite.proto
+[client-compat-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/client_compat.proto
+[connectrpc-repo]: https://buf.build/connectrpc/conformance
diff --git a/internal/app/connectconformance/results.go b/internal/app/connectconformance/results.go
index 5963ea07..f35edda8 100644
--- a/internal/app/connectconformance/results.go
+++ b/internal/app/connectconformance/results.go
@@ -167,6 +167,8 @@ func (r *testResults) assert(
 	expected := definition.ExpectedResponse
 	var errs multiErrors
 
+	fmt.Println(expected)
+
 	errs = append(errs, checkError(expected.Error, actual.Error)...)
 	errs = append(errs, checkPayloads(expected.Payloads, actual.Payloads)...)
 
diff --git a/internal/app/connectconformance/test_case_library.go b/internal/app/connectconformance/test_case_library.go
index 533b288b..0183a0a1 100644
--- a/internal/app/connectconformance/test_case_library.go
+++ b/internal/app/connectconformance/test_case_library.go
@@ -160,7 +160,7 @@ func (lib *testCaseLibrary) expandSuite(suite *conformancev1.TestSuite, configCa
 							if _, ok := configCases[cfgCase]; ok {
 								namePrefix := generateTestCasePrefix(suite, cfgCase)
 								if err := lib.expandCases(cfgCase, namePrefix, suite.TestCases); err != nil {
-									return err
+									return fmt.Errorf("failed to expand test cases for suite %s: %w", suite.Name, err)
 								}
 							}
 						}
@@ -174,6 +174,18 @@ func (lib *testCaseLibrary) expandSuite(suite *conformancev1.TestSuite, configCa
 
 func (lib *testCaseLibrary) expandCases(cfgCase configCase, namePrefix []string, testCases []*conformancev1.TestCase) error {
 	for _, testCase := range testCases {
+		if testCase.Request.TestName == "" {
+			return errors.New("suite contains a test with no name")
+		}
+		if testCase.Request.Service == "" {
+			return fmt.Errorf("test %s has no service specified", testCase.Request.TestName)
+		}
+		if testCase.Request.Method == "" {
+			return fmt.Errorf("test %s has no method specified", testCase.Request.TestName)
+		}
+		if testCase.Request.StreamType == conformancev1.StreamType_STREAM_TYPE_UNSPECIFIED {
+			return fmt.Errorf("test %s has no stream type specified", testCase.Request.TestName)
+		}
 		if testCase.Request.StreamType != cfgCase.StreamType {
 			continue
 		}
@@ -441,7 +453,7 @@ func populateExpectedResponse(testCase *conformancev1.TestCase) error {
 	// message in this situation, where the response data value is some fixed string (such as "no response definition")
 	// and whose request info will still be present, but we expect it to indicate zero request messages.
 	if len(testCase.Request.RequestMessages) == 0 {
-		return nil
+		return fmt.Errorf("test %s has no request messages specified", testCase.Request.TestName)
 	}
 
 	switch testCase.Request.StreamType {
@@ -616,6 +628,7 @@ func populateExpectedStreamResponse(testCase *conformancev1.TestCase) error {
 	if err != nil {
 		return err
 	}
+	fmt.Println("on it")
 
 	definer, ok := concreteReq.(streamResponseDefiner)
 	if !ok {

From 72b40b126919edbdbf94e90a788c64960bdaf8b4 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 14:13:01 -0500
Subject: [PATCH 10/22] Example

---
 docs/authoring_test_cases.md | 59 +++++++++++++++++++++---------------
 1 file changed, 35 insertions(+), 24 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index fd17137f..15702a82 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -31,11 +31,14 @@ The below directives are used to constrain tests within a suite:
 * `relevantProtocols` is used to limit tests only to a specific protocol, such as Connect, gRPC, or gRPC-web. If not
   specified, tests are run for all protocols.
 
-* `relevantHttpVersions` is used to limit tests to certain HTTP versions, such as HTTP 1.1, HTTP/2, or HTTP/3.
+* `relevantHttpVersions` is used to limit tests to certain HTTP versions, such as HTTP 1.1, HTTP/2, or HTTP/3. If not
+  specified, tests are run for all HTTP versions.
 
-* `relevantCodecs` is used to limit tests to certain codec formats, such as JSON or binary.
+* `relevantCodecs` is used to limit tests to certain codec formats, such as JSON or binary. If not specified, tests are
+   run for all codecs.
 
-* `relevantCompressions` is used to limit tests to certain compression algorithms, such as **gzip**, **brotli**, or **snappy**.
+* `relevantCompressions` is used to limit tests to certain compression algorithms, such as **gzip**, **brotli**, or **snappy**. If not
+  specified, tests are run for all compressions.
 
 * `connectVersionMode` allows you to either require or ignore validation of the Connect version header or query param. This
   should be left unspecified if the suite is agnostic to this validation behavior.
@@ -43,15 +46,16 @@ The below directives are used to constrain tests within a suite:
 The below `reliesOn` directives are used to signal to the test runner how the reference client or server should be
 configured when running tests:
 
-* `reliesOnTls` specifies that a suite relies on TLS. 
+* `reliesOnTls` specifies that a suite relies on TLS. Defaults to `false`.
 
 * `reliesOnTlsClientCerts` specifies that the suite relies on the _client_ using TLS certificates to authenticate with 
-  the server. Note that if this is set to `true`, `reliesOnTls` must also be `true`.
+  the server. Note that if this is set to `true`, `reliesOnTls` must also be `true`. Defaults to `false`.
 
-* `reliesOnConnectGet` specifies that the suite relies on the Connect GET protocol.
+* `reliesOnConnectGet` specifies that the suite relies on the Connect GET protocol. Defaults to `false`.
 
 * `reliesOnMessageReceiveLimit` specifies that the suite relies on support for limiting the size of received messages.
-  When `true`, the `mode` property must be set to indicate whether the client or server should support the limit.  
+  When `true`, the `mode` property must be set to indicate whether the client or server should support the limit. Defaults
+  to `false`.
 
 ## Test cases
 
@@ -135,33 +139,40 @@ To tests denoting an explicit response, search the [test suites](testsuites) dir
 
 ## Example 
 
-Taking all of the above into account, here is an example test suite:
+Taking all of the above into account, here is an example test suite that verifies a server returns the specified headers
+and trailers. The below test will only run when `mode` is `server` and will be limited to the Connect protocol using the
+JSON format and `identity` compression. In addition, it will require the server verify the Connect version header.
 
 ```yaml
-name: TLS Client Certs
-# This just does the basics with a client-cert, instead of running every test case with them.
-# TODO - Add unary and other stream type tests here also
+name: Example Test Suite
+# Constrain to only servers under test.
+mode: TEST_MODE_SERVER
+# Constrain these tests to only run over the Connect protocol.
+relevantProtocols:
+  - PROTOCOL_CONNECT
+# Constrain these tests to only use 'identity' compression.
+relevantCompressions:
+  - COMPRESSION_IDENTITY
+# Constrain these tests to only use the JSON codec.
+relevantCodecs:
+  - CODEC_JSON
+connectVersionMode:  CONNECT_VERSION_MODE_REQUIRE
 testCases:
   - request:
-      testName: client-stream
+      testName: returns-headers-and-trailers
       service: connectrpc.conformance.v1.ConformanceService
-      method: ClientStream
-      streamType: STREAM_TYPE_CLIENT_STREAM
-      requestHeaders:
-        - name: X-Conformance-Test
-          value: ["Value1","Value2"]
+      method: Unary
+      streamType: STREAM_TYPE_UNARY
       requestMessages:
-        - "@type": type.googleapis.com/connectrpc.conformance.v1.ClientStreamRequest
+        - "@type": type.googleapis.com/connectrpc.conformance.v1.UnaryRequest
           responseDefinition:
             responseHeaders:
-              - name: x-custom-header
-                value: ["foo"]
+            - name: x-custom-header
+              value: ["foo"]
             responseData: "dGVzdCByZXNwb25zZQ=="
             responseTrailers:
-              - name: x-custom-trailer
-                value: ["bing"]
-        - "@type": type.googleapis.com/connectrpc.conformance.v1.ClientStreamRequest
-          requestData: "dGVzdCByZXNwb25zZQ=="
+            - name: x-custom-trailer
+              value: ["bing"]
 ```
 
 [testsuites]: https://github.com/connectrpc/conformance/tree/main/internal/app/connectconformance/testsuites/data

From 38cdc8982ab4ad6c4dad09ad11912af673ed22ff Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 14:16:19 -0500
Subject: [PATCH 11/22] Docs

---
 docs/authoring_test_cases.md | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 15702a82..2aa50c41 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -19,22 +19,20 @@ can be used to constrain the tests run by the conformance runner or to signal va
 may be needed to execute the tests. The runner will use these directives to expand the tests in the suite into multiple
 permutations. This means that a single test defined in a suite will be run several times across various permutations.
 
-A rundown of the available directives:
-
 The below directives are used to constrain tests within a suite:
 
 * `mode` can be used to specify a suite as applying only to a specific mode (i.e. client or server). For example,
   if you are writing a suite to target only clients under test, you would specify `mode: TEST_MODE_CLIENT` and the
   tests in the suite will be run only when the `mode` specified to the runner on the command line is `client`. If not 
-  specified, tests are run regardless of the `mode` set on the command line.
+  specified in a suite, tests are run regardless of the command line `mode`.
 
-* `relevantProtocols` is used to limit tests only to a specific protocol, such as Connect, gRPC, or gRPC-web. If not
+* `relevantProtocols` is used to limit tests only to a specific protocol, such as **Connect**, **gRPC**, or **gRPC-web**. If not
   specified, tests are run for all protocols.
 
-* `relevantHttpVersions` is used to limit tests to certain HTTP versions, such as HTTP 1.1, HTTP/2, or HTTP/3. If not
+* `relevantHttpVersions` is used to limit tests to certain HTTP versions, such as **HTTP 1.1**, **HTTP/2**, or **HTTP/3**. If not
   specified, tests are run for all HTTP versions.
 
-* `relevantCodecs` is used to limit tests to certain codec formats, such as JSON or binary. If not specified, tests are
+* `relevantCodecs` is used to limit tests to certain codec formats, such as **JSON** or **binary**. If not specified, tests are
    run for all codecs.
 
 * `relevantCompressions` is used to limit tests to certain compression algorithms, such as **gzip**, **brotli**, or **snappy**. If not

From 46ffbdd1f18599972cb60c205a5f2c829eb5afa7 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 14:16:58 -0500
Subject: [PATCH 12/22] Docs

---
 internal/app/connectconformance/results.go | 2 --
 1 file changed, 2 deletions(-)

diff --git a/internal/app/connectconformance/results.go b/internal/app/connectconformance/results.go
index f35edda8..5963ea07 100644
--- a/internal/app/connectconformance/results.go
+++ b/internal/app/connectconformance/results.go
@@ -167,8 +167,6 @@ func (r *testResults) assert(
 	expected := definition.ExpectedResponse
 	var errs multiErrors
 
-	fmt.Println(expected)
-
 	errs = append(errs, checkError(expected.Error, actual.Error)...)
 	errs = append(errs, checkPayloads(expected.Payloads, actual.Payloads)...)
 

From 8c57f243679167290749dc57e31e03d0f70cfb5a Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 14:22:58 -0500
Subject: [PATCH 13/22] Docs

---
 docs/authoring_test_cases.md | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 2aa50c41..0d95c2b7 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -62,7 +62,7 @@ which defines the request which will be sent to a client during the test run. Ea
 fields:
 
 * `testName` - For naming conventions, see [below](#naming-conventions).
-* `service` - This will most likely be the fully-qualified path to the `ConformanceService`.
+* `service` - The fully-qualified path of the service this test will interact with, typically the `ConformanceService`.
 * `method` - This is a string specifying the method on `service` that will be called.
 * `streamType` - One of `STREAM_TYPE_UNARY`, `STREAM_TYPE_CLIENT_STREAM`, `STREAM_TYPE_SERVER_STREAM`, 
   `STREAM_TYPE_HALF_DUPLEX_BIDI_STREAM`, or `STREAM_TYPE_FULL_DUPLEX_BIDI_STREAM`.
@@ -103,7 +103,7 @@ prefixed with the stream type and a backslash (`/`).
 
  > [!NOTE]  
  > In the case of Bidi tests with separate cases for half-duplex operation vs. full-duplex
- > operation, you should also add `full_duplex` or `half_duplex` to the test name. 
+ > operation, you should also add `full-duplex` or `half-duplex` to the test name. 
 
  For example:
 
@@ -133,13 +133,13 @@ good examples of this.
 If you do need to specify an explicit response, simply define an `expectedResponse` block for your test case and this will
 override the auto-generated expected response in the test runner. 
 
-To tests denoting an explicit response, search the [test suites](testsuites) directory for the word `expectedResponse`.
+To see tests denoting an explicit response, search the [test suites](testsuites) directory for the word `expectedResponse`.
 
 ## Example 
 
 Taking all of the above into account, here is an example test suite that verifies a server returns the specified headers
 and trailers. The below test will only run when `mode` is `server` and will be limited to the Connect protocol using the
-JSON format and `identity` compression. In addition, it will require the server verify the Connect version header.
+JSON format and identity compression. In addition, it will require the server verify the Connect version header.
 
 ```yaml
 name: Example Test Suite
@@ -154,6 +154,7 @@ relevantCompressions:
 # Constrain these tests to only use the JSON codec.
 relevantCodecs:
   - CODEC_JSON
+# Require Connect version header verification.
 connectVersionMode:  CONNECT_VERSION_MODE_REQUIRE
 testCases:
   - request:

From f5b642d27173547ea9cb1141f26bc649322703b9 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 14:30:05 -0500
Subject: [PATCH 14/22] Docs

---
 .../connectconformance/test_case_library.go   | 25 +++++++++----------
 .../testsuites/data/tls-client-certs.yaml     |  4 +--
 2 files changed, 13 insertions(+), 16 deletions(-)

diff --git a/internal/app/connectconformance/test_case_library.go b/internal/app/connectconformance/test_case_library.go
index 0183a0a1..418131bf 100644
--- a/internal/app/connectconformance/test_case_library.go
+++ b/internal/app/connectconformance/test_case_library.go
@@ -174,18 +174,18 @@ func (lib *testCaseLibrary) expandSuite(suite *conformancev1.TestSuite, configCa
 
 func (lib *testCaseLibrary) expandCases(cfgCase configCase, namePrefix []string, testCases []*conformancev1.TestCase) error {
 	for _, testCase := range testCases {
-		if testCase.Request.TestName == "" {
-			return errors.New("suite contains a test with no name")
-		}
-		if testCase.Request.Service == "" {
-			return fmt.Errorf("test %s has no service specified", testCase.Request.TestName)
-		}
-		if testCase.Request.Method == "" {
-			return fmt.Errorf("test %s has no method specified", testCase.Request.TestName)
-		}
-		if testCase.Request.StreamType == conformancev1.StreamType_STREAM_TYPE_UNSPECIFIED {
-			return fmt.Errorf("test %s has no stream type specified", testCase.Request.TestName)
-		}
+		// if testCase.Request.TestName == "" {
+		// 	return errors.New("suite contains a test with no name")
+		// }
+		// if testCase.Request.Service == "" {
+		// 	return fmt.Errorf("test %s has no service specified", testCase.Request.TestName)
+		// }
+		// if testCase.Request.Method == "" {
+		// 	return fmt.Errorf("test %s has no method specified", testCase.Request.TestName)
+		// }
+		// if testCase.Request.StreamType == conformancev1.StreamType_STREAM_TYPE_UNSPECIFIED {
+		// 	return fmt.Errorf("test %s has no stream type specified", testCase.Request.TestName)
+		// }
 		if testCase.Request.StreamType != cfgCase.StreamType {
 			continue
 		}
@@ -628,7 +628,6 @@ func populateExpectedStreamResponse(testCase *conformancev1.TestCase) error {
 	if err != nil {
 		return err
 	}
-	fmt.Println("on it")
 
 	definer, ok := concreteReq.(streamResponseDefiner)
 	if !ok {
diff --git a/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml b/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
index c1b29957..18c41f1a 100644
--- a/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
+++ b/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
@@ -1,11 +1,8 @@
 name: TLS Client Certs
-reliesOnTls: true
-reliesOnTlsClientCerts: true
 # This just does the basics with a client-cert, instead of running every test case with them.
 # TODO - Add unary and other stream type tests here also
 testCases:
   - request:
-      testName: client-stream
       service: connectrpc.conformance.v1.ConformanceService
       method: ClientStream
       streamType: STREAM_TYPE_CLIENT_STREAM
@@ -24,3 +21,4 @@ testCases:
                 value: ["bing"]
         - "@type": type.googleapis.com/connectrpc.conformance.v1.ClientStreamRequest
           requestData: "dGVzdCByZXNwb25zZQ=="
+    expectedResponse:

From 7dff23e3e45c31c82c633a40bcd87e346d99bf5e Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 14:30:31 -0500
Subject: [PATCH 15/22] Docs

---
 .../connectconformance/testsuites/data/tls-client-certs.yaml  | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml b/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
index 18c41f1a..c1b29957 100644
--- a/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
+++ b/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
@@ -1,8 +1,11 @@
 name: TLS Client Certs
+reliesOnTls: true
+reliesOnTlsClientCerts: true
 # This just does the basics with a client-cert, instead of running every test case with them.
 # TODO - Add unary and other stream type tests here also
 testCases:
   - request:
+      testName: client-stream
       service: connectrpc.conformance.v1.ConformanceService
       method: ClientStream
       streamType: STREAM_TYPE_CLIENT_STREAM
@@ -21,4 +24,3 @@ testCases:
                 value: ["bing"]
         - "@type": type.googleapis.com/connectrpc.conformance.v1.ClientStreamRequest
           requestData: "dGVzdCByZXNwb25zZQ=="
-    expectedResponse:

From 55204cb2bdeb964c99cb6f3d52f3c3bebff464cc Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 14:31:05 -0500
Subject: [PATCH 16/22] Docs

---
 .../connectconformance/test_case_library.go   | 24 +++++++++----------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/internal/app/connectconformance/test_case_library.go b/internal/app/connectconformance/test_case_library.go
index 418131bf..26c403fd 100644
--- a/internal/app/connectconformance/test_case_library.go
+++ b/internal/app/connectconformance/test_case_library.go
@@ -174,18 +174,18 @@ func (lib *testCaseLibrary) expandSuite(suite *conformancev1.TestSuite, configCa
 
 func (lib *testCaseLibrary) expandCases(cfgCase configCase, namePrefix []string, testCases []*conformancev1.TestCase) error {
 	for _, testCase := range testCases {
-		// if testCase.Request.TestName == "" {
-		// 	return errors.New("suite contains a test with no name")
-		// }
-		// if testCase.Request.Service == "" {
-		// 	return fmt.Errorf("test %s has no service specified", testCase.Request.TestName)
-		// }
-		// if testCase.Request.Method == "" {
-		// 	return fmt.Errorf("test %s has no method specified", testCase.Request.TestName)
-		// }
-		// if testCase.Request.StreamType == conformancev1.StreamType_STREAM_TYPE_UNSPECIFIED {
-		// 	return fmt.Errorf("test %s has no stream type specified", testCase.Request.TestName)
-		// }
+		if testCase.Request.TestName == "" {
+			return errors.New("suite contains a test with no name")
+		}
+		if testCase.Request.Service == "" {
+			return fmt.Errorf("test %s has no service specified", testCase.Request.TestName)
+		}
+		if testCase.Request.Method == "" {
+			return fmt.Errorf("test %s has no method specified", testCase.Request.TestName)
+		}
+		if testCase.Request.StreamType == conformancev1.StreamType_STREAM_TYPE_UNSPECIFIED {
+			return fmt.Errorf("test %s has no stream type specified", testCase.Request.TestName)
+		}
 		if testCase.Request.StreamType != cfgCase.StreamType {
 			continue
 		}

From 5fb821220e5cb3bf3ad310b52b45cdfd2180e8a3 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Wed, 14 Feb 2024 14:54:00 -0500
Subject: [PATCH 17/22] Fix tests

---
 .../connectconformance/test_case_library.go   |  3 +-
 .../test_case_library_test.go                 | 30 +++++++++++++++++++
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/internal/app/connectconformance/test_case_library.go b/internal/app/connectconformance/test_case_library.go
index 26c403fd..8d63eccf 100644
--- a/internal/app/connectconformance/test_case_library.go
+++ b/internal/app/connectconformance/test_case_library.go
@@ -453,7 +453,8 @@ func populateExpectedResponse(testCase *conformancev1.TestCase) error {
 	// message in this situation, where the response data value is some fixed string (such as "no response definition")
 	// and whose request info will still be present, but we expect it to indicate zero request messages.
 	if len(testCase.Request.RequestMessages) == 0 {
-		return fmt.Errorf("test %s has no request messages specified", testCase.Request.TestName)
+		testCase.ExpectedResponse = &conformancev1.ClientResponseResult{}
+		return nil
 	}
 
 	switch testCase.Request.StreamType {
diff --git a/internal/app/connectconformance/test_case_library_test.go b/internal/app/connectconformance/test_case_library_test.go
index 219cdcf5..de4b8dfd 100644
--- a/internal/app/connectconformance/test_case_library_test.go
+++ b/internal/app/connectconformance/test_case_library_test.go
@@ -39,15 +39,23 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: basic-unary
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY
                       - request:
                             testName: basic-client-stream
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: ClientStream
                             streamType: STREAM_TYPE_CLIENT_STREAM
                       - request:
                             testName: basic-server-stream
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: ServerStream
                             streamType: STREAM_TYPE_SERVER_STREAM
                       - request:
                             testName: basic-bidi-stream
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: BidStream
                             streamType: STREAM_TYPE_FULL_DUPLEX_BIDI_STREAM`,
 		"tls.yaml": `
                     name: TLS
@@ -55,15 +63,23 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: tls-unary
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY
                       - request:
                             testName: tls-client-stream
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: ClientStream
                             streamType: STREAM_TYPE_CLIENT_STREAM
                       - request:
                             testName: tls-server-stream
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: ServerStream
                             streamType: STREAM_TYPE_SERVER_STREAM
                       - request:
                             testName: tls-bidi-stream
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: BidiStream
                             streamType: STREAM_TYPE_FULL_DUPLEX_BIDI_STREAM`,
 		"tls-client-certs.yaml": `
                     name: TLS Client Certs
@@ -72,6 +88,8 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: tls-client-cert-unary
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY`,
 		"connect-get.yaml": `
                     name: Connect GET
@@ -80,6 +98,8 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: connect-get-unary
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY`,
 		"connect-version-client-required.yaml": `
                     name: Connect Version Required (client)
@@ -89,6 +109,8 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: unary-without-connect-version-header
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY`,
 		"connect-version-server-required.yaml": `
                     name: Connect Version Required (server)
@@ -98,6 +120,8 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: unary-without-connect-version-header
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY`,
 		"connect-version-client-not-required.yaml": `
                     name: Connect Version Optional (client)
@@ -107,6 +131,8 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: unary-without-connect-version-header
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY`,
 		"connect-version-server-not-required.yaml": `
                     name: Connect Version Optional (server)
@@ -116,6 +142,8 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: unary-without-connect-version-header
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY`,
 		"max-receive-limit": `
                     name: Max Receive Size (server)
@@ -124,6 +152,8 @@ func TestNewTestCaseLibrary(t *testing.T) {
                     testCases:
                       - request:
                             testName: unary-exceeds-limit
+                            service: connectrpc.conformance.v1.ConformanceService
+                            method: Unary
                             streamType: STREAM_TYPE_UNARY`,
 	}
 	testSuiteData := make(map[string][]byte, len(testData))

From dfebe228eb19b876ef28b63576f2eb9cce74749f Mon Sep 17 00:00:00 2001
From: Steve Ayers <smaye81@gmail.com>
Date: Thu, 15 Feb 2024 09:11:10 -0500
Subject: [PATCH 18/22] Update docs/authoring_test_cases.md

Co-authored-by: Joshua Humphries <2035234+jhump@users.noreply.github.com>
---
 docs/authoring_test_cases.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 0d95c2b7..fe2b9de0 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -62,7 +62,7 @@ which defines the request which will be sent to a client during the test run. Ea
 fields:
 
 * `testName` - For naming conventions, see [below](#naming-conventions).
-* `service` - The fully-qualified path of the service this test will interact with, typically the `ConformanceService`.
+* `service` - The fully-qualified name of the service this test will interact with. For now, this is always `connectrpc.conformance.v1.ConformanceService`.
 * `method` - This is a string specifying the method on `service` that will be called.
 * `streamType` - One of `STREAM_TYPE_UNARY`, `STREAM_TYPE_CLIENT_STREAM`, `STREAM_TYPE_SERVER_STREAM`, 
   `STREAM_TYPE_HALF_DUPLEX_BIDI_STREAM`, or `STREAM_TYPE_FULL_DUPLEX_BIDI_STREAM`.

From c4c2b6007752f6ef10f2d8ea690c959438bfb4fa Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Thu, 15 Feb 2024 11:04:43 -0500
Subject: [PATCH 19/22] Docs

---
 docs/authoring_test_cases.md                  | 59 ++++++++++++++-----
 .../connectconformance/test_case_library.go   | 10 ++--
 .../testsuites/data/tls-client-certs.yaml     |  1 -
 3 files changed, 48 insertions(+), 22 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index fe2b9de0..5fc66e28 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -1,6 +1,6 @@
 # Authoring test cases
 
-Test cases for the conformance runner are configured in YAML files and are located [here][testsuites]. Each file 
+Test cases for the conformance runner are configured in YAML files and are located [here][test-suite-dir]. Each file 
 represents a single suite of tests. 
 
 Authoring new test cases involves either a new file, if a new suite is warranted, or an additional test case in an 
@@ -10,11 +10,15 @@ For the Protobuf definitions for the conformance runner, see the [connectrpc rep
 
 ## Test suites
 
-A suite represents a series of tests that all test the same general functionality (cancellation, timeouts, etc.). When
-defining a test suite in a YAML file, the only values that are required are the suite name (which should be unique across all suites)
-and at least one test case.
+A suite represents a series of tests that all test the same general functionality (cancellation, timeouts, etc.). Each 
+test suite YAML file represents a [`connectrpc.conformance.v1.TestSuite`][test-suite] message. So the schema of the 
+`TestSuite` message defines the schema of the YAML file. The representation of a Protobuf message in YAML is the same as its
+[JSON format][json-docs].
 
-Each suite can be configured with various directives which will apply to all tests within that suite. These directives
+When defining a test suite in a YAML file, the only values that are required are the suite name (which should be unique 
+across all suites) and at least one test case.
+
+In addition, each suite can be configured with various directives which will apply to all tests within that suite. These directives
 can be used to constrain the tests run by the conformance runner or to signal various configurations to the runner that
 may be needed to execute the tests. The runner will use these directives to expand the tests in the suite into multiple
 permutations. This means that a single test defined in a suite will be run several times across various permutations.
@@ -44,7 +48,8 @@ The below directives are used to constrain tests within a suite:
 The below `reliesOn` directives are used to signal to the test runner how the reference client or server should be
 configured when running tests:
 
-* `reliesOnTls` specifies that a suite relies on TLS. Defaults to `false`.
+* `reliesOnTls` specifies that a suite relies on TLS. If `true`, the test cases will not be run against non-TLS server 
+  configurations. Defaults to `false`.
 
 * `reliesOnTlsClientCerts` specifies that the suite relies on the _client_ using TLS certificates to authenticate with 
   the server. Note that if this is set to `true`, `reliesOnTls` must also be `true`. Defaults to `false`.
@@ -68,15 +73,34 @@ fields:
   `STREAM_TYPE_HALF_DUPLEX_BIDI_STREAM`, or `STREAM_TYPE_FULL_DUPLEX_BIDI_STREAM`.
 
 Once the above are specified, you can then define your request. For a full list of fields to specify in the request,
-see the [`ClientCompatRequest`][client-compat-proto] message in the Conformance Protobuf definitions.
+see the [`ClientCompatRequest`][client-compat-request] message in the Conformance Protobuf definitions.
+
+### Raw payloads
+
+There are two message types in the test case schema worth noting here - [`RawHTTPRequest`][raw-http-request] and 
+[`RawHTTPResponse`][raw-http-response]. Both allow for the ability to define a round-trip outside the scope of the 
+Connect framework. They are used for sending or receiving anomalous payloads during a test.
+
+#### `RawHTTPRequest`
+
+The [`RawHTTPRequest`][raw-http-request] message can be set on the `request` property in a test case. Its purpose is to model a raw HTTP 
+request. This can be used to craft custom requests with odd properties (including certain kinds of malformed requests) 
+to test edge cases in servers. This value is only handled by the reference client and should only appear in files where 
+`mode` is set to `TEST_MODE_SERVER`.
+
+#### `RawHTTPResponse`
+
+The [`RawHTTPResponse`][raw-http-response] message is the analog to [`RawHTTPRequest`][raw-http-request]. It can be set in the response definition for a unary 
+or streaming RPC type and its purpose is to model a raw HTTP response. This can be used to craft custom responses with 
+odd properties (including returning aberrant HTTP codes or certain kinds of malformed responses) to test edge cases in 
+clients. This value is only handled by the reference server and should only appear in files where `mode` is set to 
+`TEST_MODE_CLIENT`.
 
  > [!IMPORTANT]  
  > The `ClientCompatRequest` message contains some fields that should _not_ be specified in test cases.
- > These fields include:
- > * Fields 1 through 8 in the message definition. These fields are automatically populated by the test runner.
+ > * Fields 1 through 8 in the message definition are automatically populated by the test runner.
  >   If a test is specific to one of these values, it should instead be indicated in the directives for the test suite
  >   itself.
- > * Field 20 (`raw_request`). This field is only used by the reference client for sending anomalous requests.
 
 ## Naming conventions
 
@@ -84,7 +108,7 @@ Test suites and their tests within follow a loose naming convention.
 
 ### Test files (Suites)
 
-Test files should be named according to the suite inside and the general functionality being tested. In addition:
+Test files should be named according to the suite inside and the general functionality being tested. Additionally:
 
 * If a suite applies only to a certain protocol, the file name should be prefixed with that protocol. If the suite applies
   to all protocols, this can be omitted.
@@ -133,13 +157,13 @@ good examples of this.
 If you do need to specify an explicit response, simply define an `expectedResponse` block for your test case and this will
 override the auto-generated expected response in the test runner. 
 
-To see tests denoting an explicit response, search the [test suites](testsuites) directory for the word `expectedResponse`.
+To see tests denoting an explicit response, search the [test suites][test-suite-dir] directory for the word `expectedResponse`.
 
 ## Example 
 
 Taking all of the above into account, here is an example test suite that verifies a server returns the specified headers
 and trailers. The below test will only run when `mode` is `server` and will be limited to the Connect protocol using the
-JSON format and identity compression. In addition, it will require the server verify the Connect version header.
+JSON format and identity compression. Also, it will require the server verify the Connect version header.
 
 ```yaml
 name: Example Test Suite
@@ -174,7 +198,10 @@ testCases:
               value: ["bing"]
 ```
 
-[testsuites]: https://github.com/connectrpc/conformance/tree/main/internal/app/connectconformance/testsuites/data
-[suite-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/suite.proto
-[client-compat-proto]: https://buf.build/connectrpc/conformance/file/main:connectrpc/conformance/v1/client_compat.proto
+[test-suite-dir]: https://github.com/connectrpc/conformance/tree/main/internal/app/connectconformance/testsuites/data
+[client-compat-request]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ClientCompatRequest
+[raw-http-request]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.RawHTTPRequest
+[raw-http-response]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.RawHTTPResponse
+[test-suite]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.TestSuite)
 [connectrpc-repo]: https://buf.build/connectrpc/conformance
+[json-docs]: https://protobuf.dev/programming-guides/proto3/#json
diff --git a/internal/app/connectconformance/test_case_library.go b/internal/app/connectconformance/test_case_library.go
index 8d63eccf..1b14f3bf 100644
--- a/internal/app/connectconformance/test_case_library.go
+++ b/internal/app/connectconformance/test_case_library.go
@@ -173,18 +173,18 @@ func (lib *testCaseLibrary) expandSuite(suite *conformancev1.TestSuite, configCa
 }
 
 func (lib *testCaseLibrary) expandCases(cfgCase configCase, namePrefix []string, testCases []*conformancev1.TestCase) error {
-	for _, testCase := range testCases {
+	for i, testCase := range testCases {
 		if testCase.Request.TestName == "" {
-			return errors.New("suite contains a test with no name")
+			return fmt.Errorf("test case #%d: test case has no name", i+1)
 		}
 		if testCase.Request.Service == "" {
-			return fmt.Errorf("test %s has no service specified", testCase.Request.TestName)
+			return fmt.Errorf("test case #%d: test %s has no service specified", i+1, testCase.Request.TestName)
 		}
 		if testCase.Request.Method == "" {
-			return fmt.Errorf("test %s has no method specified", testCase.Request.TestName)
+			return fmt.Errorf("test case #%d: test %s has no method specified", i+1, testCase.Request.TestName)
 		}
 		if testCase.Request.StreamType == conformancev1.StreamType_STREAM_TYPE_UNSPECIFIED {
-			return fmt.Errorf("test %s has no stream type specified", testCase.Request.TestName)
+			return fmt.Errorf("test case #%d: test %s has no stream type specified", i+1, testCase.Request.TestName)
 		}
 		if testCase.Request.StreamType != cfgCase.StreamType {
 			continue
diff --git a/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml b/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
index c1b29957..aaab0973 100644
--- a/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
+++ b/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
@@ -6,7 +6,6 @@ reliesOnTlsClientCerts: true
 testCases:
   - request:
       testName: client-stream
-      service: connectrpc.conformance.v1.ConformanceService
       method: ClientStream
       streamType: STREAM_TYPE_CLIENT_STREAM
       requestHeaders:

From c36095073219fc5c7708eac52b86d037107fe4d7 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Thu, 15 Feb 2024 11:06:41 -0500
Subject: [PATCH 20/22] Docs

---
 docs/authoring_test_cases.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 5fc66e28..9d7eb997 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -202,6 +202,6 @@ testCases:
 [client-compat-request]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.ClientCompatRequest
 [raw-http-request]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.RawHTTPRequest
 [raw-http-response]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.RawHTTPResponse
-[test-suite]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.TestSuite)
+[test-suite]: https://buf.build/connectrpc/conformance/docs/main:connectrpc.conformance.v1#connectrpc.conformance.v1.TestSuite
 [connectrpc-repo]: https://buf.build/connectrpc/conformance
 [json-docs]: https://protobuf.dev/programming-guides/proto3/#json

From 5d1415455ff29bbb1ccf59b0dee678ca9db79313 Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Thu, 15 Feb 2024 11:08:18 -0500
Subject: [PATCH 21/22] Docs

---
 docs/authoring_test_cases.md | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/docs/authoring_test_cases.md b/docs/authoring_test_cases.md
index 9d7eb997..68dc213e 100644
--- a/docs/authoring_test_cases.md
+++ b/docs/authoring_test_cases.md
@@ -11,9 +11,8 @@ For the Protobuf definitions for the conformance runner, see the [connectrpc rep
 ## Test suites
 
 A suite represents a series of tests that all test the same general functionality (cancellation, timeouts, etc.). Each 
-test suite YAML file represents a [`connectrpc.conformance.v1.TestSuite`][test-suite] message. So the schema of the 
-`TestSuite` message defines the schema of the YAML file. The representation of a Protobuf message in YAML is the same as its
-[JSON format][json-docs].
+test suite YAML file represents a [`TestSuite`][test-suite] message and the schema of this message defines the schema 
+of the YAML file. The representation of a Protobuf message in YAML is the same as its [JSON format][json-docs].
 
 When defining a test suite in a YAML file, the only values that are required are the suite name (which should be unique 
 across all suites) and at least one test case.

From 06cd0597d40b6f85322a6351c842e4af784d8e2c Mon Sep 17 00:00:00 2001
From: Steve Ayers <sayers@buf.build>
Date: Thu, 15 Feb 2024 11:10:27 -0500
Subject: [PATCH 22/22] Docs

---
 .../app/connectconformance/testsuites/data/tls-client-certs.yaml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml b/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
index aaab0973..c1b29957 100644
--- a/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
+++ b/internal/app/connectconformance/testsuites/data/tls-client-certs.yaml
@@ -6,6 +6,7 @@ reliesOnTlsClientCerts: true
 testCases:
   - request:
       testName: client-stream
+      service: connectrpc.conformance.v1.ConformanceService
       method: ClientStream
       streamType: STREAM_TYPE_CLIENT_STREAM
       requestHeaders: