Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add wire tracer capabilities to reference client #768

Merged
merged 46 commits into from Feb 7, 2024
Merged

Conversation

smaye81
Copy link
Member

@smaye81 smaye81 commented Jan 30, 2024

This piggybacks off of the tracer framework to add wire tracing to the reference client. Broadly, this implements the Collector interface and intercepts the Complete function invocation to sniff the built trace and acquire wire details.

The details tracked are:

  • Actual HTTP status code
  • Actual trailers on the wire
  • Connect Raw Error Details (if applicable)

Note that the tests to verify raw error details trailers are not yet in place to keep the complexity of this PR lower. They will be added in a follow-up PR.

@smaye81 smaye81 requested a review from jhump February 1, 2024 02:25
internal/app/referenceclient/wire_tracer.go Outdated Show resolved Hide resolved
internal/app/referenceclient/wire_tracer.go Outdated Show resolved Hide resolved
internal/app/referenceclient/wire_tracer.go Outdated Show resolved Hide resolved
internal/app/referenceclient/wire_tracer.go Outdated Show resolved Hide resolved
internal/app/referenceclient/wire_tracer.go Outdated Show resolved Hide resolved
internal/app/referenceclient/impl.go Outdated Show resolved Hide resolved
internal/app/referenceclient/wire_tracer.go Outdated Show resolved Hide resolved
Trailers []*v1.Header
// The actual JSON observed on the wire in case of an error from a Connect server.
// This will only be non-nil if the protocol is Connect and an error occurred.
ConnectErrorRaw *structpb.Struct
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside: I got another one we'll want to capture. Not that you need to add it here; I can come back and add it soon.

But we will also want to capture the raw trailers in a gRPC-Web end-stream message. Because we'll want to validate that they were all lower-case on the wire.

internal/tracer/middleware.go Outdated Show resolved Hide resolved
internal/tracer/middleware.go Outdated Show resolved Hide resolved
@smaye81 smaye81 marked this pull request as ready for review February 2, 2024 20:06
@smaye81 smaye81 requested a review from jhump February 2, 2024 20:24
internal/app/referenceclient/middleware.go Outdated Show resolved Hide resolved
internal/app/referenceclient/middleware.go Outdated Show resolved Hide resolved
"connectrpc.com/conformance/internal/tracer"
)

type wireInterceptor struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think it makes sense to put this in the same file as the other tracing stuff.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally had it that way and then felt uneasy about that file containing so many various types and functions. I moved them back though. Let me know if it seems off.

internal/app/referenceclient/tracer.go Outdated Show resolved Hide resolved
internal/app/referenceclient/impl.go Outdated Show resolved Hide resolved
// Any HTTP trailers observed after the response body. These do NOT
// include trailers that conveyed via the body, as done in the gRPC-Web
// and Connect streaming protocols.
repeated Header actual_http_trailers = 3;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider moving the feedback field from ClientCompatResponse to here. Then you could also put any error message from trying to compute wire details into that field.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case then, we should probably rename WireDetails to something else since it no longer is just details about the wire. Unless we consider the feedback a 'wire' detail. But, maybe something to reflect it's a reference-client only field i.e. ReferenceClientMetadata or something.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the feedback would mostly be about wire details, so it would be fine to keep the name.

I don't actually have any examples of how we'd use and added it just for symmetry with the server. The server currently just uses it to complain if the incoming request doesn't have the expected properties (like HTTP version, protocol, codec, compression, etc). Technically, those are really wire details. Semantics are totally handled via the normal assertions in the test runner that compare the expected and actual responses.

Copy link
Member

@jhump jhump left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

.github/workflows/buf.yaml Show resolved Hide resolved
internal/tracer/builder.go Show resolved Hide resolved
smaye81 and others added 4 commits February 7, 2024 10:31
Previously, computing the expected response would add an expected
query parameter for the `message` param, with an encoded value.
But the problem is that not all clients would necessarily encode the
message in exactly the same way, and that's okay. Some will choose
to base64-encode (which is usually more efficient with binary
payloads). Also, there's no _canonical_ protobuf binary format, so
there could be subtle differences in the byte output, even for
equivalent messages. Finally, there's no _canonical_ encoding for
compressed payloads -- different default compression settings or other
subtle differences in implementation can lead to more than one valid
compressed encoding of the same data.
@smaye81 smaye81 merged commit f5edb25 into main Feb 7, 2024
7 checks passed
@smaye81 smaye81 deleted the sayers/http_tracer branch February 7, 2024 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants