[TEX-537] Add support for gRPC-web request body inspection #548
Conversation
timdawborn
force-pushed
the
TEX-537-grpc-web-request-body-inspection
branch
from
4018c8b
to
22d1717
Compare
timdawborn
changed the base branch from
master
to
TEX-537-refactor-compression
| @@ -57,7 +57,8 @@ | |||
| "deep-diff": "^1.0.2", | |||
| "fs-extra": "^11.2.0", | |||
| "js-yaml": "^4.1.0", | |||
| "string-similarity": "^4.0.4" | |||
| "string-similarity": "^4.0.4", | |||
| "unicode-properties": "^1.4.1" | |||
There was a problem hiding this comment.
Adds a new library to get basic unicode properties for Unicode code points. As far as I could tell, NodeJS does not have anything like this builtin (equivalent of the Python unicodedata package).
| case "json": | ||
| return convertJsonMessageToObject(message); | ||
| case "proto": | ||
| return convertProtoMessageToObject(message); |
There was a problem hiding this comment.
gRPC-Web payloads can have various sub-content types. We handle json and proto here. All others are unhandled, and will result in the default binary comparison logic being used for the request payloads.
There was a problem hiding this comment.
The changes in this file are best viewed with the diff ignoring whitespace. A bunch of indentation changes makes the diff with whitespace very hard to read compared to what the changes actually are.
| } else if (contentType.startsWith("application/grpc-web-text")) { | ||
| return countBodyDifferencesGrpcWebText( | ||
| request1, | ||
| contentType1, | ||
| request2, | ||
| contentType2, | ||
| rewriteBeforeDiffRules, | ||
| ); | ||
| } else if (contentType.startsWith("application/grpc-web")) { | ||
| return countBodyDifferencesGrpcWeb( | ||
| request1, | ||
| contentType1, | ||
| request2, | ||
| contentType2, | ||
| rewriteBeforeDiffRules, | ||
| ); |
There was a problem hiding this comment.
Adds special similarity scoring logic for gRPC-Web request types, like we do for other content types like JSON.
There was a problem hiding this comment.
Really neat. Code for the protobuf spec LGTM (caveat: I didn't read the official spec), tests LGTM, and similarity logic LGTM. Will be great to see it in action. 🙂
The protobuf decoding part might even be worth being its own library some day 😄 It looks more powerful than https://github.com/konsumer/rawproto which seems to be the only alternative for now.
Removes the functionality to unframe gRPC-web+json requests (added in #501 and #503) in favour of the proper solution to handle grpc-web requests in full (added in #548). This functionality didn't work quite as expected anyway, so it's good to clean it up in favour of the the full, proper working solution.
Adds support to proxay to perform gRPC-web request body inspection when computing request similarity. This is done by inspecting the content type of the request. If it's
application/grpc-weborapplication/grpc-web-text, we invoke new body-to-object decoding logic, like we do forapplication/jsonrequests. If this body-to-object decoding logic returns an object, the normal object similarity scoring happens, which is what we're after all along.