thrift: framed/unframed transport, binary/compact protocol #3509

zuercher · 2018-05-30T22:42:06Z

Provides Thrift protocol primitives, with tests, that will be used to create Thrift protocol filters in a subsequent PR. Implements the Thrift framed and unframed transports as well as an "auto" transport that detects the transport based on the first few bytes in a Buffer. Similarly, the Thrift binary and compact protocols are implemented along with an "auto" protocol.

Signed-off-by: Stephan Zuercher stephan@turbinelabs.io

Risk Level: Low - no filters currently make use of these primitives
Testing: unit tests, manual integration testing with a work-in-progress filter in a private branch
Docs Changes: n/a
Release Notes: n/a
Relates To: #2247

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

zuercher · 2018-05-30T22:42:21Z

@brian-pane PTAL

brian-pane

Thanks for implementing this! I read through the header files and added some notes. I'll read the .cc files over the next couple of days.

brian-pane · 2018-05-31T03:11:55Z

source/extensions/filters/network/thrift_proxy/protocol.h

+#include "common/common/macros.h"
+#include "common/singleton/const_singleton.h"
+
+#include "absl/types/optional.h"


Is this needed in protocol.h? It doesn't look like this file uses absl::optional (but I might have missed something).

brian-pane · 2018-05-31T03:15:46Z

source/extensions/filters/network/thrift_proxy/protocol.h

+#include "envoy/buffer/buffer.h"
+#include "envoy/common/pure.h"
+
+#include "common/common/assert.h"


Does anything in protocol.h depend on this include?

brian-pane · 2018-05-31T03:21:39Z

source/extensions/filters/network/thrift_proxy/protocol.h

+ * Thrift protocol message types.
+ * See https://github.com/apache/thrift/blob/master/lib/cpp/src/thrift/protocol/TProtocol.h
+ */
+enum MessageType {


I recommend using enum class rather than plain enum here, so that the value names (Call, Reply, etc) get namespaced within MessageType.

brian-pane · 2018-05-31T03:21:49Z

source/extensions/filters/network/thrift_proxy/protocol.h

+ * Thrift protocol struct field types.
+ * See https://github.com/apache/thrift/blob/master/lib/cpp/src/thrift/protocol/TProtocol.h
+ */
+enum FieldType {


enum class

brian-pane · 2018-05-31T03:31:14Z

source/extensions/filters/network/thrift_proxy/protocol.h

+/**
+ * Names of available Protocol implementations.
+ */
+class ProtocolNameValues {


It looks like the only use of this class is for each Protocol subclass to return a string representation of its own name. I think it would be cleaner to just make each Protocol subclass define its own name string internally. That way, we won't have to modify this central ProtocolNameValues class every time a new Protocol subclass is added.

By and large these are used only during debug logging, but my original intention was to avoid allocating these strings. At some point I decided to make the auto protocol report its underlying protocol and ended up constructing or copying strings in all the name() methods.

Since I later added setProtocol to make the AutoProtocolImpl easier to test, I went ahead and made AutoProtocolImpl::name() return a field value, updated by setProtocol. This allowed me to have Protocol::name() return a const std::string&. So no more allocations, but I still have ProtocolNames.

I can move these strings into the Protocol implementations, but per the style guide, I'd need to use the CONSTRUCT_ON_FIRST_USE macro. AutoProtocolImpl would require it's own private method to wrap CONSTRUCT_ON_FIRST_USE for "auto". I think putting all these strings in this class and using ConstSingleton ends up being a little cleaner.

brian-pane · 2018-05-31T03:33:06Z

source/extensions/filters/network/thrift_proxy/transport.h

+/**
+ * Names of available Transport implementations.
+ */
+class TransportNameValues {


I have the same suggestion for this as for ProtocolNameValues: make each subclass of Transport define its own name string, rather than collecting the names in a centralized class that has to be updated every time there's a new Transport subclass.

brian-pane · 2018-05-31T03:36:05Z

source/extensions/filters/network/thrift_proxy/transport.h

+   * decodeFrameStart decodes the start of a transport message, potentially invoking callbacks.
+   *
+   * @param buffer the currently buffered thrift data.
+   * @return bool true if a complete frame header was successfully consumed, false if more data


If the frame header is consumed, that means the corresponding bytes are trimmed from the buffer, right?

Yes. I'll update the comment (and all the others that behave the same way).

alyssawilk · 2018-05-31T14:24:49Z

Ignoring the code, would you be up for adding CODEOWNERS per https://docs.google.com/document/d/1eDQQSxqx2khTXfa2vVm4vqkyRwXYkPzZCcbjxJ2_AvA/edit ? You're clearly set on the maintainer front but good to have documented for future reviews :-)

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

zuercher · 2018-06-01T16:52:02Z

@brian-pane I think I addressed your comments.

brian-pane · 2018-06-04T04:07:14Z

source/extensions/filters/network/thrift_proxy/binary_protocol.cc

+    if (buffer.length() < 3) {
+      return false;
+    }
+    field_id = BufferHelper::peekI16(buffer, 1);


Is it valid for the field ID to be negative?

The protocol specifications all refer to it as a signed integer with no mention of the validity of negative values.

The Thrift IDL documents make no mention of the range of values allowed. In fact, the grammar defines it as an IntConstant, which implies negative values are legal. Empirically, the Thrift compiler will only accept values from 1 to 32767.

In practice, the generated bindings allow negative field IDs, but ignore them like any other unknown field ID.

I'm going to leave this as-is from that standpoint that it's better to be a bit lenient in what's accepted.

brian-pane · 2018-06-04T04:18:32Z

source/extensions/filters/network/thrift_proxy/binary_protocol.cc

+}
+
+bool BinaryProtocolImpl::readDouble(Buffer::Instance& buffer, double& value) {
+  ASSERT(sizeof(double) == sizeof(uint64_t));


Since Envoy is using C++14, I propose using the new static_assert here, since that will work for both debug and non-debug builds. (https://en.cppreference.com/w/cpp/language/static_assert)

brian-pane · 2018-06-04T04:25:23Z

source/extensions/filters/network/thrift_proxy/binary_protocol.cc

+
+const uint16_t BinaryProtocolImpl::Magic = 0x8001;
+
+bool BinaryProtocolImpl::readMessageBegin(Buffer::Instance& buffer, std::string& name,


THanks for writing all this parsing code. We really need a lexer generator, but I haven’t yet found one that’s well suited to binary protocols.

brian-pane · 2018-06-04T04:37:51Z

source/extensions/filters/network/thrift_proxy/buffer_helper.cc

+namespace ThriftProxy {
+
+int8_t BufferHelper::peekI8(Buffer::Instance& buffer, size_t offset) {
+  ASSERT(buffer.length() >= 1);


What do you think about throwing an exception here if the buffer length is too small, than asserting? Even though that would mean an extra conditional branch in the critical path, the branch would always go in the same direction in the absence of bugs, so the CPU should be able to predict the branch with near-perfect accuracy.

Seems reasonable.

brian-pane · 2018-06-04T04:42:00Z

source/extensions/filters/network/thrift_proxy/buffer_helper.cc

+int8_t BufferHelper::peekI8(Buffer::Instance& buffer, size_t offset) {
+  ASSERT(buffer.length() >= 1);
+  int8_t i;
+  buffer.copyOut(offset, 1, &i);


It might be worthwhile to add special methods to Buffer::Instance for the 1, 2, and 4 byte special cases, since a general-purpose copy will be slow for those. But I’m happy to wait and see if this shows up as an issue in profiles first.

I think I'll hold off for now. If it's useful, we'll want to use across a bunch of protocols and it should go in its own PR.

brian-pane · 2018-06-04T04:43:48Z

source/extensions/filters/network/thrift_proxy/buffer_helper.cc

+}
+
+double BufferHelper::drainDouble(Buffer::Instance& buffer) {
+  ASSERT(sizeof(double) == sizeof(uint64_t));


static_assert

Also, for the union trick below to work, double and uint64_t have to have the same endianness. It’s probably possible to test for that at compile time, but off the top of my head I can’t think of a simple way to do it.

It's possible to do this at compile time in C with a trick similar to the union used below, but not in C++.

In fact, according to what I just read (https://stackoverflow.com/a/24431286 and https://stackoverflow.com/questions/11373203/accessing-inactive-union-member-and-undefined-behavior/11996970) the union below shouldn't be allowed either (though both Apache Thrift and Protobuf use it).

brian-pane

Here’s my first batch of comments on the .cc files. I’ll read more tomorrow.

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

rgs1 · 2018-06-04T19:44:10Z

source/extensions/filters/network/thrift_proxy/binary_protocol.h

+};
+
+/**
+ * BinaryProtocolImpl implements the Thrift Binary protocol with non-strict (e.g. lax) message


nit: LaxBinaryProtocolImpl...

rgs1 · 2018-06-04T19:47:49Z

source/extensions/filters/network/thrift_proxy/binary_protocol.h

+};
+
+/**
+ * BinaryProtocolImpl implements the Thrift Binary protocol with non-strict (e.g. lax) message


nit: LaxBinaryProtocolImpl

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

brian-pane · 2018-06-05T16:03:24Z

source/extensions/filters/network/thrift_proxy/protocol.cc

+
+    uint16_t version = BufferHelper::peekU16(buffer);
+    if (BinaryProtocolImpl::isMagic(version)) {
+      setProtocol(std::make_unique<BinaryProtocolImpl>());


Is it possible to determine at this point in the processing whether we need a LaxBinaryProtocolImpl?

The only guarantee that the non-strict binary protocol makes is that the leading bit is 0, so misidentifying another protocol seems pretty likely and the result would be interpreting the first four bytes as message name length. For instance, with auto-transport and auto-protocol, an errant HTTP request would be interpreted as an unframed, non-strict binary thrift message with a message name of ~1.1 GB.

I think longer term, we'll want to have a configurable limit on the size of encoded strings (perhaps separate limits for message names vs. fields). With that in place, I'd feel more comfortable allowing the non-strict binary protocol to be detected automatically.

brian-pane

Here's my next batch of notes. The last thing remaining for me to read is compact_protocol.cc, which I'll try to get finished in the next day.

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

brian-pane

I just finished reading through the rest of the PR, and looks good to me.

mattklein123

I skimmed and it all looks sane to me. I'm sure we can flush out any issues once we start putting traffic through it. Very nice!

thrift: framed/unframed transport, binary/compact protocol

e97093f

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

brian-pane reviewed May 31, 2018

View reviewed changes

alyssawilk assigned brian-pane May 31, 2018

zuercher added 3 commits May 31, 2018 09:52

CODEOWNERS

6f1fb98

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

make transport/proto names const&; review comments

072d9a1

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

given Protocol mock a default name

d326835

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

brian-pane reviewed Jun 4, 2018

View reviewed changes

review comments (static_assert, throw on underflow)

8ce1ff6

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

rgs1 reviewed Jun 4, 2018

View reviewed changes

zuercher added 3 commits June 4, 2018 14:14

use memcpy instead of the union trick

33c5c90

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

fix typo

1bc1d77

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

link slides instead of video

c607c4c

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

brian-pane reviewed Jun 5, 2018

View reviewed changes

zuercher added 3 commits June 5, 2018 09:58

remove size_t, fix offset handling in BufferHelper, test underflows

f82f2ee

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

missed one underflow

eb9e1ce

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

remove stray TODO comment

78a3d22

Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>

brian-pane reviewed Jun 7, 2018

View reviewed changes

Merge branch 'master' into stephan/thrift-primitives

027a243

zuercher assigned mattklein123 Jun 7, 2018

mattklein123 approved these changes Jun 7, 2018

View reviewed changes

zuercher merged commit 2c37930 into envoyproxy:master Jun 7, 2018

zuercher deleted the stephan/thrift-primitives branch July 19, 2018 20:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

thrift: framed/unframed transport, binary/compact protocol #3509

thrift: framed/unframed transport, binary/compact protocol #3509

zuercher commented May 30, 2018

zuercher commented May 30, 2018

brian-pane left a comment

brian-pane May 31, 2018

brian-pane May 31, 2018

brian-pane May 31, 2018

brian-pane May 31, 2018

brian-pane May 31, 2018

zuercher May 31, 2018

brian-pane May 31, 2018

brian-pane May 31, 2018

zuercher May 31, 2018

alyssawilk commented May 31, 2018

zuercher commented Jun 1, 2018

brian-pane Jun 4, 2018

zuercher Jun 4, 2018

zuercher Jun 4, 2018

brian-pane Jun 4, 2018

zuercher Jun 4, 2018

brian-pane Jun 4, 2018

brian-pane Jun 4, 2018

zuercher Jun 4, 2018

brian-pane Jun 4, 2018

zuercher Jun 4, 2018

brian-pane Jun 4, 2018

brian-pane Jun 4, 2018

zuercher Jun 4, 2018

brian-pane left a comment

rgs1 Jun 4, 2018

rgs1 Jun 4, 2018

brian-pane Jun 5, 2018

zuercher Jun 5, 2018

brian-pane left a comment

brian-pane left a comment

mattklein123 left a comment


		const uint16_t BinaryProtocolImpl::Magic = 0x8001;

		bool BinaryProtocolImpl::readMessageBegin(Buffer::Instance& buffer, std::string& name,

thrift: framed/unframed transport, binary/compact protocol #3509

thrift: framed/unframed transport, binary/compact protocol #3509

Conversation

zuercher commented May 30, 2018

zuercher commented May 30, 2018

brian-pane left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alyssawilk commented May 31, 2018

zuercher commented Jun 1, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brian-pane left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brian-pane left a comment

Choose a reason for hiding this comment

brian-pane left a comment

Choose a reason for hiding this comment

mattklein123 left a comment

Choose a reason for hiding this comment