Add support for an optional, secure gRPC service

This commit adds a gRPC service to Stenographer and implements a call to retrieve PCAP via stenoread. The gRPC service is designed to only use a secure channel (due to the sensitivity of packet data) with SSL certificates that are managed separately from those created by stenokeys.sh (due to users needing to manage remote access to many servers running Stenographer, though technically users could share the certs used by the local Stenographer HTTP service). This commit explicitly does not address any client-side integration with the gRPC service, but clients can build interaction based on the protobuf and documentation in DESIGN.md. Below is a summary of the additions and changes:
- Added rpc/rpc.go
- Added protobuf/steno.proto
- Changed config/config.go to support Rpc configuration
- Changed stenographer.go to check for Rpc configuration and (if found) run gRPC server as a goroutine
- Updated DESIGN docs to describe Rpc functionality

DESIGN.md

@@ -278,6 +278,45 @@ packet data out in a nice format.  With various options, tcpdump could do
 further filtering (by TCP flags, etc), write its input to disk (-w out.pcap), or
 do all the other things tcpdump is so good at.
 
+### gRPC ###
+
+Stenographer has gRPC support that enables secure, remote interactions with the program. Given the sensitive nature of packet data and the requirements of many users to manage a fleet of servers running Stenographer, the gRPC channel only supports encryption with client authentication and expects the administrator to use certificates that are managed separately from those generated by `stenokeys.sh` (for easily generating certificates, take a look at Square's [certstrap](https://github.com/square/certstrap) utility). The protobuf that defines Stenographer's gRPC service can be found in protobuf/steno.proto.
+
+gRPC support is optional and can be enabled by adding an Rpc dictionary of settings to `steno.conf`. An example configuration is shown below:
+```json
+, "Rpc": { "CaCert": "/path/to/rpc/ca/cert"
+         , "ServerKey": "/path/to/rpc/key"
+         , "ServerCert": "/path/to/rpc/cert"
+         , "ServerPort": 8443
+         , "ServerPcapPath": "/path/to/rpc/pcap/directory"
+         , "ServerPcapMaxSize": 1000000000
+         , "ClientPcapChunkSize": 1000
+         , "ClientPcapMaxSize": 5000000
+  }
+```
+
+#### RetrievePcap ####
+
+This call allows clients to remotely retrieve PCAP via `stenoread`. To retrieve PCAP, clients send the service a unique identifier, the size of PCAP file chunks to stream in return, the maximum size of the PCAP file to return, and the `stenoread` query used to parse packet data. In response, clients receive streams of messages containing the unique identifier and PCAP file chunks (which need to be reassembled client-side). Below is a minimalist example (shown in Python) of how a client can request PCAP and save it to local disk:
+```py
+with grpc.secure_channel(server, creds) as channel:
+    stub = steno_pb2_grpc.StenographerStub(channel)
+    pb = steno_pb2.PcapRequest()
+    pb.uid = str(uuid.uuid4())
+    pb.chunk_size = 1000
+    pb.max_size = 500000
+    pb.query = 'after 5m ago and tcp'
+    pcap_file = os.path.join('.', '{}.pcap'.format(uid))
+    with open(pcap_file, 'wb') as fout:
+        for response in stub.RetrievePcap(pb):
+            fout.write(response.pcap)
+```
+
+`RetrievePcap` requires the gRPC server to be configured with the following fields (in addition to any fields that require the server to startup):
+- ServerPcapPath: local path to the directory where `stenoread` PCAP is temporarily stored
+- ServerPcapMaxSize: upper limit on how much PCAP a client is allowed to receive (used to restrict clients from receiving excessively large PCAPs)
+- ClientPcapChunkSize: size of the PCAP chunks to stream to the client (used if the client has not specified a size in the request)
+- ClientPcapMaxSize: upper limit on how much PCAP a client will receive (used if the client has not specified a size in the request)
 
 ### Defense In Depth ###
 

config/config.go

@@ -47,8 +47,21 @@ type ThreadConfig struct {
 	MaxDirectoryFiles  int `json:",omitempty"`
 }
 
+// RpcConfig is a json-decoded configuration for running the gRPC server.
+type RpcConfig struct {
+        CaCert                  string
+        ServerKey               string
+        ServerCert              string
+        ServerPort              int
+        ServerPcapPath          string
+        ServerPcapMaxSize       int64
+        ClientPcapChunkSize     int64
+        ClientPcapMaxSize       int64
+}
+
 // Config is a json-decoded configuration for running stenographer.
 type Config struct {
+        Rpc           *RpcConfig
 	StenotypePath string
 	Threads       []ThreadConfig
 	Interface     string

protobuf/steno.proto

@@ -0,0 +1,31 @@
+// Copyright 2019 Josh Liburdi and Google Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+syntax = "proto3";
+package steno;
+
+service Stenographer {
+  rpc RetrievePcap(PcapRequest) returns (stream PcapResponse) {}
+}
+
+message PcapRequest {
+  string uid = 1;
+  int64 chunk_size = 2;
+  int64 max_size = 3;
+  string query = 4;
+}
+
+message PcapResponse {
+  string uid = 1;
+  bytes pcap = 2;
+}