Add support for an optional, secure gRPC service

DESIGN.md

@@ -278,6 +278,45 @@ packet data out in a nice format.  With various options, tcpdump could do
 further filtering (by TCP flags, etc), write its input to disk (-w out.pcap), or
 do all the other things tcpdump is so good at.
 
+### gRPC ###
+
+Stenographer has gRPC support that enables secure, remote interactions with the program. Given the sensitive nature of packet data and the requirements of many users to manage a fleet of servers running Stenographer, the gRPC channel only supports encryption with client authentication and expects the administrator to use certificates that are managed separately from those generated by `stenokeys.sh` (for easily generating certificates, take a look at Square's [certstrap](https://github.com/square/certstrap) utility). The protobuf that defines Stenographer's gRPC service can be found in protobuf/steno.proto.
+
+gRPC support is optional and can be enabled by adding an Rpc dictionary of settings to `steno.conf`. An example configuration is shown below:
+```json
+, "Rpc": { "CaCert": "/path/to/rpc/ca/cert"
+         , "ServerKey": "/path/to/rpc/key"
+         , "ServerCert": "/path/to/rpc/cert"
+         , "ServerPort": 8443
+         , "ServerPcapPath": "/path/to/rpc/pcap/directory"
+         , "ServerPcapMaxSize": 1000000000
+         , "ClientPcapChunkSize": 1000
+         , "ClientPcapMaxSize": 5000000
+  }
+```
+
+#### RetrievePcap ####
+
+This call allows clients to remotely retrieve PCAP via `stenoread`. To retrieve PCAP, clients send the service a unique identifier, the size of PCAP file chunks to stream in return, the maximum size of the PCAP file to return, and the `stenoread` query used to parse packet data. In response, clients receive streams of messages containing the unique identifier and PCAP file chunks (which need to be reassembled client-side). Below is a minimalist example (shown in Python) of how a client can request PCAP and save it to local disk:
+```py
+with grpc.secure_channel(server, creds) as channel:
+    stub = steno_pb2_grpc.StenographerStub(channel)
+    pb = steno_pb2.PcapRequest()
+    pb.uid = str(uuid.uuid4())
+    pb.chunk_size = 1000
+    pb.max_size = 500000
+    pb.query = 'after 5m ago and tcp'
+    pcap_file = os.path.join('.', '{}.pcap'.format(uid))
+    with open(pcap_file, 'wb') as fout:
+        for response in stub.RetrievePcap(pb):
+            fout.write(response.pcap)
+```
+
+`RetrievePcap` requires the gRPC server to be configured with the following fields (in addition to any fields that require the server to startup):
+- ServerPcapPath: local path to the directory where `stenoread` PCAP is temporarily stored
+- ServerPcapMaxSize: upper limit on how much PCAP a client is allowed to receive (used to restrict clients from receiving excessively large PCAPs)
+- ClientPcapChunkSize: size of the PCAP chunks to stream to the client (used if the client has not specified a size in the request)
+- ClientPcapMaxSize: upper limit on how much PCAP a client will receive (used if the client has not specified a size in the request)
 
 ### Defense In Depth ###
 

config/config.go

@@ -47,8 +47,21 @@ type ThreadConfig struct {
 	MaxDirectoryFiles  int `json:",omitempty"`
 }
 
+// RpcConfig is a json-decoded configuration for running the gRPC server.
+type RpcConfig struct {
+        CaCert                  string
+        ServerKey               string
+        ServerCert              string
+        ServerPort              int
+        ServerPcapPath          string
+        ServerPcapMaxSize       int64
+        ClientPcapChunkSize     int64
+        ClientPcapMaxSize       int64
+}
+
 // Config is a json-decoded configuration for running stenographer.
 type Config struct {
+        Rpc           *RpcConfig
 	StenotypePath string
 	Threads       []ThreadConfig
 	Interface     string

protobuf/steno.proto

@@ -0,0 +1,31 @@
+// Copyright 2019 Josh Liburdi and Google Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+syntax = "proto3";
+package steno;
+
+service Stenographer {
+  rpc RetrievePcap(PcapRequest) returns (stream PcapResponse) {}
+}
+
+message PcapRequest {
+  string uid = 1;
+  int64 chunk_size = 2;
+  int64 max_size = 3;
+  string query = 4;
+}
+
+message PcapResponse {
+  string uid = 1;
+  bytes pcap = 2;
+}