dare-cli

Overview

for demo purposes, we have created a cli daemon using hashicorp's plugin-over-rpc system.

the system has 3 binaries :

• encryptor-plugin : a single binary that just encrypts data at a given path . it cannot be executed on it's own ,main dare daemon has to be the one that starts it due to using plugin-over-rpc system
• decryptor-plugin: a single binary that just decrypts data at a given path . it cannot be executed on it's own ,main dare daemon has to be the one that starts it due to using plugin-over-rpc system
• dare : main software . it starts a long running daemon. it also exposes a json RPC 2.0 api endpoint

users interact with the api endpoint it exposes. By sending curl request to localhost:8080/rpc it executes enxryption requests or decryption request; the type of service it executes is based on the json request it recieves. How it executes request ? lets say it got a request for encryption :

1. It would start the binary that is encryptor plugin (path to that binary has to be fed to dare at start , eg bin/encryptor-plugin)

2. it would establish connection over tcp socket with the binary . at this instance, dare daemon would act as a client of encryptor plugin which behaves as server. there is a handshake process based on a shared secret for dare-plugin connection establishment

3. the plugin would do it's job, i.e open file, encrypt the file and store it and it would send a reply back to dare daemon with post and pre encryption hashes.

4. dare daemon would reply to user a json message that is its reply.

request procedure calls

message format

messages are autogenerated with protocol buffer compiler. the same message format is used both for marshalling/unmarshalling JSON messages and gRPC messages. the protocol buffer file is located at $PWD/model/model.proto

the following are our messages

message Hash {
    string md5 = 1;
    string sha256 = 2;
}
message EncryptRequest {
    string source = 1;
    string destination = 2;
    string key = 3;
}

message EncryptResponse {
    Hash output_hash = 1;
    string random_nonce = 2;
    string random_key = 3;
}

message DecryptRequest {
    string source = 1;
    string destination = 2;
    string nonce = 3;
    string key = 4;

}

message DecryptResponse {
    Hash output_hash = 1;
}

in encrypt request , a source file path (plain text input) , destination path (encrypted ouput) and an optional 32 byte encryption key is needed. in case encryption key is not provided, the plugin backend would automatically generate a random key and put it in random_key field of encrypt response. a random nonce is always generated and included in encrypt response. for validation purposes , encrypted file md5 hash and sha256 is also inclueded in the response

in decrypt request , besides a source path (encrypted file input) and destination path (decrypted output), a nonce and encryption key is required. nonce is autogenerated and should be taken from encrypt response . encryption key is either already known (user-defined) or autogenerated and included in encrypt response which must be provided to this request. decrypt response contains md5 hash and sha256 hash of decrypted file.

daemon <-> plugin

the comminucation between dare daemon and plugin's uses google's protocol buffer encoding format and gRPC for connection esteblishment and rpc execution.

the code for encoding/decoding messages and establishing connection between client and server , and message exchange is auto generated with protocol buffer compiler(protoc). to recompile the protocol buffer file, open terminal in $PWD/model and run the following

make proto

keep in mind that besides main protoc binary , you would need to install the other dependancies, which can be installed by running the following int terminal

GO111MODULE=off go get -v github.com/golang/protobuf/protoc-gen-go
GO111MODULE=off go get -v github.com/gogo/protobuf/proto
GO111MODULE=off go get -v github.com/gogo/protobuf/jsonpb
GO111MODULE=off go get -v github.com/gogo/protobuf/protoc-gen-gogo
GO111MODULE=off go get -v github.com/gogo/protobuf/gogoproto
GO111MODULE=off go get -v github.com/gogo/protobuf/protoc-gen-gofast
GO111MODULE=off go get -v github.com/gogo/protobuf/protoc-gen-gogofast
GO111MODULE=off go get -v github.com/gogo/protobuf/protoc-gen-gogofaster
GO111MODULE=off go get -v github.com/gogo/protobuf/protoc-gen-gogoslick 

thanks to using plugin-over-rpc system , anyone can write plugin for dare daemon in any language and expand it's capabalities; since message exchange between dare daemon and plugin happens over tcp socket and decoder and encoder is autogenerated based on language with protoc, which can generate code for any language.

to learn more about plugin-over-rpc format, it's internals and security model head over and read
go-plugin internals

daemon <-> user

comminucation between dare daemon and user uses simple json encoding and following JSON RPC 2.0 specification due to using this specification, our api endpoint is significantly simpler, in fact, all rpc calls are send to a single endpoint /rpc.

JSON RPC 2.0 has a very specific message format. the following are examples for Encrypt Request and Decrypt Request JSON messages :

• Encrypt Request

{
  "jsonrpc": "2.0",
  "method": "Service.Encrypt",
  "params": {
    "source": "/tmp/plain",
    "destination": "/tmp/encrypted",
    "key": "63b76723eb3f9d4f4862b73ff7e39b93c4de129feb4885f1f3feb74dd456e3a5"
  },
  "id": "1"
}

• Decrypt Request

{
  "jsonrpc": "2.0",
  "method": "Service.Decrypt",
  "params": {
    "source": "/tmp/encrypted",
    "destination": "/tmp/decrypted",
    "nonce": "e12ffdfa6cb6e56238935e32604cfa5538d3ad51a3542daa",
    "key": "63b76723eb3f9d4f4862b73ff7e39b93c4de129feb4885f1f3feb74dd456e3a5"
  },
  "id": "2"
}

assuming you have curl and jq installed, are using a POSIX shell , and the api is running at port 8080, you can encode and send messages with the following commands

• Encrypt Request

jq -n \
  --arg source "/tmp/plain" \
  --arg destination "/tmp/encrypted" \
  --arg key "63b76723eb3f9d4f4862b73ff7e39b93c4de129feb4885f1f3feb74dd456e3a5" \
  --arg id "1" \
  --arg method "Service.Encrypt" \
 '{"jsonrpc": "2.0", "method":$method,"params":{"source": $source, "destination":$destination,"key":$key},"id": $id}' | curl \
    -X POST  \
 	--silent \
	--header "Content-type: application/json" \
    --data @- \
    http://127.0.0.1:8080/rpc  | jq -r

• Decrypt Request

jq -n \
  --arg source "/tmp/encrypted" \
  --arg destination "/tmp/decrypted" \
	 --arg nonce "e12ffdfa6cb6e56238935e32604cfa5538d3ad51a3542daa" \
  --arg key "63b76723eb3f9d4f4862b73ff7e39b93c4de129feb4885f1f3feb74dd456e3a5" \
  --arg id "2" \
  --arg method "Service.Decrypt" \
 '{"jsonrpc": "2.0", "method":$method,"params":{"source": $source, "destination":$destination, "nonce":$nonce, "key":$key},"id": $id}' | curl \
	-X POST  \
	--silent \
    --header "Content-type: application/json" \
    --data @- \
    http://127.0.0.1:8080/rpc | jq -r

build

to simplify build process, we have prepared a pipeline with gnu make. the make pipeline is located at $PWD/build and go targets are under $PWD/build/target/go. the pipeline is highly customizable so one can experiment with it. It also can run build in a docker container. build environment configuration is located under $PWD/build/target/buildenv.

• to build for your current os run make build. build result binaries would be under $PWD/bin.
• to cross-compile for linux current os run make build-linux.build result binaries would be under $PWD/bin/linux.
• to cross-compile for darwin (mac os) current os run build-mac-os.build result binaries would be under $PWD/bin/darwin/.
• to cross-compile for windows current os run build-windows . build result binaries would be under $PWD/bin/windows. change the extension to .exe after build.

subcommands

daemon

this is the main subcommand.It starts data at rest encryption daemon. it is a long running process that exposes an API endpoint at /rpc which intercepts user JSON messages, relays them to encrypt/decrypt plugins.

considering that it is a daemon , it would be best to start it as a background process so that it doesn't block your terminal use the following command to start the daemon as a background process , assuming that you are running a posix shell and $PWD/server.log is where you want to store deamon output.

port="8080";output="$PWD/server.log";$PWD/bin/dare daemon --api-addr="127.0.0.1:$port" > "$output" 2>&1 &

to kill any existing daemon process ,in case there is a runtime error which is mostly due to trying to bind to a port that a previously daemon is already bound to , run the following

for pid in $(ps  | grep "dare" | awk '{print $$1}'); do kill -9 "$pid"; done

to simplify starting the daemon, we have already created a make target in main makefile located at $PWD/Makefile. simply run make run which would kill any running daemon instances and starts it in background mode. to change the port it binds to , update PORT variable at the start of $PWD/Makefile.

keygen

coming up with 32 byte long hex encoded encryption key string can be tedious. this subcommand helps with randomly generating a new 32 byte long hex encoded encryption key string.

dd

most often in linux , to generate random files , we use gnu coreutils dd tool . the following is an example that generates a 50 MB file with dd and stores it at /tmp/plain

dd if=/dev/urandom of=/tmp/plain bs=1048576 count=50

the generated file is a human unreadable blob.

now, one might be using windows in which dd is not available or wants to generate a human readable blob, then they can use dare dd subcommand. it generates a human readable json file, filled with lorem ipsums.

to simplify random file generation, we have setup two make targets in main makefile make linux-dd and make dd

• make linux-dd generates random blob with gnu coreutils dd tool
• make dd generates a json file filled with random data using dare dd subcommand

to customize behavior , change FILE_SIZE and PLAIN_PATH variables at the start of $PWD/Makefile. keep in mind that the unit of number in FILE_SIZE is Megabytes.

demo

to fast track and simplify demo process , we have created two make targets make demo-encrypt and make demo-decrypt

follow the following procedure to build, run , generate random file, encrypt and decrypt it :

1. build : make build

2. run daemon : make run you can see server output by opening server.log . e.g:

tail -f server.log

3. generate a random file either by running make dd or make linux-dd .

4. try encrypting file by running make demo-encrypt . based on current makefile variables, it generates a random 50MB file and encrypts it

5. open $PWD/Makefile and update NONCE and KEY variables in it with the response you get from daemon

6. run make demo-decrypt