Foo Protocol Proxy

  _____             ____            _                  _   ____                      
 |  ___|__   ___   |  _ \ _ __ ___ | |_ ___   ___ ___ | | |  _ \ _ __ _____  ___   _ 
 | |_ / _ \ / _ \  | |_) | '__/ _ \| __/ _ \ / __/ _ \| | | |_) | '__/ _ \ \/ / | | |
 |  _| (_) | (_) | |  __/| | | (_) | || (_) | (_| (_) | | |  __/| | | (_) >  <| |_| |
 |_|  \___/ \___/  |_|   |_|  \___/ \__\___/ \___\___/|_| |_|   |_|  \___/_/\_\\__, |
                                                                               |___/ 

is a Golang implementation for proxy receiver over Foo protocol.

Table of Contents

• Overview
• Getting Started
  • Prerequisites
  • Installation
  • Test Driver
• Tests
• Coding - Structure & Design
• Todo

Overview

Foo is a simple, powerful, extensible, cloud-native request/response protocol. Foo protocol messages are UTF-8 encoded strings with the following BNF-ish definition:

Msg  := Type <whitespace> Seq [<whitespace> Data] '\n'
Type := "REQ" | "ACK" | "NAK"
Seq  := <integer>
Data := <string without newline>

When the Foo server receives a TCP connection initiated by a Foo client, A session is started and a set of messages are exchanged between the two parties, till the client terminates the connection eventually.

The clients sign their messages to the server with "REQ" along with the required action, and the server might respond with acknowledgement "ACK" or denial "NAK" signature.

Example

A->B: <connect>
A<-B: REQ 1 Hey\n
A<-B: ACK 1 Hello\n
A->B: REQ 2 Hey there\n
A<-B: ACK 2\n
A->B: REQ 3 Hey\n
A->B: REQ 4 Hey\n
A->B: REQ 5 Hey\n
A<-B: ACK 3 What\n
A<-B: ACK 4 What do you want\n
A->B: REQ 6 Hey\n
A<-B: NAK 5 Stop it\n
A<-B: NAK 6 Stop doing that\n
A->B: <disconnect>

The proxy has a reporting features like:

• Reporting stats in JSON format to stdout when sending SIGUSR2 signal to the process.
• Reporting stats in JSON format over HTTP /stats or /metrics.
• Health check over HTTP /health or /status.
• Data recovery after failure using a sliding window of 10s time frame.

JSON Sample Response

{"msg_total":10,"msg_req":10,"msg_ack":8,"msg_nak":2,"request_rate_1s":0.005,"request_rate_10s":0.004,"response_rate_1s":0.004,"response_rate_10s":0.003}

Getting Started

Prerequisites

• Golang installation, having $GOPATH properly set.
• Optional, required only if using Docker approach.
  • Docker
  • Docker Compose

Installation

To install Foo Protocol Proxy

$ go get github.com/ahmedkamals/foo-protocol-proxy

Test Driver

You can use the following steps as a testing procedure

• Server

  $ build/assets/bin/server-$(uname -s | tr '[:upper:]' '[:lower:]') -listen ":8001"

• Proxy

  • Normal Approach

    Running proxy on the host directly.

    $ make help get-deps go-install
      export OS=$(uname -s | tr '[:upper:]' '[:lower:]')
      export ARCH=amd64
      export FORWARDING_PORT=":8001"
      export LISTENING_PORT=":8002"
      export HTTP_ADDRESS="0.0.0.0:8088"
      export RECOVERY_PATH=".data/recovery.json"
      export ARGS="-forward $FORWARDING_PORT -listen $LISTENING_PORT -http $HTTP_ADDRESS -recovery-path $RECOVERY_PATH"
      make run OS=$OS ARCH=$ARCH args=$ARGS

    Environment:

    • OS - the current operating system, e.g. (linux, darwin, ...etc.) - default is the current operating system.
    • ARCH - the current system architecture, e.g. (amd64, 386)= default is amd64

    Params:

    • FORWARDING_PORT - e.g. ":8001"
    • LISTENING_PORT - e.g. ":8002"
    • HTTP_ADDRESS - e.g. "0.0.0.0:8088"
    • RECOVERY_PATH - e.g. ".data/recovery.json"

    Sending SIGUSR2 Signal

    $ make kill args=-SIGUSR2

  • Docker Approach

    Running proxy through docker container. currently Linux only works fine.

    $ export ARCH="amd64"
      export FORWARDING_PORT=8001
      export LISTENING_PORT=8002
      export HTTP_ADDRESS=8088
      export RECOVERY_PATH=".data/recovery.json"
      export ARGS="-f $FORWARDING_PORT -l $LISTENING_PORT -h $HTTP_ADDRESS -r $RECOVERY_PATH"
      make deploy ARCH=$ARCH args=$ARGS

    Params:

    • f - e.g. 8001
    • l - e.g. 8002
    • h - e.g. 8088
    • r - e.g. "data/recovery.json"
    • p - e.g. "ahmedkamal"
    • t - e.g. "0.0.1"

    Sending SIGUSR2 Signal

    $ make docker-kill args=-SIGUSR2
      make docker-logs

• Multiple Client Connections

  $ for i in {0..100}
    do 
       docker start foo.client;
    done

Tests

Not all items covered, just made few examples.

$ make test

Coding - Structure & Design

Item	Description
Dispatcher	parses configuration, builds configuration object, and passes it to the proxy.
Proxy	orchestrates the interactions between the components.
Listner	awaits for client connections, and on every new connection, a BridgeConnection instance is created.
BridgeConnection	acts as Bi-directional communication object, that passes data forward and backward to the server.
Analyzer	performs analysis by sniffing all the data read and written from the server.
Stats	wraps stats data the would be flushed to stdout upon request.
TimeTable	contains snapshot of aggregated number of requests/responses at specific timestamp.
Saver	handles reading/writing data.
Recovery	handles storing/retrieval of backup data.
HTTPServer	reports metrics/stats over HTTP using the path /stats or /mertrics, also used for health check using /health /heatrbeat or /status.

Todo:

- Resource pooling for connections, to enable reuse of a limited number of open connections with the server, and to requeue unused ones. - Performance and memory optimization. - More unit tests coverage. - Refactoring.

Enjoy!