Overview

This small utility setup server/cli: "upload-only" ftp-like server; with TLS + grpc transport.

Motivation

When doing multi-cloud deployment of Hyperledger Fabric, peers of different organizations run on different cloud providers. It needs a mechanism to share tls root certs, and/or crypto material, as an out-of-band communication process. This utility is created as Pod, besides "peer" pod. It enables the out-of-band file exchange process uses the same networking transport of inter-peer communications. A max filesize is 4 MB. TLS is required for SNI-based routing.

Pre-requisite

• Go v1.15 +
• Protocol buffer compiler
• Golang editor

Instructions

NAME:
   gupload - Upload and download files with grpcs

USAGE:
   gupload [global options] command [command options] [arguments...]

VERSION:
   0.0.0

COMMANDS:
   serve     initiates a gRPC upload server (max 4MB per file)
   upload    upload a file (max 4MB per file)
   download  download a file
   help, h   Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help (default: false)
   --version, -v  print the version (default: false)

Use serve to initiate a gRPC server and upload to upload a file to a given address.

Serve GRPC

grpc is the default mechanism used for both clients and servers.

There are two forms of running it:

• via TLS-based http2

To use tls-based connections:

# Create a server
./build/gupload serve --key ./cert/tls.key --certificate ./cert/tls.crt

# When doing local development with above cert/key pair;
# see this issue https://github.com/golang/go/issues/39568
# if we use localhost in the tls cert for local dev, need to set below env
# this workaround may later break, for golang version beyong v1.15
export GODEBUG=x509ignoreCN=0

Upload a file

# Upload a file: with mandatory fields
./build/gupload upload \
    --cacert ./cert/tls.crt \
    --infile README.md \
    --public \
    --outfile README.md \
    --address localhost:1313

If public flag is false, the uploaded filename will be placed at fileserver directory; its filename will be main.go. Or otherwise, the uploaded file will be sent to fileserver/public directory in the server.

Note that gupload client is not able to download non-public files.

The default address is localhost:1313.

Also, can use --servername-override, when TLS is enabled.

Download a file

./build/gupload download \
    --cacert ./cert/tls.crt \
    --file test.txt \
    --address localhost:1313

It will download file from fileserver/public directory.

Credits

The tool is adapted from:

• https://github.com/cirocosta/gupload
• https://github.com/techschool/pcbook-go

Reference Info

• protobuff for go
• go-grpc-tutorial
• youtube tutorial #1
• youtube tutorial #2
• example 1
• example 2
• publish to gh registry
• example 3
• health check example
• health check code

Development

# generate protocol buffers
protoc --proto_path=core --go_out=plugins=grpc:core --go_opt=paths=source_relative core/service.proto

# compile
go build -i -v -o build/gupload main.go

# to trigger the docker image creation and send to Github Container Registry
git tag v0.0.2

git push origin v0.0.2

Manually release is currently used, need to update version number in VERSION.txt, and main.go. It needs to use the same version number above, as git tag number. The GitHub Action will use the tag number, to create and publish docker image to gcr.io.

If later, there needs a richer functionality in gupload, I may consider to change to automatic release numbering.

New Health check

Secure

# secure server
./build/gupload serve --key ./cert/tls.key --certificate ./cert/tls.crt

./build/gupload serve --key ./cert/tlskey-peer0-bochk.pem --certificate ./cert/tlscert-peer0-bochk.pem

# secure client
./build/gupload ping --address localhost:1313 --cacert ./cert/tls.crt

./build/gupload ping --address peer0-bochk:1313 --cacert ./cert/tlscert-peer0-bochk.pem

Insecure

# server
./build/gupload serve

# client
./build/gupload ping --address localhost:1313