Skip to content
GPB and gRPC testing
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
gclient Add security to gRPC connection Apr 30, 2017
gproto Add client, folder separation Apr 24, 2017
gserver Add security to gRPC connection Apr 30, 2017
.gitignore .DS_Store gone forever Apr 30, 2017
LICENSE Initial commit Apr 11, 2017
README.md Add two ways to read a protobuf file Mar 22, 2018
add_router.go Adding first GPB encode/decode example Apr 11, 2017
cert.pem Add security to gRPC connection Apr 30, 2017
devices.proto Delete redundant proto file Apr 24, 2017
key.pem Add security to gRPC connection Apr 30, 2017
list_routers.go Add example to create a JSON output May 5, 2017
routers.data Add client, folder separation Apr 24, 2017
routers.json Add example to create a JSON output May 5, 2017
save_json.go

README.md

gRPC and GPB for Networking Engineers

GPB and gRPC testing. Based on the protobuf examples and Pluralsight training.

Table of contents

Code Examples

	routers := &pb.Routers{}
	router := &pb.Router{}

	router.IP = []byte("2001:db8::123:44:4")
	router.Hostname = "router4.cisco.com"

	routers.Router = append(routers.Router, router)

If we inspect routers.data.

$ hexdump -c routers.data
0000000  \n   &  \n 020   r   o   u   t   e   r   .   c   i   s   c   o
0000010   .   c   o   m 022 022   2   0   0   1   :   d   b   8   :   :
0000020   1   2   3   :   1   2   :   1  \n   '  \n 021   r   o   u   t
0000030   e   r   2   .   c   i   s   c   o   .   c   o   m 022 022   2
0000040   0   0   1   :   d   b   8   :   :   1   2   3   :   1   2   :
0000050   2  \n   '  \n 021   r   o   u   t   e   r   3   .   c   i   s
0000060   c   o   .   c   o   m 022 022   2   0   0   1   :   d   b   8
0000070   :   :   1   2   3   :   3   3   :   3  \n   '  \n 021   r   o
0000080   u   t   e   r   4   .   c   i   s   c   o   .   c   o   m 022
0000090 022   2   0   0   1   :   d   b   8   :   :   1   2   3   :   4
00000a0   4   :   4
00000a3
$ cat routers.data | protoc --decode_raw
1 {
  1: "router.cisco.com"
  2: "2001:db8::123:12:1"
}
1 {
  1: "router2.cisco.com"
  2: "2001:db8::123:12:2"
}
1 {
  1: "router3.cisco.com"
  2: "2001:db8::123:33:3"
}
1 {
  1: "router4.cisco.com"
  2: "2001:db8::123:44:4"
}
in, err := ioutil.ReadFile(fname)
if err != nil {
	log.Fatalln("Error reading file:", err)
}
routers := &pb.Routers{}
if err := proto.Unmarshal(in, routers); err != nil {
	log.Fatalln("Failed to parse the routers file:", err)
}
  • data.go assigns values to different instances of our Routers struct. Example:
var router = []*pb.Router{
	&pb.Router{
		Hostname: "router1.cisco.com",
		IP:       []byte("2001:db8::111:11:1"),
	},
}

routers := pb.Routers{router}
type server struct{}

func (s *server) GetByHostname(ctx context.Context,
	in *pb.GetByHostnameRequest) (*pb.Router, error) {
	return nil, nil
}
...
conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
	log.Fatalf("did not connect: %v", err)
}
defer conn.Close()

client := pb.NewDeviceServiceClient(conn)
...

Compiling your protocol buffers

  • protoc --go_out=gproto devices.proto only defines the GPB part, to read and write as demonstrated in list_routers.go and add_router.go.
  • protoc --go_out=plugins=grpc:gproto devices.proto adds the RPC services. It creates gproto/devices.pb.go. You need this one to run the client and server below.

Understanding GPB encoding

Let's print out the GPB encoded slice of bytes

out, err := proto.Marshal(routers)
if err != nil {
	log.Fatalln("Failed to encode routers:", err)
}
fmt.Printf("%X", out)

After grouping the output for convenience, we get something like:

0A 26 0A 10 72 6F 75 74 65 72 2E 63 69 73 63 6F 2E 63 6F 6D
12 12 32 30 30 31 3A 64 62 38 3A 3A 31 32 33 3A 31 32 3A 31
0A 27 0A 11 72 6F 75 74 65 72 32 2E 63 69 73 63 6F 2E 63 6F 6D
12 12 32 30 30 31 3A 64 62 38 3A 3A 31 32 33 3A 31 32 3A 32
0A 27 0A 11 72 6F 75 74 65 72 33 2E 63 69 73 63 6F 2E 63 6F 6D
12 12 32 30 30 31 3A 64 62 38 3A 3A 31 32 33 3A 33 33 3A 33
0A 27 0A 11 72 6F 75 74 65 72 34 2E 63 69 73 63 6F 2E 63 6F 6D
12 12 32 30 30 31 3A 64 62 38 3A 3A 31 32 33 3A 34 34 3A 34

Considering the definitions on the proto file (devices.proto)

message Router {
  string hostname = 1;
  bytes IP = 2; 
}

message Routers {
  repeated Router router = 1;
}

Protobuf uses Varint to serialize integers. The last three bits of the number store the wire type. Having this in mind and how to convert Hex to ASCII, the first 40 bytes (or two rows from the output) translate to:

Hex  Description
0a  tag: router(1), field encoding: LENGTH_DELIMITED(2)
26  "router".length(): 38
0a  tag: hostname(1), field encoding: LENGTH_DELIMITED(2)
10  "hostname".length(): 16 
72 'r'
6F 'o'
75 'u'
74 't'
65 'e'
72 'r'
2E '.'
63 'c'
69 'i'
73 's'
63 'c'
6F 'o'
2E '.'
63 'c'
6F 'o'
6D 'm'
12 tag: IP(2), field encoding: LENGTH_DELIMITED(2)
12 "IP".length(): 18
32 '2'
30 '0'
30 '0'
31 '1'
...
31 '1'

Its equivalent in JSON would be something like this (routers.json):

{
  "Router": [
    {
      "Hostname": "router.cisco.com",
      "IP": "2001:db8::123:12:1"
    }
  ]
}

Understanding Go proto code

Marshal takes the protocol buffer and encodes it into the wire format, returning the data.

func Marshal(pb Message) ([]byte, error)

Unmarshal parses the protocol buffer representation in buf and places the decoded result in pb

func Unmarshal(buf []byte, pb Message) error

Message is implemented by generated protocol buffer messages.

type Message interface {
    Reset()
    String() string
    ProtoMessage()
}

In our example generated code devices.pb.go, Router and Routers structs are defined

type Router struct {
	Hostname string `protobuf:"bytes,1,opt,name=hostname" json:"hostname,omitempty"`
	IP       []byte `protobuf:"bytes,2,opt,name=IP,proto3" json:"IP,omitempty"`
}
type Routers struct {
	Router []*Router `protobuf:"bytes,1,rep,name=router" json:"router,omitempty"`
}

Both implement the Message interface

func (m *Router) Reset()                    { *m = Router{} }
func (m *Router) String() string            { return proto.CompactTextString(m) }
func (*Router) ProtoMessage()               {}
func (m *Routers) Reset()                    { *m = Routers{} }
func (m *Routers) String() string            { return proto.CompactTextString(m) }
func (*Routers) ProtoMessage()               {}

Compiling the code

  • gRPC client: go build -o client gclient/main.go
  • gRPC server: go build -o server gserver/*.go

Running some examples

  • Examples are pretty static for now. The client just executes a method based on the arguments the command line provides.
switch *option {
case 1:
	SendMetadata(client)
case 2:
	GetByHostname(client)
case 3:
	GetAll(client)
case 4:
	Save(client)
case 5:
	SaveAll(client)
}
  • SaveAll looks like this, the client prints the devices it wants to add and the server prints the new complete list.
$ ./client -o 5
hostname:"router8.cisco.com" IP:"2001:db8::888:88:8" 
hostname:"router9.cisco.com" IP:"2001:db8::999:99:9" 
$ ./server
2017/04/29 20:27:35 Starting server on port :50051
hostname:"router1.cisco.com" IP:"2001:db8::111:11:1" 
hostname:"router2.cisco.com" IP:"2001:db8::222:22:2" 
hostname:"router3.cisco.com" IP:"2001:db8::333:33:3" 
hostname:"router8.cisco.com" IP:"2001:db8::888:88:8" 
hostname:"router9.cisco.com" IP:"2001:db8::999:99:9"

Generating Server Certificate and Private Key

This is optional in order to generate secure connections. We create a new private key 'key.pem' and a server certificate 'cert.pem'

$ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=localhost'
Generating a 4096 bit RSA private key
................................++
.....................................................++
writing new private key to 'key.pem'
-----
$ 

Links

You can’t perform that action at this time.