Overview:

protoc-wedge compiler is an extension to protoc-c and can be used with the following options:

1. avro_out: Generate Apache Avro wedge broker schema from protobuf
2. desc_out: Generate JSON descriptor table from protobuf
3. avsc_out: Generate Apache Avro schema (i.e avsc) from protobuf
4. go_out: Generate golang structs from protobuf for use with JSON

pre-requisites

• protobuf 3

Installation:

Once you download the source code from git, update the protobuf-c submodule that is referenced using the following commands:

$ git submodule init

$ git submodule update

Now run the following commands to perform the instalation:

$ autoreconf --install

$ ./configure

$ make

$ sudo make install

On successful completion of the installation, protoc-thrift binary would be added to /usr/local/bin

Usage:

JSON descriptor map generation:

To generate the JSON descriptor map corresponding to a set of proto files test1.proto, test2.proto, run the following command:

$protoc-wedge --desc_out=. test1.proto test2.proto

The resultant file "WedgeProtoDescTable.json" will be generated in the same directory. This file will be used an an input for JSON marshalling/unmarshalling and can be used by gRPC, kafka plugins

Avsc descriptor map generation:

To generate the avsc schema(s) correponding to a set of proto files test1.proto, test2.proto, run the following command:

$protoc-wedge --avsc_out=. test1.proto test2.proto

An avsc file will be generated for each proto file. The proto files having services specified (i.e) RPCs will have all the record including imports as a part of single avsc file. The order of imports should be maintained and the dependant files should be generated before the proto files importing them. For example, if c.proto imports from b.proto which in turn imports from a.proto, then the order of generation will be

$protoc-wedge --avsc_out=. a.proto

$protoc-wedge --avsc_out=. a.proto b.proto

$protoc-wedge --avsc_out=. a.proto b.proto c.proto

c.avsc corresponding to c.proto will have all the information from a.proto as well as b.proto so that the avsc file for a service has all the record information needed. The multiple steps with invoking the compiler is due to the fact that the avsc file generated from an imported file is required for generating the schema of a subsequent file

This file will be provided an an input for marshalling/unmarshalling avro binary data to go structs and will be used by kafka plugin

Avro descriptor map generation:

To generate the Avro descriptor map correponding to a set of proto files test1.proto, test2.proto, run the following command:

$protoc-wedge --avro_out=. test1.proto test2.proto

The resultant file "WedgeAvroDescTable.json" will be generated in the same directory. This file will be provided as an input for conversion between avro and protobuf in the wedge broker.

Golang structs generation:

To generate golang structs to execute RPCs from a set of proto files test1.proto, test2.proto, run the following command:

$protoc-wedge --go_out=. test1.proto test2.proto

The resultant file "WedgeDesc.go" will have the json marshalling information to generate the JSON Payload for each RPC. This functionality will eventually be extended to marshal more formarts in the future for go clients.