Protocol Buffers are a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more, originally designed at Google (see).
protobuf.js is a pure JavaScript implementation for node and the browser. It efficiently encodes plain objects and custom classes and works out of the box with .proto files.
Recommended read: Changes in protobuf.js 6.0
- Optimized for performance
- Exhaustive browser support
- Managed TypeScript definitions
- Elaborate API documentation
- Convenient CLI utilities
- Seamless browserify integration
-
Usage
How to include protobuf.js in your project. -
Examples
A few examples to get you started. -
Module Structure
A brief introduction to the structure of the exported module. -
Documentation
A list of available documentation resources. -
Command line
How to use the command line utility. -
Building
How to build the library and its components yourself. -
Performance
A few internals and a benchmark on performance. -
Compatibility
Notes on compatibility regarding browsers and optional libraries.
$> npm install protobufjs
var protobuf = require("protobufjs");
Development:
<script src="//cdn.rawgit.com/dcodeIO/protobuf.js/6.1.0/dist/protobuf.js"></script>
Production:
<script src="//cdn.rawgit.com/dcodeIO/protobuf.js/6.1.0/dist/protobuf.min.js"></script>
The protobuf
namespace will be available globally.
NOTE: Remember to replace the version tag with the exact release your project depends upon.
// awesome.proto
package awesomepackage;
syntax = "proto3";
message AwesomeMessage {
string awesome_field = 1; // becomes awesomeField
}
protobuf.load("awesome.proto", function(err, root) {
if (err) throw err;
// Obtain a message type
var AwesomeMessage = root.lookup("awesomepackage.AwesomeMessage");
// Create a new message
var message = AwesomeMessage.create({ awesomeField: "AwesomeString" });
// Encode a message
var buffer = AwesomeMessage.encode(message).finish();
// ... do something with buffer
// Or, encode a plain object
var buffer = AwesomeMessage.encode({ awesomeField: "AwesomeString" }).finish();
// ... do something with buffer
// Decode a buffer
var message = AwesomeMessage.decode(buffer);
// ... do something with message
// If your application uses length-delimited buffers, there is also encodeDelimited and decodeDelimited.
});
You can also use promises by omitting the callback:
protobuf.load("awesome.proto")
.then(function(root) {
...
});
...
var Root = protobuf.Root,
Type = protobuf.Type,
Field = protobuf.Field;
var AwesomeMessage = new Type("AwesomeMessage").add(new Field("awesomeField", 1, "string"));
var root = new Root().define("awesomepackage").add(AwesomeMessage);
// Continue at "Create a new message" above
...
...
function AwesomeMessage(properties) {
protobuf.Message.call(this, properties);
}
protobuf.Class.create(root.lookup("awesomepackage.AwesomeMessage") /* or use reflection */, AwesomeMessage);
var message = new AwesomeMessage({ awesomeField: "AwesomeString" });
// Continue at "Encode a message" above
Custom classes are automatically populated with static encode
, encodeDelimited
, decode
, decodeDelimited
and verify
methods and reference their reflected type via the $type
property. Note that there are no methods (just $type
) on instances by default as method names might conflict with field names.
// greeter.proto
service Greeter {
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}
...
var Greeter = root.lookup("Greeter");
var greeter = Greeter.create(rpcImpl, false, false); // rpcImpl (see below), requestDelimited?, responseDelimited?
greeter.sayHello({ name: 'you' }, function(err, response) {
console.log('Greeting:', response.message);
});
To make this work, all you have to do is provide an rpcImpl
, which is an asynchronous function that takes the reflected service method, the binary HelloRequest and a node-style callback as its parameters. For example:
function rpcImpl(method, requestData, callback) {
// perform the request using an HTTP request or a WebSocket for example
var responseData = ...;
// and call the callback with the binary response afterwards:
callback(null, responseData);
}
There is also an example for streaming RPC.
/// <reference path="node_modules/protobufjs/types/protobuf.js.d.ts" />
import * as protobuf from "protobufjs";
...
The library exports a flat protobuf
namespace including but not restricted to the following members, ordered by category:
-
load(filename:
string|string[]
, [root:Root
], [callback:function(err: Error, [root: Root])
]):Promise|undefined
[source]
Loads one or multiple .proto or preprocessed .json files into a common root namespace. -
loadSync(filename:
string|string[]
, [root:Root
]):Root
[source]
Synchronously loads one or multiple .proto or preprocessed .json files into a common root namespace (node only). -
parse(source:
string
):Object
[source]
Parses the given .proto source and returns an object with the parsed contents.
-
Writer [source]
Wire format writer usingUint8Array
if available, otherwiseArray
. -
Reader [source]
Wire format reader usingUint8Array
if available, otherwiseArray
.
-
Namespace extends ReflectionObject [source]
Base class of all reflection objects containing nested objects. -
Root extends Namespace [source]
Root namespace. -
Type extends Namespace [source]
Reflected message type. -
Field extends ReflectionObject [source]
Reflected message field. -
MapField extends Field [source]
Reflected message map field. -
Enum extends ReflectionObject [source]
Reflected enum. -
Service extends Namespace [source]
Reflected service. -
Method extends ReflectionObject [source]
Reflected service method.
-
Class [source]
Runtime class providing the tools to create your own custom classes. -
Message [source]
Abstract runtime message.
- util [source]
Various utility functions.
For less common members, see the API documentation.
The pbjs
command line utility can be used to bundle and translate between .proto and .json files.
Consolidates imports and converts between file formats.
-t, --target Specifies the target format. Also accepts a path to require a custom target.
json JSON representation
json-module JSON representation as a module (AMD, CommonJS, global)
proto2 Protocol Buffers, Version 2
proto3 Protocol Buffers, Version 3
static Static code without reflection
static-module Static code without reflection as a module (AMD, CommonJS, global)
-p, --path Adds a directory to the include path.
-o, --out Saves to a file instead of writing to stdout.
-w, --wrap Specifies the wrapper to use for *-module targets. Also accepts a path.
default Default wrapper supporting both CommonJS and AMD
commonjs CommonJS only wrapper
amd AMD only wrapper
-r, --root Specifies an alternative protobuf.roots name for *-module targets.
usage: pbjs [options] file1.proto file2.json ...
For production environments it is recommended to bundle all your .proto files to a single .json file, which reduces the number of network requests and parser invocations required:
$> pbjs -t json file1.proto file2.proto > bundle.json
Now, either include this file in your final bundle:
var root = protobuf.Root.fromJSON(require("./bundle.json"));
or load it the usual way:
protobuf.load("bundle.json", function(err, root) {
...
});
To build the library or its components yourself, clone it from GitHub and install the development dependencies:
$> git clone https://github.com/dcodeIO/protobuf.js.git
$> cd protobuf.js
$> npm install --dev
Building the development and production versions with their respective source maps to dist/
:
$> npm run build
Building the documentation to docs/
:
$> npm run docs
Building the TypeScript definition to types/
:
$> npm run types
protobuf.js integrates into any browserify build-process. There are a few possible tweaks:
- If performance is a concern or IE8 support is required, you should make sure to exclude the browserified
buffer
module and let protobuf.js do its thing with Uint8Array/Array instead. - If you do not need int64 support, you can exclude the
long
module. - If your application does not rely on the following modules and package size is a concern, you can also exclude
process
,_process
andfs
. - If you have any special requirements, there is the bundler as a reference.
The package includes a benchmark that tries to compare performance to native JSON as far as this is possible. On an i7-2600K running node 6.9.1 it yields:
benchmarking encoding performance ...
Type.encode to buffer x 479,876 ops/sec ±0.64% (92 runs sampled)
JSON.stringify to string x 311,489 ops/sec ±0.84% (87 runs sampled)
JSON.stringify to buffer x 175,079 ops/sec ±1.48% (82 runs sampled)
Type.encode to buffer was fastest
JSON.stringify to string was 35.2% slower
JSON.stringify to buffer was 63.8% slower
benchmarking decoding performance ...
Type.decode from buffer x 1,267,612 ops/sec ±1.18% (90 runs sampled)
JSON.parse from string x 291,707 ops/sec ±1.12% (92 runs sampled)
JSON.parse from buffer x 262,640 ops/sec ±0.77% (89 runs sampled)
Type.decode from buffer was fastest
JSON.parse from string was 77.0% slower
JSON.parse from buffer was 79.2% slower
benchmarking combined performance ...
Type to/from buffer x 248,897 ops/sec ±0.89% (90 runs sampled)
JSON to/from string x 126,848 ops/sec ±0.75% (92 runs sampled)
JSON to/from buffer x 89,854 ops/sec ±0.79% (93 runs sampled)
Type to/from buffer was fastest
JSON to/from string was 49.0% slower
JSON to/from buffer was 63.9% slower
benchmarking verifying performance ...
Type.verify x 5,941,014 ops/sec ±0.96% (90 runs sampled)
Type.verify was fastest
Note that JSON is a native binding nowadays and as such is about as fast as it possibly can get. So, how can protobuf.js be faster?
- The benchmark is somewhat flawed.
- Reader and writer interfaces configure themselves according to the environment to eliminate redundant conditionals.
- Node-specific reader and writer subclasses benefit from node's buffer binding.
- Reflection has built-in code generation that builds type-specific encoders, decoders and verifiers at runtime.
- Encoders and decoders do not verify that required fields are present (with proto3 this is dead code anyway). There is a
verify
method to check this manually instead - where applicable. - For entirely bogus values encoders intentionally rely on runtime errors to be thrown somewhere down the road.
- Quite a bit of V8-specific profiling is accountable for everything else.
Note that code generation requires new Function(...)
(basically eval
) support and that an equivalent but slower fallback will be used where unsupported.
You can also run the benchmark ...
$> npm run bench
and the profiler yourself (the latter requires a recent version of node):
$> npm run prof <encode|decode|encode-browser|decode-browser> [iterations=10000000]
Note that as of this writing, the benchmark suite performs significantly slower on node 7.2.0 compared to 6.9.1 because moths.
- Because the internals of this package do not rely on
google/protobuf/descriptor.proto
, options are parsed and presented literally. - If typed arrays are not supported by the environment, plain arrays will be used instead.
- Support for pre-ES5 environments like IE8 can be achieved by using a polyfill and, instead of using property getters and setters on reflection objects, calling the respective functions prefixed with
get
,set
oris
directly (i.e. callingType#getFieldsById()
instead of accessingType#fieldsById
). - If you need a proper way to work with 64 bit values (uint64, int64 etc.), you can install long.js alongside this library. All 64 bit numbers will then be returned as a
Long
instance instead of a possibly unsafe JavaScript number (see).
License: Apache License, Version 2.0, bundled external libraries may have their own license