Need to better document that toObject() is *not* proto3 JSON

[markjen]

proto3 JSON mapping specifies special output for some well known types (e.g. Timestamp, Duration). However, the generated _pb.js files for these types treats them as regular protobufs.

Repro code:

var tspb = require('google-protobuf/google/protobuf/timestamp_pb.js');
var t = new tspb.Timestamp();
console.log(t.toObject());

Expected output:
0001-01-01T00:00:00Z

Observed output:
{ seconds: 0, nanos: 0 }

[xfxyjwf]

This is a known issue. The JSON support for well-known types hasn't been implemented in javascript yet.

[haberman]

Yes, to be clear, toObject() is not proto3 JSON, it is a pre-existing JSON mapping that does not conform to proto3 JSON.

[haberman]

I suppose the only remaining issue here is that we need to better document that toObject() is not proto3 JSON.

[pradyuman]

Is there a toObject function that does conform to proto3 JSON? Currently, enums are not being translated with the "string" value and that's making my development a real pain.

[haberman]

No, proto3 JSON is currently not supported with JavaScript.

Currently there is not a lot of demand for this inside Google. And whenever we add a feature we have to make sure it doesn't increase code size for people who are not using that feature.

If there was a lot of demand for this from open-source contributors, we could consider how this could be added without causing code size problems.

[pradyuman]

So what's a good way to get around this limitation right now? I was thinking about sending protos over the wire to my react website, but this means I need to bundle the generated protos within my overall javascript bundle which increases the size of the overall payload in a way normal JSON would not.

I feel like if proto3 is going to be the next major evolution of protobuf, eventually a proper toObject function will need to be created. One of the features about proto3 that was supposed to be pretty big was interoperability with JSON. Without proper json encoding/decoding functions, it makes it incredibly hard to use grpc/proto3 without lots of work to patch over these kinds of limitations.

[oseiskar]

I'm also appalled at this missing feature. The documentation makes you believe in language-independent support for serialization in both binary protobuf and JSON. Don't like protobuf & GRPC in your API? No problem, proto3 has a built-in JSON-mapping providing an effortless conversion to "traditional" REST/JSON. Protouf would just define a schema and provide some validation.

But this does not really work in JavaScript since the JSON will be different from what you would get from a Java or Python back-end.

This would be understandable if it only affected some hacky types like Any, but all you need is an enum (AN_OPTION vs 1) or a Timestamp (ISO-8601 vs { seconds: 123, nanos: 456 }) and you're in trouble. I think this level of missing functionality would need a big not supported in all programming languages warning in right in the main documentation chapter on JSON support, not a nasty surprise hidden in the language-specific docs.

[zefyrr]

Agree with @oseiskar that documentation needs to be updated. How can we push for support of proto3 json in javascript? Code-size issues surely can be solved via 'require', no?

[chadbr]

Currently there is not a lot of demand for this inside Google. And whenever we add a feature we have to make sure it doesn't increase code size for people who are not using that feature.

If there was a lot of demand for this from open-source contributors, we could consider how this could be added without causing code size problems.

@haberman for what it's worth...

I've discussed using protobuf instead of json over http with countless people. The conclusion is usually 'Google does not seem interested'.

I think there is a bit of a double edged sword here -- the community doesn't show interest because Google doesn't show interest.

I, for one, have been looking for 1st class browser support for protobuf for several years.

Thanks, Chad