Protobuf seems like a lot of overhead for this use case?

[kentonv]

Hello. I'm the guy who wrote Protobuf v2 and open sourced it at Google. (I also, just in the last year, built Cloudflare Workers, a non-Node JavaScript server runtime embedding V8, with a focus on secure sandboxing and high multi-tenancy.)

I was surprised by the choice of Protobuf for intra-process communications within Deno. Protobuf's backwards compatibility guarantees and compact wire representation offer no benefit here, while the serialize/parse round-trip on every I/O seems like it would be pretty expensive.

I suppose the main motivation for using Protobuf here is to get convenient code for transmitting data that is compatible across languages?

Have you considered Cap'n Proto for this? I created Cap'n Proto shortly after leaving Google. It works a lot like Protobuf, except that the generated getters and setters operate directly on a backing byte array (e.g. ArrayBuffer in JavaScript), using a C-struct-like layout. Since you can directly share that byte array between code in different languages, you can construct a message in one language and then consume it in another without even making a copy in between, much less a serialize/parse round trip. Communication between a sandbox and a supervisor was in fact a motivating use case for Cap'n Proto.

Cap'n Proto is well-supported in C++, Rust, and Go. There is also a mostly-complete TypeScript implementation. Admittedly the TypeScript implementation has not received a whole lot of real-world usage, but there's interest I may be willing to directly adopt and support it for you. (I expect to need it in my own work soon anyway.)

[ry]

@kentonv Thank you for pointing this out! To be honest, I was under the impression that the protobuf did operate on the underlying ArrayBuffer. Are there any recent benchmarks comparing PB to Cap'n Proto in JS? I will investigate Cap'n Proto. While you're here, do you have any opinion on FIDL?
(For streaming data, we intend to send raw ArrayBuffers back and forth, and avoiding serialization entirely. But better performance is always better...)

[MasterJames]

Maybe dumb question here but should not SSL handle compression sufficiently in a modern world and the extra cycles of encoding and decoding and uglify etc really not worth the exercise in _ in which it would just inflate when SSL and HTTP try to compress something compressed like it's not worth doing it twice or even counter productive.
With steaming you want to break everything into pieces in the end so maybe even at the file level but using an open socket also seems like breaking down a stream a balance of efficientcy (using existing transport regiment) and avoiding redundancy.

[ry]

@MasterJames This is regarding internal communication between V8 and privileged controlling code - there's no SSL.

[whtiehack]

@ry Ask a digression. Are you sure you want to develop "Deno" in c++?

[kentonv]

@ry With Protobuf you generally construct an object tree representing a message first, then you serialize it into an ArrayBuffer all at once, then you parse the buffer on the other side into another object tree, and then you consume that object tree. With Cap'n Proto, you directly construct the message in the ArrayBuffer. As in, when you call message.setFoo(123), it actually does something like uint32Array[offset] = 123 right then and there. Similarly, on the other end, you use the message directly from the backing buffer.

More importantly, though, the encoding does not try to do any variable-width encoding like Protobuf does. For example, Protobuf encodes integers in a variable-width format so that small integers encode to one byte. This is great when sending messages over a network but is a waste when communicating within the same address space.

Regarding benchmarks -- I've spent a lot of time benchmarking serialization systems, and unfortunately, my conclusion is that benchmark results are almost always meaningless. Different serialization schemes will tend to win on different kinds of inputs data due to their differing design trade-offs. Also, caching effects and working set size tend to be a huge factor in performance which is near-impossible to capture in micro-benchmarks. To have a really meaningful benchmark, someone needs to write two versions of a real-world application using two different serializations and compare them... but that's an enormous amount of work that almost no one ever does. Cap'n Proto, for its part, tends to show up in microbenchmarks as "infinity times faster" than Protobuf because serialization benchmarks tend to measure specifically the parse/serialize step, which Cap'n Proto doesn't have -- but of course that's not really a meaningful result either.

@whtiehack Cap'n Proto does not require C++.

[MasterJames]

Right of course I somehow knew that. Sorry just tuning in I'll try to keep quite.

[matiasinsaurralde]

I've started to learn about Cap'n Proto after reading these comments, very interesting!
Couldn't continue my experiment because there were some issues with capnp-ts, seems that capnproto-js isn't an alternative as it's pretty much tied to C++/V8 code.
A simplified "translation" of our current schema could look like this:

@0xac5d44cb6d8d6291;

struct CapMsg {
    channel @0 :Channel;
    command @1 :Command;

    # START
    startCmdCwd @2 :Text;
    startCmdArgv @3 :List(Text);
    startCmdDebugFlag @4 :Bool;
    startCmdMainJs @5 :Text;
    startCmdMainMap  @6 :Text;

    # CODE_FETCH
    codeFetchModuleSpecifier @7 :Text;
    codeFetchContainingFile @8 :Text;

    # CODE_FETCH_RES
    codeFetchResModuleName @9 :Text;
    codeFetchResFilename @10 :Text;
    codeFetchResSourceCode @11 :Text;
    codeFetchResOutputCode @12 :Text;

    # READ_FILE_SYNC
    readFileSyncFilename @13 :Text;
    readFileSyncData @14 :Data;

    enum Channel {
        start @0;
        os @1;
    }

    enum Command {
        startCmd @0;
        codeFetch @1;
        codeFetchRes @2;
        readFileSync @3;
        readFileSyncRes @4;
    }
}

[kentonv]

@matiasinsaurralde I recommend using a union here:

struct CapMsg {
    channel @0 :Channel;
    
    union {
        startCmd @1 :StartCmd;
        codeFetch @2 :CodeFetch;
        codeFetchRes @3 :CodeFetchRes;
        readFileSync @4 :ReadFileSync;
        readFileSyncRes @5 :ReadFileSyncRes;
    }
    
    struct StartCmd {
        cwd @0 :Text;
        argv @1 :List(Text);
        debugFlag @2 :Bool;
        mainJs @3 :Text;
        mainMap @4 :Text;
    }

    struct CodeFetch {
        moduleSpecifier @0 :Text;
        containingFile @1 :Text;
    }
    
    # ...
}

[benjamingr]

Hey, at my previous job I switched a project from Protobuf to Cap'n' Proto and saw a big performance boost - that said, that was a very serialisation heavy project where the extra copy was proving to be the bottleneck.

I was under the impression that protobuf was used in deno to explicitly create a copy - but if that is not required I definitely think Cap'n Proto is worth considering. I think other alternatives I looked at at the time (like FlatBuffers) are also worth a look.

[kentonv]

I was under the impression that protobuf was used in deno to explicitly create a copy

If a copy is needed for security, adding in a memcpy() will still be dramatically faster than serializing/parsing a protobuf.

A copy might be needed to avoid TOCTOU bugs if JavaScript could be manipulating the buffer concurrently in another thread. If JS is blocked while the message is processed, or if it can be processed carefully to avoid TOCOU, then you can avoid the copy.

[benjamingr]

If a copy is needed for security, adding in a memcpy() will still be dramatically faster than serializing/parsing a protobuf.

Would love to see benchmarks on that @kentonv :)

[qti3e]

@kentonv In initial days of deno, we were using a different struct for each message but it was adding ~20kb for each message to final typescript declaration file (see d765300), what happens here if we try to use a union?

[kentonv]

@benjamingr You don't need a benchmark to prove that memcpy() is faster than any copying parser/serializer.

@qti3e I'm not sure exactly why Protobuf oneof would lead to code bloat, but I suspect you'll find that Cap'n Proto generated code is much smaller than Protobuf in general. Though, admittedly, I haven't tested the TypeScript implementation in particular, so this is something that should be verified.

[benjamingr]

You don't need a benchmark to prove that memcpy() is faster than any copying parser/serializer.

Yeah but you would need one to show the impact of it on a similar app