Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: exploring CBOR, protobuf #51

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

chore: exploring CBOR, protobuf #51

wants to merge 5 commits into from

Conversation

dckc
Copy link
Collaborator

@dckc dckc commented May 19, 2023

refs: #3

This includes CBOR work noted earlier plus some playing around with protobuf and passables.

I don't know whether it should ever get merged.

cc @zarutian (I can't seem to request review from you).

@zarutian
Copy link

@dckc ya rang?

taking a look.

@kriskowal
Copy link
Collaborator

cc @gibson042 for CBOR experience.

Comment on lines +27 to +28
encoder.pushAny("S");
encoder.pushAny(value.toString());
Copy link
Collaborator Author

@dckc dckc May 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops... this should be 1 array of 2 items: the "S" tag and the string

Comment on lines +33 to +38
} else if (value instanceof Error) {
encoder._pushTag(SELF_DESCRIBED);
encoder.pushAny("E");
encoder._pushInt(2, MT.ARRAY);
encoder.pushAny(value.name);
encoder.pushAny(value.message);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

likewise, this SELF_DESCRIBED should be followed by 1 array of 3 items: "E" tag, name, message.

@kriskowal
Copy link
Collaborator

I’ve got no CBOR experience, but I’m glad we’ve got evidence that CBOR can house O’Cap’n’s data model. I see possible directions going forward:

  1. We can make the data representation layer pluggable. That would be handy for the web where JSON is king and nothing in JS will ever approach the performance of native JSON.parse.
  2. It may be that @erights’s and I believe @gibson042’s Encode Passable depends upon the bytewise comparability of an acylic passable subtree in a way that would vary between Syrup, JSON, CBOR, Protobuf encoding layers (please confirm or deny!) In which case, it would make the more sense to commit to the concrete Encode Passable representation, which in turn closely resembles the spirit of Syrup (and Netstring at the message framing layer!)
  3. Maybe the bytewise comparability invariants can be preserved in CBOR?

@gibson042
Copy link
Collaborator

Maybe the bytewise comparability invariants can be preserved in CBOR?

I believe so, because CBOR is strictly more general than JSON. It supports false/null/true and arbitrary IEEE 754 binary64 values (the finite subset of which corresponds with JavaScript-interoperable JSON numbers), and obviously also lists and dictionaries. And we could probably take advantage of some affordances that are lacking in JSON, such as arbitrarily-sized integers, undefined/±Infinity/NaN, and non-string dictionary keys. But there's a subtle gotcha as well—a CBOR text string (major type 3) is UTF-8 and therefore cannot represent all JSON strings (specifically those including unpaired surrogates such as "\uDEAD") without a higher-order encoding, although a CBOR byte string (major type 2) can do so if it is subject to interpretation as either CESU-8 or WTF-16 (with the latter probably being more convenient for sorting by JavaScript string semantics).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants