API for structured serialized data

[devsnek]

Problem

JSON is being left in the dust as we get more and more stuff for JS, and it probably won't be getting any updates.

Goals

• support all the js types (except for things like weakmaps, functions, etc, keep state safe)
• supports cold storage
• standardized format so we can use it between engines
• forward compatibility

Intuitions

• JSON.parse/JSON.stringify type api
• StructuredSerializeForStorage
• optionally async/streaming(?)
• binary format

Prior art

• https://github.com/addaleax/cold-storage
• https://nodejs.org/api/v8.html#v8_serialization_api (just a wrapper around v8's serdes)

Other considerations

• non-js servers wanting to generate this format for client consumption (maintain a reference C impl? steal an existing system like ETF?)
• new mime type, new method for fetch Response

[bmeck]

Symbol.for should be cross realm safe, idk if we can show exceptions on the differences for these versus other symbols.

[macdja38]

Would be nice to have circular references for something like this as well.

[guest271314]

One possible option would be creating dynamic modules which could be exported and imported w3c/FileAPI#97 (comment).

[devsnek]

@guest271314 this format shouldn't perform any evaluation (all stored data should be completely static)

[guest271314]

The data can be stored as text and only evaluated when necessary.

[devsnek]

this format cannot rely on any form of evaluation; it presents inherent security and performance concerns.

[guest271314]

Is a goal to encrypt the data? Plain text could be converted to and from an ArrayBuffer. Are import and export insufficient? What are the use cases?

[devsnek]

the goal here is a static format for storing serialized javascript values analogous to JSON but with support for many more types. this api would most likely be similar to https://nodejs.org/api/v8.html#v8_serialization_api but without the class or transferables support.

[guest271314]

Map and WeakMap do not meet requirement?

[jakearchibald]

It feels like the intent here is to expose https://html.spec.whatwg.org/multipage/structured-data.html#serializable-objects as some kind of format.

[devsnek]

@jakearchibald indeed that is what inspired me to open this

[Jamesernator]

There might be security considerations for platform objects (step 18) that are [Serializable] as they might expose data that isn't usually exposed to user-land.

[pierogitus]

I think this is worth another look. As I just mentioned in the fetch repo, CBOR with tags is well suited to this problem. Its also part of WebAuthn so there must be some CBOR logic already happening in all the major browsers.