Status of operational transform JSON types

[eMarek]

I would like to ask what is current status (or progress) of OP JSON types. I am asking because README.md says it defaults to ot-json0, but json1 spec says that JSON1 OT type will replace the JSON0 OT type. Does ShareDB already supports JSON1 type or not? I see that JSON0 is not even listed at ottypes docs. I am asking because I am working on a new project and I would like to pick the one which will be maintained in the future yet stable enough for production.

[curran]

json0 is stable and has an ecosystem around it.

json1 is arguably superior but is relatively new and has no ecosystem around it yet.

ShareDB should definitely work witn json1, but I haven seen any examples of it yet. That would be a great example to add.

[eMarek]

I tried it to use json1 but I was unsuccessful. That's the reason why I opened this ticket. Yes, any example would be welcome. Actually I think I will stick with json0 since I need this package https://www.npmjs.com/package/json0-ot-diff for my usecase and I haven't found json1 alternative.

[alecgibson]

I don't think there have been many people trying to use json1 with ShareDB, but we did merge a recent pull request to improve its behaviour (#374), so we're certainly open to making it all work nicely together. That being said, I can't see json1 becoming the "default" type any time particularly soon, just because json0 is so embedded in the sharedb core.

Why were you unsuccessful when trying to use it?

[josephg]

Practically speaking JSON0 works fine. The biggest drawback is it can't handle arbitrary moves. If that isn't a problem for your use case then I wouldn't worry about it much at this point. (And even if it is, you can just delete from the old location and re-insert at the new location. json0 will lose concurrent edits, but thats usually pretty rare.)

It should be pretty easy to convert json0 ops into json1 ops in a dozen lines of code or so - json1 is explicitly designed to be compatible. But those lines don't exist, and there's still a few differences that will need review. (Like by default json1 will throw errors in some cases instead of deleting user data). All the rough edges like this haven't been sanded off yet.

If you want to hit the ground running with sharedb I suggest sticking with json0 for now. Its well tested and good.

[eMarek]

@alecgibson The way I tried to use it is this:

    const doc = shareConnection.get('myCollection', 'myDocument)
    doc.fetch((error: Error) => {
      if (error) {
        window.console.error(error)
        return
      }
      if (doc.type === null) {
        doc.create([], 'ot-json1')
      }
    })

It throws ERR_DOC_TYPE_NOT_RECOGNIZED. I was following the instructions how to create document in README.md. This part doc.create(data[, type][, options][, function(err) {...}]). As far as I understand second param is type, right?

@josephg Understood. Okay cool, to be honest I prefer stability so I would pick json0. The thing is that I came across an error. I am not 100% sure where does it happen. Give me couple of hours and I will prepare an example.

Thank you all for quick responses. I appreciate it!

[alecgibson]

https://github.com/share/sharedb#err_doc_type_not_recognized

[eMarek]

I was a bit confused where should I register type, on my client or server? So I tried both, but it didn't work. I installed ot-json1 with yarn add ot-json1.

My server looks like this:

var express = require('express')
var http = require('http')
var WS = require('ws')
var WebSocketJSONStream = require('@teamwork/websocket-json-stream')
var ShareDB = require('sharedb');
var otJson1 = require('ot-json1')

const hostname = '0.0.0.0'
const httpPort = 7000

ShareDB.types.register(otJson1.type);
const share = new ShareDB()

const httpServer = http.createServer(express())

var wss = new WS.Server({server: httpServer});
wss.on('connection', function(ws:any) {
  var stream = new WebSocketJSONStream(ws);
  share.listen(stream);
});

httpServer.listen(httpPort, hostname, () => {
  console.log(`Server running at http://${hostname}:${httpPort}/`)
})

And my client looks like this:

import ReconnectingWebSocket from 'reconnecting-websocket'
import ShareDB from 'sharedb/lib/client'
// import otJson1 from 'ot-json1' // ES6 import returns undefined!
const otJson1 = require('ot-json1')

const Socket = new ReconnectingWebSocket('ws://localhost:7000')
ShareDB.types.register(otJson1.type);

const Share = new ShareDB.Connection(Socket)

const doc = Share.get('myCollection', 'myDocument')
doc.create([{foo: "bar"}], 'ot-json1')

Error which I receive is:

ShareDBError {
  code: "ERR_DOC_TYPE_NOT_RECOGNIZED",
  message: "Missing type undefined"
}

[alecgibson]

Missing type undefined

The name for JSON1 is json1 (not ot-json1)

[eMarek]

Ohh, got it. It works now. Even in mongo I see correct type.

Thanks for help!

[curran]

Nice! A PR that adds a small json1 example to this repo under /examples would be warmly welcome.

[eMarek]

@curran Here: #382

@josephg Ignore what I mentioned before regarding the possible error. It was my mistake with some race conditions.

As far as I am concerned we can close this ticket. I got json1 working. I just need to decide which one I will use. But this is up to me I guess. Thanks for quick help once again.

[josephg]

Both rust and foundationdb have forums set up for community discussion and general conversations like this. I’m wondering if it would be useful to set something like that up for us. And maybe invite ot / crdt / braid protocol / automerge folks...

Something like this https://forums.foundationdb.org/

[eMarek]

Hey, could someone merge this PR #382 so I can close this ticket.

[alecgibson]

Yeah we should have a look at tomorrow's PR meeting.