Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collaborative editing using WebRTC #1930

Open
ehg opened this issue Jul 17, 2017 · 26 comments

Comments

@ehg
Copy link
Contributor

@ehg ehg commented Jul 17, 2017

Collaborative editing in WordPress has been on our minds for a while. What we've been lacking is a way to support this on self-hosted sites, without the need for a 'centralised' service, whilst retaining compatibility with hosts who may not have the ability to dedicate long-running processes that lends itself to most collaborative editing techniques.

I proposed an experiment, to @abhishekgahlot — what if we were able to use WebRTC, with the combination of WP-API endpoints, to accomplish this goal?

The original plan was to go full-on Google Docs style collaboration, with p2p Operational Transforms, or CRDTs. We then took a step back, and thought, due to the nature of blocks — how about just using traditional locking, similar to the current WordPress editor, but per block. We decided this should be enough granularity for now, and will get us to a proof of concept a lot faster.

If this proof-of-concept is feasible, we can of course implement "full" collaborative editing at a later date. It also opens up the possibility of audio/video/text chat, if we ever see a need.

There are some things we need to watch out for when considering the implementation:

  • Latency – is WebRTC going to be fast enough?
  • Scalability – how many editors can we reasonably support?
  • Reliability – what happens on dodgy WiFi, or if a user disconnects etc.
  • Compatibility – WebRTC isn't supported on IE11, and some older browsers. Also, how do we get around firewalls?

This issue is to track the various approaches we're trying out, and will be updated as we progress in our experimentation. For now, a pull request is available at #1877, including more specific design details. Thanks @abhishekgahlot :)

Here's a preview of the proposed initial design, kindly provided by @jasmussen:

block level locking

@westonruter

This comment has been minimized.

Copy link
Member

@westonruter westonruter commented Jul 17, 2017

Post locking in WordPress currently has a 150 second window, so WebRTC will surely have a double-magnitude reduction in latency compared to that. Similarly, it will have a magnitude reduction in latency compared to WP Heartbeat, which itself is often disabled by hosts. That's the beauty of WebRTC: there aren't hosting requirements… mostly.

I assume this would mean WordPress would need to include a STUN and TURN functionality?

@abhishekgahlot

This comment has been minimized.

Copy link
Member

@abhishekgahlot abhishekgahlot commented Jul 18, 2017

@westonruter Yes, We will be using STUN/TURN ( Hole punching and for relaying ) also all the network traffic will be P2P and encrypted. Signaling will be either XHR or WebSocket based.

@westonruter

This comment has been minimized.

Copy link
Member

@westonruter westonruter commented Jul 18, 2017

@abhishekgahlot but how much of this will be able to run on the server with WordPress itself? Will there be a need for an external service? For example, most hosts wouldn't support WebSockets. It would be great (required?) if it could be self-contained.

@abhishekgahlot

This comment has been minimized.

Copy link
Member

@abhishekgahlot abhishekgahlot commented Jul 19, 2017

@westonruter As of now, everything works standalone without the need of WebSockets for 2 peers. Because signaling is done by XHR and number of requests to verify how many peers are there is less. What I meant was we can utilize WebSockets for faster initial signaling process.

@ehg

This comment has been minimized.

Copy link
Contributor Author

@ehg ehg commented Jul 19, 2017

For example, most hosts wouldn't support WebSockets. It would be great (required?) if it could be self-contained.

Yes, this would be one of the constraints. We need to go as far as possible for it to be standalone, and not rely on any centralised service, hence the initial idea of WebRTC + OT/CRDT in the first place.

@jasmussen

This comment has been minimized.

Copy link
Contributor

@jasmussen jasmussen commented Jul 21, 2017

Love this ticket.

I would attach just a quick note, that it's important to only focus on the colored block outlines in this post, and imagine them in the current design — that is, the 2px border with a color and a name attached, rather than a full width dotted line. The mockup is very very old and discarded for many reasons. Though I do miss the Spinal Tap boilerplate text ;)

@jasmussen

This comment has been minimized.

Copy link
Contributor

@jasmussen jasmussen commented Jul 24, 2017

Here's a new mockup. CC: @abhishekgahlot and #1877:

collab editing

@westonruter

This comment has been minimized.

Copy link
Member

@westonruter westonruter commented Jul 24, 2017

@jasmussen should the block be read-only for user B when user A is editing it? When it has the presence indicator, would it be locked?

@jasmussen

This comment has been minimized.

Copy link
Contributor

@jasmussen jasmussen commented Jul 25, 2017

@jasmussen should the block be read-only for user B when user A is editing it? When it has the presence indicator, would it be locked?

Yes, in the mockup above, I'm editing the first paragraph, and Abhishek is editing the 3rd paragraph. The 3rd paragraph is locked to me, and the first paragraph is locked to Abhishek.

@westonruter

This comment has been minimized.

Copy link
Member

@westonruter westonruter commented Jul 25, 2017

@jasmussen and how would the locked state manifest itself? None of the controls would appear when you select it? Maybe it should have a message overlay like “This block is currently being edited by X” to make the locked state explicit? Otherwise a user could be confused as to why they can't edit the block, since Google Docs allows concurrently editing the same “blocks” (paragraphs).

@jasmussen

This comment has been minimized.

Copy link
Contributor

@jasmussen jasmussen commented Jul 26, 2017

and how would the locked state manifest itself? None of the controls would appear when you select it?

That's been the idea so far, yes.

Maybe it should have a message overlay like “This block is currently being edited by X” to make the locked state explicit? Otherwise a user could be confused as to why they can't edit the block, since Google Docs allows concurrently editing the same “blocks” (paragraphs).

This is a really good point. I like the more verbose "Being edited by [username]" label. We might even show a graying overlay on a locked block, perhaps even change the cursor from the caret to a stop sign when you mouse over a locked block. I think some of this can be worked out in implementation, but these are good ideas.

@jasmussen

This comment has been minimized.

Copy link
Contributor

@jasmussen jasmussen commented Aug 1, 2017

When it becomes time to color the various borders of blocks locked to others, we shouldn't use arbitrary colors. We should use from a preset array. Here are some colors I'd suggest using:

red: #dc3232
orange: #f56e28
yellow: #ffb900
green: #46b450
purple: #826eb4

These colors are from @hugobaeta's extended WordPress color palette: https://codepen.io/hugobaeta/full/RNOzoV/

If more than 6 people are editing the same post, we can just repeat the colors — there's also a name attached to each block anyway. Or we can double the amount of colors by also using a lighter version of the same shade.

We can use these colors by adding them to classnames, and then simply apply a CSS class to a block that's locked. CC: @abhishekgahlot

@gziolo gziolo self-assigned this Sep 26, 2017
@rheinardkorf

This comment has been minimized.

Copy link

@rheinardkorf rheinardkorf commented Oct 1, 2017

Is there a prototype for this issue yet?

I'm currently working on a scenario where multiple repeatable editors are required. In most cases this is not a big concern is there is only one editor. But, multiple users may need to collaborate. Would be nice to start playing around to get a feel for how this could work.

EDIT: Never mind, just saw the working PR for this.

@gziolo

This comment has been minimized.

Copy link
Member

@gziolo gziolo commented Oct 2, 2017

Yes, there is one as you already discovered: #1877. I’m linking to make sure everyone else is also able to easily locate this in progress PR.

@gziolo

This comment has been minimized.

Copy link
Member

@gziolo gziolo commented Oct 2, 2017

@jasmussen can we include in the mockup a list of all users editing the post/page. @mtias suggested to show connected authors in the top left.

@jasmussen

This comment has been minimized.

Copy link
Contributor

@jasmussen jasmussen commented Oct 2, 2017

I'm a bit weary of putting stuff inside the main editor canvas. If we hope to load the theme stylesheet there in the future, we can't know for sure which kinds of crazy backgrounds floating items will sit on (see also my comment on the recent Word Count feature).

In this case, seeing the active editors, it feels like we should emulate the Docs pattern. It also feels like this is a feature that's okay to completely hide on the mobile breakpoint.

In the following mockup, the toolbar is slightly rearranged and tweaked (see #2460), freeing up space to show little avatars next to the save indicator:

collab editing

CC: @karmatosed

@gziolo

This comment has been minimized.

Copy link
Member

@gziolo gziolo commented Oct 2, 2017

In this case, seeing the active editors, it feels like we should emulate the Docs pattern. It also feels like this is a feature that's okay to completely hide on the mobile breakpoint.

This is how I pictured it myself, too 👍

@jeffpaul

This comment has been minimized.

Copy link
Member

@jeffpaul jeffpaul commented Feb 22, 2018

This ticket was mentioned in Slack in #core-editor by jeffpaul. View the logs.

@Lewiscowles1986

This comment has been minimized.

Copy link

@Lewiscowles1986 Lewiscowles1986 commented Apr 18, 2018

Hello, I've been meaning to check this out since @karmatosed mentioned at WordCamp London.

My concerns are

  • WordPress meta and posts seem to be "document-oriented" in their relationship
    • how to deal with race conditions if person {A} updates block {X} when person {B} updates block {Y} on the same post.
    • Are blocks already stored separately? In my wp install it seems to all be in the post_content db field.
    • is it okay that neither request can reasonably know about eachother, so resolution of races might be problematic
  • Technical complexity of deploying a STUN / TURN server that is turn-key and safe

It looks AMAZING, and I really want this for WordPress, because it's amazing. But like the initial TinyMCE wp.views, I feel like there won't be significant enough a portion of people that use or know about this to be able to contribute, or extend into their own plugins.

@gziolo

This comment has been minimized.

Copy link
Member

@gziolo gziolo commented Apr 19, 2018

Technical complexity of deploying a STUN / TURN server that is turn-key and safe

In the proof of concept we have, there is no server involved. It works for 2 peers and one of them plays the role of the server to make sure that it can take precedence when resolving conflicts. They communicate between each other using WebRTC.

Are blocks already stored separately? In my wp install it seems to all be in the post_content db field.

Inside Gutenberg all blocks are stored independently in the memory, so we were able to lock only those blocks that are currently being edited. It needs some more work to make sure it works with multi select or drag&drop but those are issues that can be resolved with good design decisions.

how to deal with race conditions if person {A} updates block {X} when person {B} updates block {Y} on the same post.

We replay the same (Redux) actions on each client to make sure that their internal in-memory state is in sync.

@Lewiscowles1986

This comment has been minimized.

Copy link

@Lewiscowles1986 Lewiscowles1986 commented Apr 19, 2018

Thanks, so it's peer-to-peer in the proof of concept.

@gziolo

This comment has been minimized.

Copy link
Member

@gziolo gziolo commented Apr 19, 2018

Thanks, so it's peer-to-peer in the proof of concept.

Yes, this is what it is. It only makes sense in WP core when it doesn't require additional setup and external resources. We can make it extensible so plugin developers could make it work for more peers using the STUN / TURN server.

@Lewiscowles1986

This comment has been minimized.

Copy link

@Lewiscowles1986 Lewiscowles1986 commented Apr 19, 2018

The idea I had upon reading would be that like multisite and other features WP_DEBUG you'd probably use defines and if they were not present, you'd enable shared hosting to just work without (since it likely would have too much latency for shared hosting to be effective anyway).

The concern over STUN/TURN if it were a thing would be hackers getting into the servers, either leeching for their own services, or maliciously using them for attacks like unprotected memcache server daemon.

Anyway. Thanks for PoC, very interesting idea and choices. 👍

@karmatosed karmatosed removed the Design label Apr 24, 2018
@mtias mtias added Future and removed [Status] In Progress labels Jul 9, 2018
@mtias mtias added Future and removed Future labels Oct 7, 2018
@youknowriad youknowriad modified the milestones: WordPress 5.x, Future Jan 9, 2019
@youknowriad youknowriad modified the milestones: Future, Future: Phase 2 Jan 21, 2019
@gziolo gziolo removed their assignment Feb 8, 2019
@stebansaa

This comment has been minimized.

Copy link

@stebansaa stebansaa commented Mar 13, 2019

What is the status of this feature?

@gziolo

This comment has been minimized.

Copy link
Member

@gziolo gziolo commented Mar 13, 2019

It isn't actively worked on nor explored. It won't be a priority until Phase 2 of Gutenberg is close to beeing finished.

@gziolo

This comment has been minimized.

Copy link
Member

@gziolo gziolo commented Aug 13, 2019

I’m sharing a link to an article from The New York Times for a future reference:
We Built Collaborative Editing for Our Newsroom’s CMS. Here’s how.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.