[Feature request]: Collaboration support

[CodeDoctorDE]

Is your feature request related to a problem? Please describe

To use the app with many people isn't very easy.
To support people that want to mock up ideas with the team, we need to support editing a document with multiple users.

Describe your feature request!

Add collaboration support including some connection types:

• Local websocket server (like in games the lan mode)
• External websocket server (IP-Forwarding is required)
• (In the future an extra butterfly server)

A global signaling server to allow collaboration support without extra server would be nice but I can't host it. Feel free to share ideas.

Additional context

This issue won't be implemented soon.
This is only to track the progress of it and to share some ideas.

Code of Conduct

• I agree to follow this project's Code of Conduct

[CodeDoctorDE]

We can use some public stun server: https://gist.github.com/mondain/b0ec1cf5f60ae726202e

Google has some stun servers to use it:

• stun.l.google.com:19302
• stun1.l.google.com:19302
• stun2.l.google.com:19302
• stun3.l.google.com:19302
• stun4.l.google.com:19302

I can use it on the web and if someone click on "Create session" we can show a warning to the user that they need to accept that a connection to the google servers will be created and they should accept to the privacy policy of them.
This isn't an ideal solution for privacy aware users but can help to have this option without having an external service.

What do you think?

[CodeDoctorDE]

Added in 2.0.0-beta.16

[vi]

How do I check it on https://preview.butterfly.linwood.dev/ ?

[CodeDoctorDE]

I will post a guide soon but here a few things:

1. you can enable it there and use the connect and start button like explained in the blog post.
2. You can't start a collaboration server on web due to browser restrictions
3. The server you start is a unsecure http server (that means you need to forward this using a reverse proxy or ngrok to connect using the web and then use wss:// as protocol because the browser doesn't allow unsecure connections). The default port is 28005.

If you have more questions feel free to ask them. If you have ideas how to make this process easier, feel free to contact me. I'm open for ideas!

[vi]

in the blog post

Where is the blog post?

connect and start button

Where are connect and start buttons? I see Settings -> Experiments -> Collaboration switch, but don't see any effect of the switch on https://preview.butterfly.linwood.dev/.

I also see empty "Connections" tab which states "Web not supported" (not sure why, sites typically can connect to arbitrary wss:// sockets). This tab is displayed even when the experiment is switched off.

[CodeDoctorDE]

Where is the blog post?

I post everything on my linwood homepage: https://www.linwood.dev/butterfly/2.0.0-beta.16 which you can find a link in Read more.

This is the connect button.
The create button is accessible in the non-web version in the titlebar (the share button).
The connections tab is for synchronisation. It doesn't relate to collaboration.

[vi]

Can Butterly use a dedicated backend (not another Butterfly app, just something permanently running on server) so that all Butterfly instances use client WebSocket conenctions, not acting as a WebSocket server?

Will Web version of collaboration follow in some next version? Usage of WebSockets just naturally invites the feature to work in browsers.

Is Butterfly protocol documented? Can it work e.g. over my WebSocket broadcaster?

[CodeDoctorDE]

Can Butterly use a dedicated backend (not another Butterfly app, just something permanently running on server) so that all Butterfly instances use client WebSocket conenctions, not acting as a WebSocket server?

I will look at this. This looks like a good idea.

Will Web version of collaboration follow in some next version? Usage of WebSockets just naturally invites the feature to work in browsers.

It already can be used as client. It only doesn't allow you to host the instance. This is a browser restriction that can't be fixed.

Is Butterfly protocol documented? Can it work e.g. over my WebSocket broadcaster?

I will do this. But for now, it just sends json data around using websocket messages. It's just websockets, I didn't change anything at it. You can also use insomnia to see how these messages looks like.

[vi]

It already can be used as client.

Where is the button on Web version that activates the client?

To join this session, the other user needs to click on the connect button in the plus menu in the home page.

OK, now found it.

First try (connecting to a silent WebSocket server that does not return any messages): it spins a loading indication for some time, then outputs "TimeoutException after 0:00:10.000000: Future not completed", seemingly without sending anything to the WebSocket.

Is the difference between client and server solely in usage of WebSocket protocol (accept connection vs connect) and after the connection establishment they are symmetric? Or the distinction between server and client is deeper, e.g. only one of them acts as a source of truth and the other is just obeying commands?

[vi]

This is a browser restriction that can't be fixed.

Indeed. But I think with some simple server-side helper (I can implement it) one of clients can effectively become a (logical) server. Physically it would still connect to wss://, but incoming messages would indicate that somebody else connected to the same URL and the helper would proxy content to the pseudo-server.

[CodeDoctorDE]

TimeoutException means that it couldn't found the server. What did you use? ws://localhost should work fine if you are on the same computer and using two native butterfly apps (not web).
The server acts like a broadcaster. The collaboration feature works by sending events to everyone (for example ElementsCreated, ...). There are special events like "connections" to get all connection ids.
A server would be simple to program, half of the code could be copied from the butterfly app.
I would put that on my todo list.
A server-side helper could simplify everything but it has some problems:

1. The phelosophy of butterfly is decentralized. Such a concept could break it.
2. I don't know how much traffic it has and if I want to host this thing. What if we have 500 people trying to use this service?
   I would try to implement this like these collabora servers with nextcloud (https://www.collaboraoffice.com/code-install-and-test/).

[vi]

The phelosophy of butterfly is decentralized. Such a concept could break it.

Using of helper server can be an optional feature, allowing additional connection method besides the truly decentralized one.

What did you use?

My own service, to discover the protocol. First time it failed to connect due to implicit port 28005. It connected to the WebSocket endpoint when I added :443, but still showed the same error (without sending any messages to the WebSocket). It probably expects server to speak first.

If you want, you can experiment with this URL (obfuscated): base64_decode("d3NzOi8vdm" + "ktc2VydmVyLm9yZ"+"zo0NDMvd3Nicm9hZC9idXR0ZXJmbHkK") - it forwards each WebSocket message to all other connected clients.

ws://localhost should work fine if you are on the same computer and using two native butterfly apps (not web).

F-Droid version of the android app is not yet up to date for this. I'll postpone further experiments until it is available.

[CodeDoctorDE]

Using of helper server can be an optional feature, allowing additional connection method besides the truly decentralized one.

That would be a good idea. Like I said I don't know if my server can handle if many users will use this service.

I tested your server but like you said the server needs to send an init message.

I can show you how it looks like:

[vi]

Sending binary WebSocket message {"receiver": 700143684, "function":"init", "message": { "data": [] } } to Butterfly Web did not produce any notable results.

Note that message content appears to be trimmed on the screenshot above (and it links to a 404 page, so no full version of the picture).

[CodeDoctorDE]

The receiver is a generated connection id from the server and need to be send by the server.
I think currently it's not possible using a broadcaster but I will implement this server in the near future.
I programed a rpc system to send data from client to client (because of this you find a function and receiver field in the data).
Sorry for these limitations this feature is currently very new and in a minimum viable product state.
I would move this discussion on the matrix or discord server.