-
-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WebRTC Data Sync Feedback #1683
Comments
使用Docker部署的应用,然后NGINX反代;在同一台电脑上开多个窗口,多个窗口之间的确可以同步,但是2个不同的设备(电脑和手机)都显示已连接,但是不会同步。已确认频道名称和密码都是一样的。 |
Use Docker to deploy applications, and then use NGINX to reverse generation; open multiple windows on the same computer, and the multiple windows can indeed be synchronized, but two different devices (computer and mobile phone) are displayed as connected, but will not be synchronized. . Confirmed that the channel name and password are the same. |
不太理解,为什么同步需要两个设备同时在线,如果一个是公司电脑,一个是家里电脑,这个场景下就无法使用功能同步呀 |
I don’t quite understand why synchronization requires two devices to be online at the same time. If one is a company computer and the other is a home computer, functional synchronization cannot be used in this scenario. |
这个是和 WebRTC 本身的通信特性有关系,由于目前的技术选型(纯前端、无服务端)的情况下,两个设备的数据同步只能通过点对点通信的形式达成。当一个设备在线,一个设备离线的情况下,我们无从感知数据到底应该从哪来,只有当两台设备都在线的时候,双发数据才能通信。 其实这种模式更像是一个在线聊天室,大家都在线时才能看到对方的数据,然后达成同步。
是的,你说的没错,那这也我实现完这个功能之后发现的问题。WebRTC 这种纯点对点的方式在某些情况下并无法完全满足用户的诉求,同时也存在一些数据同步层面的问题。这也是我认为这个功能目前也仅算是「实验」的原因。 但是这个模式最大的意义,是让我确信,基于 YJS 我们可以很轻松地实现跨端的数据同步,至于 WebRTC,只是达成同步的一个模式而已 。而我们未来更进一步的,则是在 YJS 这个同步引擎基础上,引入更多的服务端存储层,例如 基于 Redis 的 Upstash 、MongoDB 、甚至于 S3 、文件系统等等。而到这个时候,就是我们的云同步体验达到完美的阶段。 |
This is related to the communication characteristics of WebRTC itself. Due to the current technology selection (pure front-end, no server), data synchronization between two devices can only be achieved through point-to-point communication. When one device is online and the other is offline, we have no way of knowing where the data should come from. Only when both devices are online can dual-send data communicate. In fact, this model is more like an online chat room. Only when everyone is online can they see each other's data and then achieve synchronization.
Yes, you are right, then this is also a problem I discovered after implementing this function. The purely point-to-point approach of WebRTC cannot fully meet the needs of users in some cases, and there are also some data synchronization issues. This is why I think this feature is only an "experiment" at the moment. But the greatest significance of this model is that it convinces me that based on YJS we can easily achieve cross-end data synchronization. As for WebRTC, it is just a model for achieving synchronization. What we will go one step further in the future is to introduce more server-side storage layers based on the YJS synchronization engine, such as Redis-based Upstash, MongoDB, and even S3, file systems, etc. By this time, our cloud synchronization experience has reached the perfect stage. |
@coulsontl 有没有试过换一个网络呀 |
@coulsontl Have you tried changing the network? |
试过,不行 |
Tried, doesn't work |
和我的情况不太一样,我是分别用vercel和本地docker部署的: |
The situation is different from mine. I deployed it using vercel and local docker respectively: |
问题:现在是指同步聊天记录,不会同步设置里的配置参数吗? |
Question: Now it means synchronizing chat records, but will it not synchronize the configuration parameters in the settings? |
https://github.com/vrtmrz/obsidian-livesync |
https://github.com/vrtmrz/obsidian-livesync |
@GentlemanHu 看了下不会采用,它需要用户手动处理冲突,体验很不好的。我们后续的方案可以做到完全无冲突 |
@GentlemanHu After looking at it, I won’t adopt it. It requires users to manually handle conflicts, and the experience is very bad. Our subsequent plans can be completely conflict-free |
同步一直没成功过,换了好几个版本也没成功过 (mac-win firefox),内网,外网都不行。 同时,看起来还会向 另外,我认为 webrtc 是个糟糕的选择,理由如下:
为什么不把数据放到本地呢?借助 docker volumes 或许你根本不需要多写什么逻辑,多端根本不需要同步,而且十分稳定,我觉得你花时间重构这部分代码比你折腾 rtc 更有效率。当然,还是尊重作者的个人倾向,很赞的工具! 以上 |
The synchronization has never been successful, even after changing several versions (mac-win firefox), neither on the internal network nor on the external network. Also, it looks like a connection is being sent to Also, I think webrtc is a bad choice for the following reasons:
Why not put the data locally? With docker volumes, you may not need to write any more logic at all. The multiple terminals do not need to be synchronized at all, and they are very stable. I think it is more efficient for you to spend time refactoring this part of the code than to toss around rtc. Of course, I still respect the author’s personal preferences. It’s a great tool! above |
找到问题了,之前为了防止DNS泄露,开了Chrome的 WebRTC Network Limiter 插件,把这个插件关了就好了;希望作者能出个离线同步的功能。 |
I found a pattern. As long as the device using lobehub and the machine where lobehub is deployed are in the same LAN, they will never be synchronized. If they are not in the same LAN, it will be fine. |
+1 to leverage Docker volumes + Sqlite, so basically store history, API Key credentials in DB instead of in Browser. |
后面应该会有版本自己部署信令服务器的,其实就是一个频道中转,和聊天一样。数据存本地也必须基于 yjs 的同步协作框架,必须后端部署 wss 和持久化的数据库。不过我估计有后端的应该是 saas服务了 |
There should be a version in the future that deploys its own signaling server. It is actually a channel relay, just like chat. The local data storage must also be based on the yjs synchronization collaboration framework, and wss and persistent database must be deployed on the backend. But I guess the backend should be a saas service. |
上述服务的代码在:https://github.com/arvinxx/y-webrtc-signaling/blob/main/index.js ,直接拷贝的官方的代码:https://github.com/yjs/y-webrtc/blob/master/bin/server.js 。 如果你担心数据安全问题,后续可以提供自行部署信令服务器的选项。但根据你的描述,我感觉后续数据库的版本更适合你。
其实严格意义上来说,WebRTC 是不适合做数据同步,它像是一个「实时同步」的技术方案。比如同时打开两个浏览器并开启同步的模式情况下,在A浏览器发送的消息可以实时在B浏览器中看到。这个是 WebRTC 所实现的不一样的特性,比如未来我们是否可以基于这种模式实现一个「多人在线的 AI 聊天室」? 不过,我也承认现在这种方案的确存在你说的问题,所以这只是一个「实验性」的技术路径,完美的同步方案不会采用 WebRTC。
换成标准数据库同步方案一样有这种问题吧,数据是否同步到了服务器,丢失了怎么办,后续处理怎么办。但本地优先的方案,至少不会出现刷新页面后本地数据没了的情况。
如果你说的本地是指服务端数据库,我们后续是会做的:#1768 (comment) 。 但是你说实现这个方案会更有效率,我是不敢苟同的,数据存储一份在本地是 LobeChat 的实现理念,纯服务端的方案也只是一个增量的模式,而不是完全推翻重来。 Local First 的体验会优于 Cloud First (可以查看 https://electric-sql.com 演示的 demo)。 另外一点,我也知道 traditional 数据库方案传统稳定,但是技术/体验创新,往往是非共识下诞生的。作为开源产品而不是商业化产品,我认为是需要在 LobeChat 上尝试新的技术方案,进而诞生新的东西。 |
The code of the above service is at: https://github.com/arvinxx/y-webrtc-signaling/blob/main/index.js. The official code for direct copy: https://github.com/yjs/y-webrtc /blob/master/bin/server.js. If you are worried about data security issues, you will be provided with the option of deploying your own signaling server later. But based on your description, I feel that subsequent database versions are more suitable for you.
In fact, strictly speaking, WebRTC is not suitable for data synchronization. It is like a "real-time synchronization" technical solution. For example, if you check in two browsers at the same time and turn on the synchronization mode, the messages sent in browser A can be seen in browser B in real time. This is a different feature implemented by WebRTC. For example, can we implement a "multiplayer online AI chat room" based on this model in the future? However, I also admit that this current solution does have the problems you mentioned, so this is just an "experimental" technical path. A perfect synchronization solution will not use WebRTC.
If you switch to a standard database synchronization solution, you will still have the same problem. Is the data synchronized to the server? What should I do if it is lost? What should I do with the subsequent processing? But with the local-first solution, at least the local data will not be lost after refreshing the page.
If the local you are talking about refers to the server database, we will do the following: #1768 (comment). But you say that implementing this solution will be more efficient. I disagree. Storing data locally is LobeChat's implementation philosophy. The pure server-side solution is only an incremental model, rather than a complete reinvention. The Local First experience will be better than Cloud First (you can view the demo at https://electric-sql.com). Another point, let me tell you, I also know that traditional database solutions are traditionally stable, but technology/experience innovations are often born out of non-consensus. As an open source product rather than a commercial product, I think it is necessary to try new technical solutions on LobeChat to create something new. |
这极大程度的解决了我的困惑,很期待你们的创新进展!开源项目就需要这种破旧立新的勇气,支持~,不过作为用户,还是希望尽早看到本地数据库的版本,因为之前一次 bug,数据丢失过一次。当然,还是按照你们自己的开发进度来就好。 |
This has solved my confusion to a great extent, and I look forward to your innovative progress! Open source projects require this kind of courage to break the old and create new ones, support~, but as a user, I still hope to see the version of the local database as soon as possible, because a bug caused data loss once before. **Of course, just follow your own development progress. ** |
Hi, first of all thank you for your work on data sync, it's a key feature for me since I use many devices! Just one question - from my experiments it seems that agents are currently not being synced across devices. Are there any plans to implement agent sync? |
LobeChat can also support websocket. An additional yjs websocket server, user self-host it. yjs websocket server is always running, and can persist yjs document data by using sqlite. Also, I'm a little surprised that LobeChat uses yjs. I would like to recommend a useful library for yjs, hocuspocusjs, which can easily achieve synchronization and persistence. This is not an advertisement, just a casual mention, because I am also working on a project which use yjs and this library. |
基于这个特性,对于个人而言,在数据备份角度,确实是需要有一台(如家庭服务器)的设备长期在线,可以从这个角度考虑增加这样的设备角色,方便用户配置这一台备份设备。 |
Based on this feature, for individuals, from the perspective of data backup, it is indeed necessary to have a device (such as a home server) online for a long time. From this perspective, you can consider adding such a device role to facilitate users to configure this backup device. |
@BaituLime 不知道啥原因这个崩了,我刚重启了,你再试试 ![]() 后续我把这个信令服务端的部署放开来吧,允许大家自行添加。这样估计能解决一部分问题 |
Unfortunately I have trouble getting lobe-chat WebRTC to work in Firefox 125.0.3
|
I can't tell you how much I'm looking forward to this moment, as well as your db implementation =) |
这是来自QQ邮箱的假期自动回复邮件。您好,我最近正在休假中,无法亲自回复您的邮件。我将在假期结束后,尽快给您回复。
|
This is a holiday automatic reply email from QQ mailbox. Hello, I am currently on vacation and cannot reply to your email personally. I will reply to you as soon as possible after the vacation. |
We use this issue to track all WebRTC relative data sync issue
我们使用本 issue 来跟踪所有数据同步相关的问题,任何和数据同步相关的问题,请在此问题下记录
The text was updated successfully, but these errors were encountered: