Sync still very slow on Linux #1023
Comments
|
I have been experiencing similar issues for some time now (since around v1.0.10 I would guess, but don't quote me on that). Interestingly, I have observed that when syncing to a webdav server on my local network, I have no problems. When syncing my Android phones, I have no problems. When syncing to the same webdav server from elsewhere on the internet, it can take many minutes to sync one or two (or even zero) text notes. I never had enough details to report, as there is nothing interesting in the server logs, but I can at least provide that much. In dav.log, you can see a sync which creates one 1 line (+title) text note taking 2 minutes 3 seconds due to a very long idle period. This is from Joplin 1.0.117, against a wsgidav server behind an nginx reverse proxy (not using nginx dav). According to my memory (which may be faulty), a similar sync from a system on the same LAN as that server completes in 1-2 seconds. Using the same joplin profile, workstation and server with the command line Joplin client, the sync is also completed in 1-2 seconds, so this appears to be a desktop-exclusive issue. |
|
Same problem on my end. VERY slow sync. During sync, the "Updated remote items: 1." stays on my screen for 5~20 minutes. This is manifesting even when only changing a single word in a single text note. (All encrypted, FWIW). Am on a gigabit connection. All other applications and web services responsive. Dropbox client responsive. Mobile app (phone and tablet) synced with same account are responsive and take under 3 seconds to sync the same changes. Ubuntu 18.10; Joplin 1.0.117 (prod, linux) I hope this report helps! EDIT: just saw v.118 Beta released, popped-up right after posting this reply. Will install and report back if this is fixed on my end. :) |
|
I think this is a duplicate of another issue where the Linux GUI doesn't respond during sync. There's currently no fix for this. Maybe someone with knowledge of Linux desktop could figure it out. |
|
As I said in the beginning, it is not a gui-update problem - sync is slow regardless how often you update the gui. And some primitive stracing just shows my assumption: joplin is just doing nothing, it is not busy with anything. It seems more of a problem of internal deadlock (waiting for some resources), and once in a while the sync-task will have access to the resource, sync an item, and then wait again. I have good knowledge in linux programming (also in JavaScript), so I would be willing to invest some time to track down the problem. But I need some basic understanding of the code (the big picture, i.e. architecture, structure and flow of control) to get started. Of course I could also re-engineer the code, but that would be a waste of time assuming that you already know about that. |
|
There's no deadlock problem on other platforms so I don't think it's an issue with the algorithm itself. My assumption is that there's something in some Linux distros that makes the app sleep or give it very low priority, maybe a power management feature, and that slows down or stops the sync process. Or maybe something that slows down the web requests? In terms of architecture, the synchronisation happens in the same thread as everything else. The algorithm basically is comparing what's on the WebDAV service, and what's on the SQLite database and create/delete/update as needed. The whole thing is JS code running on V8 engine. I can provide more info if needed.
When it seems to be waiting like this, does it unlock if you do something in the UI? For example, open the config screen, change something and save? |
Only a very very misconfigured system would do that. Even embedded systems woud "race to idle", i.e. get work done and then go to sleep.
I don't believe that. Joplin (i.e. the underlying chrome) uses many threads with a lot of interprocess communication. The UI is always responsive, so the actual sync cannot happen in the UI-thread.
When I look at my Apache-logs, this seems indeed the case. When I don't do anything in the UI, Joplin sends an item to the webdav every 120 seconds. When I switch between notebooks, it triggers a send at once. So Joplin does not need an UI update to synchronize an item, but it surely speeds it up. Since the Joplin-code is the same on other platforms (?), this might be a problem of the chrome-implementation of Linux. |
|
I confirm this behavior. Looking forward to a fix. |
|
For documentation purposes: First test: Disable scheduled GUI events, which triggered very often. While debugging the application I’ve noticed, that every 50ms a GUI event is triggered, which looked, if the size of the application window has changed. The causing interval was quiet easy to find and could also be disabled. After starting the synchronization again, I’ve tried to find what the cpu is doing while sending the actual request to the server. But there was nothing. The application was waiting for something, but it was not an http response from the webdav server. And the server did not receive any bytes either. Second test: Extracting the webdav sync code and try to send a backup with it to the webdav server. I’ve copied the important functions (like creating the options/headers for the node-fetch call) for creating new files on the webdav server from your code into a script, which I’ve placed inside of the project. With this I also wanted to make sure, that I won’t get any different dependencies by accident for the testscript. Result: The synchronization was fast and had no problems. (I’ve tested this multiple times.) Third test: Change the GUI to call the testscript instead of the actual synchronization code. This should show, if the problem is within the webdav synchronization code (which I thought was unlikely because it works on all other platforms and should be independent of the platform), or if there is a global (configuration) problem. Result: Calling the testscript from within the GUI, resulted in the same extremely slow synchronization. So it looked like a global (configuration) problem. Fourth test: Looking into the network calls to see, when the application does what and when it is waiting. This should show, if the application really waits for the webdav server or another network resource, or if the CPU just does not trigger the call, doing something different. Here I found something: On a Linux host I’ve found that the application tries to establish the connection to the webdav server. The server responds correctly with a “SYN, ACK” to which the application responds with an “ACK” package. Shortly after that I’ve found that the server did not receive the last “ACK” and resends the “SYN, ACK” package, after around 20 seconds. The application then sends another “ACK” package, which only rarely fixes this connection. More often the webdav server will reset and close the connection. Somehow node-fetch and http or something deeper down does not get this. Then the application waits for the configured timeout (from shim.fetchWithRetry). On a windows host I could not find this “lost” packages. After searching for similar issues I found some threads on different sites but no suitable, clean solution. (Other than updating to a newer nodejs version, but if this would really help is quiet unsure.) Because of this I thought to either decrease the connection timeout, not knowing which other parts of the application would change or if those would actually need the 2 minute timeout. But even then the sync would still be slower than on the other platforms. After implementing the keep alive, I’ve tested it with multiple full-syncs of my notes and also with syncing updates. It is not really satisfying to fall back to this workaround though. |
|
Fifth test: Using the command line version Result: The synchronization was fast. @laurent22 Is there a way to use both the GUI and the command line client with the same working directory? Would they interfere / break each other if done so? |
Yes it's likely something will break. For example if the CLI app syncs and the desktop is running at the same time, the desktop app won't know it needs to reload the data from the database, so it might overwrite items that were synced. There's probably a ton of other edge cases I can't think about because it's not meant to be used that way. |
|
Don't know, if it's helpful, but I have the same problem with sync on three different Kubuntu 18.10/19.04 machines. While sync'ing with my NextCloud with my Android device needs only a few seconds (feels like less than 2 seconds in local WLAN, and 6 seconds from the outside world over the internet), it needs minutes with the Linux client. It's no networking problem. One Linux client is connected via 1 Gbps ethernet to the NextCloud / Apache-webserver, the other Linux client is a Notebook with 1.x Gbps WLAN, the third is a Linux guest VM running on the same host where NextCloud / Apache-webserver runs. While the minutes of sync'ing I can use the Joplin GUI without reduced speed, fully responsive. When sync'ing from Android device, the Apache logfile says: Finished in 6 seconds. Same thing to the same NextCloud-URL from the Linux client: Finished in 4m 21s... same problem on each Linux machine. |
|
Next thing...
|
|
Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions. |
|
This is still a problem, dear stale bot. |
|
Some ideas there for a solution: #1931 |
|
I am experiencing this too. It appears to me as if the Linux Desktop client hangs while syncing. It uses no network while syncing for long periods of time until you click somewhere in the GUI. |
|
Lately, this bug has been cropping up across all three of my platforms: Linux, iOS, and Android. The Webdav provider I'm using is pCloud. |
|
I have the same issue on Linux Mint 19.3. Syncing is very slow, and on top of this, it completely ignores the setting for syncing frequency. Even though it is set to 24h, it appears to me that it is syncing every 5 min. |
Operating system
Application
This is a sort of reopen of #312 which IMHO has a wrong title and is misleading. Sync still is very slow even after the relase that "maybe" fixed #312.
My investigations don't show any influence of GUI-activities on the sync process itself. The observations are purely anecdotal.
What I found out so far is:
The logic during sync needs some improvement, since it seems to work like this:
I still have to verify the "do nothing" part (will do some strace when I find time), but whatever Joplin does during this time dos not use any measurable OS-resources.
So one thing that really has to be done is to move the open/close out of the loop. This is not responsible for the slow sync, but it is a sensible optimization anyway.
I also tried to understand where all this is going on in the code but no success yet. Maybe someone can give me some pointers to the relevant source-files then I would further investigate the issue.
Bernhard
The text was updated successfully, but these errors were encountered: