New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync progresses only when doing some action in UI #312

Closed
blissb2599 opened this Issue Mar 18, 2018 · 26 comments

Comments

Projects
None yet
@blissb2599

blissb2599 commented Mar 18, 2018

Operating system

  • Windows
  • macOS
  • [x ] Linux Lubuntu 17.10
  • Android
  • iOS

Application

  • [ x] Desktop Joplin-1.0.78-x86_64.AppImage
  • Mobile
  • Terminal

I first installed Joplin on Windows, set up the sync target to be my Nextcloud instance, and imported my Evernote enex file. That worked OK. It lost the notebooks I had set up, but I re-created them and moved the notes appropriately (I only have a few hundred at this point).

Then I installed Joplin on one of my Linux machines. When setting the sync target to my Nextcloud instance, it only syncs 1 notebook, and that takes a very long time. That notebook only has 5 entries all of which are text only, no graphics or associated files, and the sync takes at least 5 minutes. Nothing I do gets it to sync the other 3 notebooks.

I tried syncing the Nextcloud directory to my local machine, and pointing to that using the file system target. It syncs quickly and gets all the notebooks, but does not correctly display images.

@laurent22

This comment has been minimized.

Owner

laurent22 commented Mar 19, 2018

@blissb2599

This comment has been minimized.

blissb2599 commented Mar 19, 2018

My apologies -- I didn't enable debugging. Will do so and report back.

Just as a followup -- I left Joplin up and running overnight. It never progressed past the first entries, but I did note (which I didn't see before) that it was on item 5/247.

I'll look for logs later this afternoon, when I get back to that machine.

@laurent22

This comment has been minimized.

Owner

laurent22 commented Mar 19, 2018

There's some info on how to enable debugging and view logs there - https://github.com/laurent22/joplin/blob/master/readme/debugging.md

Please have a look at it but from the sound of it you simply have many notes and/or a slow connection or large notes or resources, so it takes time. Also keep in mind that there's a throttle mechanism in Nextcloud if you fail to login multiple times, which really slow things down.

@blissb2599

This comment has been minimized.

blissb2599 commented Mar 19, 2018

Hello again.

I've now followed the instructions for debugging -- created the flags.txt file in ~/.config/Joplin with the line: --open-dev-tools --log-level debug and restarted the application. I don't see any "Developer tools" or a "Console" tab.

As for your other thoughts --

  • This is all being done on my home LAN.
  • There are fewer than 200 notes total. Some do have images and documents associated, but none more than a few meg at most.
  • I've synced the entire folder from Nextcloud to the local machine in less than 1 minute, so network speed isn't the problem.
  • Nextcloud is not reporting any failed login attempts -- which I would think should have been a problem when verifying the connection in Joplin.
  • I'm not having any difficulty with other WebDAV enabled applications (e.g. KeePass, LibreOffice, Nextcloud Sync, etc.)

Again, I appreciate your time and effort! I'd really like to get this working across my several systems so I can finally eliminate Evernote.

Bill

@jcgerhard

This comment has been minimized.

Contributor

jcgerhard commented Mar 19, 2018

The flags.txt has to be put in .config/joplin-desktop

@blissb2599

This comment has been minimized.

blissb2599 commented Mar 19, 2018

OK -- just so you know, here's what it says in the document you linked to:
Desktop application

Add a file named "flags.txt" in the config directory (should be ~/.config/joplin or c:\Users\YOUR_NAME.config\joplin) ...

I've moved it to joplin-desktop, restarted, and am seeing the debugging window... which looks straight out of chrome, and there are log files in the .config/joplin-desktop folder.
Here's the log file. If you need the database log file, let me know so I can remove my credential info from it.

log.txt

@blissb2599

This comment has been minimized.

blissb2599 commented Mar 19, 2018

More strange behavior... when I click to a different "page" (for example, the Tools->General Options item, and then go back, the sync count increases -- and I'm seeing more items in the "joplin-desktop/resources" directory. It's as if forcing a page reload calls the sync function and pulls the next item.

[edit] Clicking the "layout" link reloads the page and increments the sync counter. Cycling through the layouts looks like it is downloading everything - one item at a time.

[edit 2] I just kept clicking the "layout" link. Eventually everything was synced to the local computer - including notebooks, notes, and tags. After syncing everything, I went to my Windows computer, added a couple of notes with images, let it sync (which works perfectly on Windows). Back to the Linux box, and had to do the "layout" clicking routine again, downloading one item at a time.

@blissb2599 blissb2599 referenced this issue Mar 20, 2018

Closed

Can't Sync with Nextcloud 13 #315

0 of 6 tasks complete
@laurent22

This comment has been minimized.

Owner

laurent22 commented Mar 23, 2018

The layout routine, is it on Linux only or does it do it on Windows too?

@blissb

This comment has been minimized.

blissb commented Mar 23, 2018

It was never necessary on Windows, as things sync as expected. Only on the Linux systems did I use that "work around."

@kristog

This comment has been minimized.

kristog commented May 15, 2018

Hello, I also have the same issue on linux (archlinux with gnome desktop)

The fetched items count increments only when I do some actions in the joplin interface, it can be switching layout or clicking on tags or notebooks : one click results in one fetched item incrementation.

No problem on windows or android.

log.txt

@wdaniel

This comment has been minimized.

wdaniel commented Jul 14, 2018

Same issue on OpenSuse Tumbleweed (kernel 4.16) with xfce desktop
no problem on Android, Windows 7, Mac OS X High Sierra

Sending F10 every second speeds syncing up.

@redam

This comment has been minimized.

redam commented Aug 14, 2018

Same issue here on Ubuntu 18.04, doing some actions (hiding/showing sidebar for example) make joplin finishes the sync while I waited several minutes for it to sync only 1 item.

@Shamp0o

This comment has been minimized.

Shamp0o commented Aug 15, 2018

Same on Mint 19. Works fine on Windows and Android.

@PhilippWoelfel

This comment has been minimized.

PhilippWoelfel commented Aug 22, 2018

Same on OpenSuSE 15.0. With both, Dropbox and Nextcloud sync

@caseyhoward

This comment has been minimized.

caseyhoward commented Aug 24, 2018

Broken for me in Mint 19 and Fedora 28 also. Mounting with davfs2 and using the local directory option seems to work though. The Nextcloud setting is also broken.

@dev4223

This comment has been minimized.

dev4223 commented Aug 27, 2018

Same issue on OpenSuse Leap 42.3

@dev4223

This comment has been minimized.

dev4223 commented Aug 27, 2018

This triggering effect has to be done only for the notes. After the notes are synced the other items (tags, ...) sync by joplin itself.

@laurent22

This comment has been minimized.

Owner

laurent22 commented Sep 21, 2018

The fetched items count increments only when I do some actions in the joplin interface, it can be switching layout or clicking on tags or notebooks : one click results in one fetched item incrementation.

Could this be related to some power saving feature of the OS? What if you disable most of them (just for testing)?

@laurent22 laurent22 changed the title from Incomplete and slow sync with WebDAV to Sync progresses only when doing some action in UI Sep 21, 2018

@laurent22 laurent22 added bug and removed enhancement labels Sep 21, 2018

@dev4223

This comment has been minimized.

dev4223 commented Sep 22, 2018

No, I don't think so, because I was constantly working on my laptop during the sync.

@wdaniel

This comment has been minimized.

wdaniel commented Sep 23, 2018

Attached tshark log, pulled on the server side, for about 3 minutes, in the middle of a sync
Joplin-synclog-104.txt

192.168.0.96 - server
192.168.0.104 - client

@JPvRiel

This comment has been minimized.

JPvRiel commented Sep 30, 2018

Could this be related to some power saving feature of the OS

Just add, fully disabled power saving (Ubuntu LTS 16.04), locked the desktop, left it overnight, and the next morning, it seemed the sync count resumed (no progress). This confirms @dev4223's assumption

No, I don't think so, because I was constantly working on my laptop during the sync.

There's also ani impact on E2EE and setting up other clients. One of the first things is should priority sync is the master key. So far everything else was synced before the master key, so got stalled setting up E2EE on other devices waiting for this to happen. The mobile client event reports "decrypting" while synching without having the master key installed (will need to log a separate bug for that). I see as per Encryption Sync Issues, that doing just a single note setup with E2EE before evernote import a good workaround.

@JPvRiel

This comment has been minimized.

JPvRiel commented Sep 30, 2018

Workaround for slow initial bulk sync after evernote import? is a hack I tried and it worked out well in my case.

TL;DR: Rather use local file system sync pointed at the Dropbox/App/Joplin folder. And for Android, the "Offline files" option in the Dropbox app get stored in an sdcard scratch folder that helps do something similar, but after getting the bulk sync done, switch back to Dropbox API sync method because Dropbox on mobile isn't watching that scratch folder for updates to files.

Instead of 2 days of fighting with sync, I was all done after a few minutes.

I also noticed that local file system sync doesn't seem to stall without the UI being in focus, or at least it's so fast, it doesn't matter that much, so maybe the bug here is quite specific to the Joplin Dropbox sync "driver".

Syncing via the Dropbox native file sync app on the OS goes much faster.

@laurent22 laurent22 added the essential label Oct 12, 2018

@laurent22

This comment has been minimized.

Owner

laurent22 commented Oct 12, 2018

Idea for a fix: on Linux run forceUpdate() at regular interval while syncing. Also remove energy-saving feature which doesn't help fixing this bug and cause other issues.

@laurent22 laurent22 closed this in e4166e9 Oct 12, 2018

@Shamp0o

This comment has been minimized.

Shamp0o commented Oct 25, 2018

I installed Manjaro (xfce) on two machines last week and haven't faced any UI freezes or sync problems. Mint (xfce) had caused those problems before as stated above. Used appimage in both distros as the AUR package failed to build.
Maybe this helps.

@kstevens715

This comment has been minimized.

kstevens715 commented Nov 5, 2018

I'm running Joplin 1.0.114, which I believe includes e4166e9 but I'm still experiencing this issue on Ubuntu 18.04 and syncing to NextCloud 13. I have a couple hundred notes to sync from my laptop to NextCloud but I have to click around in the user interface for progress to be made. Each time I click on something a file will upload. But if I don't click for 5 minutes, it stalls out the entire time until I click on something again. And it's not just the UI not updating as I check in NextCloud and sure enough a file only gets uploaded when I click on something in Joplin.

I've been messing with this all day, but just found this GitHub issue. I'm about to call it a night, but when I get a chance I'm going to do some more digging.

@neolovich

This comment has been minimized.

neolovich commented Nov 11, 2018

I'm running Joplin 1.0.114 too on Linux Mint 19. Nextcloud 14.0.3.0, PHP 7.0.32. Syncing takes about a view minutes for one or two notes. The android app on LineageOS 15.1 works well so far. Takes also its time (20 to 30 seconds i guess) but it works.
I tried the workaround JPvRiel suggested (Save the notes local on the computer and synchronize with nextcloud client) but then synchronization of the pictures doesnt work (looks like a "broken link" then).
Is there any possibility to make the synchronization on a Linux machine a bit faster? Or as a alternative: Get the synchronization via nextcloud client working including the pictures?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment