[FEATURE] Multi-device synchronization of Wispar database

[ShaoweiCheng]

Is your feature request related to a problem? Please describe.
I would like a more convenient way to sync the Wispar database (excluding downloaded papers) across multiple devices (such as Android eReaders, iPads, and cellphones). The main goal is to keep the record of hidden publications consistent across devices.

Currently, my workaround (e.g., between two devices A & B) is as follows:

1. On device A, export the database and save it to the local drive. (On iPad, this is not possible due to permission restrictions.)
2. On device A, use another app (e.g., OneSync) to sync the local database file with a cloud drive.
3. On device B, sync the database file from the cloud drive and save it to the local drive.
4. Import the synced database file into Wispar on device B.

Since all of this must be done manually each time I switch devices, it is inconvenient. Furthermore, if I forget to perform the sync, the hidden publication record may become inconsistent across devices.

Describe the solution you'd like
I would like Wispar to provide an option that allows users to store the latest database file directly in a cloud drive (e.g., Google Drive, Dropbox, OneDrive), and use the cloud drive’s API to read and write the database file. This would make synchronization much more convenient.

When the “save database in cloud” option is enabled, the following features are suggested:

1. Startup sync: On startup, Wispar should attempt to read a fixed database file from a user-defined cloud drive path to update the local database. If this fails, it should fall back to the local database.
2. Manual upload: During runtime, provide a manual option (e.g., within the Wispar icon in the top-right corner) to upload the database file to the cloud drive.
3. Scheduled backup: Allow users to set a custom interval (e.g., every 30 minutes) for automatically backing up the database file to the cloud.
4. Exit sync: On program exit, Wispar should automatically upload the latest database file to the cloud before closing.
5. Conflict handling: Implement measures to prevent multi-device conflicts (e.g., check the modification time of the database in the cloud drive and the local copy; if the cloud version is newer, overwrite the local database, and if the local version is newer, upload it to the cloud).

Describe alternatives you've considered
I haven’t considered other alternatives, but if there is a simpler way to automatically sync hidden publication records across devices, I would be glad to use it.

Additional context
none.

[Scriptbash]

Hey, sorry for the long delay! Thanks for suggesting this, that's something I meant to do for a long time.

I'm still not 100% sure how I will proceed, but I think I would simply allow users to pick a directory where to save the database. So instead of storing the database inside a directory only accessible by Wispar, it could be save in the device Documents for example. A third party sync client like Syncthing, Onesync, etc. could be used to synchronize automatically the database across devices. The only thing is that I'm not sure how they handle conflicts. It seems like that's the way some apps like Mihon do it too.

I would prefer not adding specific APIs for cloud storage, since I think it would be very difficult to have everyone's provider and it would probably add quite a bit of maintenance.

As for excluding downloaded papers, it may be a bit difficult, but possible. With the way it's currently implemented, I would need to check if the PDF file exists before creating a downloads card. The only problem I see with this is that if you have downloads on device A and different downloads on B, both database would contain the PDF file names of A and B. If you uninstall the app on device A, there wouldn't be a way to remove the file names of PDF files that no longer exist. Eventually, there could be a lot of unnecessary entries in the database. For now I would include the PDF files, but I'll continue to think about it. There are now also graphical abstracts, they are store in a way similar to PDF files. I would probably have the same problem with excluding them.

I'll add automatic backups too, they would be done on app startup. The other possibility is using the background service, but that wouldn't be reliable on iOS and there's a high chance that the backup wouldn't finish before the service is killed by either Android or iOS.

On iPad, this is not possible due to permission restrictions.

I changed the way the database export works in the previous update. I think you have to create a new folder and select that folder for the export. Previously, you would pick a file, but now it's a folder. On my iPad it's currently working. Can you confirm that creating a new folder still doesn't work? If so, can you check the logs in the app settings? Just be careful as there might be some information that shouldn't be shared. You can also tap and hold a log card to copy the content of only that card. If you prefer, you can send it by email at wispar-app@protonmail.com

Nevermind I also have a permission issue, I'll look into it!

[ShaoweiCheng]

Thank you very much for your response and your attention to my questions. I think it's a great idea to let dedicated syncing apps handle the sync job, as you suggested.

If it were possible to specify that the database be stored as a file in a user-designated directory, it would greatly simplify keeping Wispar synchronized across multiple devices. Ideally, the database file in that directory would reflect changes in real time—for example, immediately updating after a card is hidden. Alternatively, the database changes should at least be saved to the file periodically, based on a user-defined time interval.

Regarding the separation of downloaded PDFs from other database content, I now realize this may not be essential for me. By using Zotero, I can manage all my PDF downloads there, and don't need to handle PDF files directly within Wispar. Therefore, my Wispar database now contains no PDFs.

Thank you again for your support, and I look forward to your continued improvements.

[Scriptbash]

So, I've started working on it and it was very easy for Android. For iOS it was much more complicated, as I had to use native code to be able to get the permissions to read and write permanently in a user picked folder.

In the database settings, I've added a toggle "Custom database location" that opens a folder picker. When a folder is picked, the database files are all moved to the selected directory. When the toggle gets turned off, the files are moved back to the app directory. I briefly tried to sync the database between my Android phone and my iPad and it worked! Syncthing is limited on iOS however, since apps can't run constantly in the background. That means the Syncthing app must be open for the sync to happen, but that's still much better than exporting/importing the database constantly.

I still have a few things to do (e.g. check if a database is already in the picked folder and ask the user if they want to overwrite the files or not). I will probably release a beta on TestFlight this time around to make sure everything works correctly.

[Scriptbash]

I've been using a custom location for my database for several days on my Android and iPad and there doesn't seem to have an issue!

I should be able to get an update ready in the upcoming days, so before storing the database outside of the app directory, please make a backup and save it in a different location than the folder you will use for storing the database files.

The folder Wispar will use for the database file must be created using Wispar. If you make a folder with syncthing for instance, you will get a permission error.

When setting up the sync between your devices, it's very likely that the files will get erased/overwritten. Once the sync is established, you can import the backup and you should be good to go! Syncing between Android and iOS (at least with syncthing) is quite finicky though. Maybe it works better between two Android devices.

[ShaoweiCheng]

Great to hear the news! Thank you for your effort and dedication. I’m looking forward to using the new version of the app.