Synchronize bookmarks and notes to Google Drive

[tuomas2]

(Note: plans here are outdated, implementation more or less ready in #2584)

I'm planning to implement this by writing journal of events in files in sync folder (in Google drive) that can be replayed on synced devices. Periodic cleanup of the folder can be performed and full database dump will be stored.

I.e. sync folder would contain something like this:

- database-20190101010101.db #Full database
- journal-20190101020101.json #Changes since last database dump
- journal-20190201030101.json #Changes since last journal entry
-...
- database-20190201010101.db #Full database will be written at some point as a new starting point, then old journals and database dumps can be removed.

[tuomas2]

From user:

Missing an account sync ,so you can highlight the verse but it is only for you device, after reinstall, you are loosing data (some manual saving is possible), but not good for user.

[MasterOfTheTiger]

Maybe MEGA or Nextcloud as well?

[steelwil]

Do not forget about reading plan progress.

[mattzab]

An app called Fitnotes syncs to a hidden Google Drive folder in the background approximately one hour after the user has input data. This allows access on multiple devices.

I would specifically recommend against using MEGA simply because of its recent association with the Chinese Government.

[tuomas2]

Not sure but maybe this can be service transparent (like installing document from zip).

[timbze]

Maybe this idea is overkill, but perhaps this feature could be mostly implemented in a separate Kotlin native library, that way it would be easier to also sync with other sword apps on desktop and IOS.

There would need to be the option to work with different types of data separately, e.g. one app might want to only sync bookmarks.

[tuomas2]

Developer:

Perhaps easier starting point would be to add uuid's and timestamps to all db rows in all tables and make it first possible to "merge" backup database (instead of replacing one). Then, if same uuid is in both db's, then use more recent one.

[agrogers]

Hi Tuomas

I think this is becoming even more important now that we can use BSA (Bible Study App) for so much.

I think it is worth considering a really simple solution. Just backup the DB to google drive automatically (perhaps when the app loses focus) and check that file when starting BSA gets focus. If the DBs are out of date prompt the user to restore it with some info on when it was last backed up and from what device.

This is how I do it now when switching between devices but it is tedious and I can forget sometimes.

Or if it can't be done manually, add an option to the backup form to allow 'Synching' which basically uploads or downloads based some version or datetime stamp.

[tuomas2]

simple solution better than no solution at all. this might work in many cases well enough.

[agrogers]

My feeling is:

1. Implement a manual and easy option ASAP.
2. Try to automate it a little.
3. Implement a last change always wins fancier option on a record by record basis.

[tuomas2]

apparently it is possible to have limited access to google drive like this

:

[mattzab]

That's how FitNotes works, as I mentioned above .

There's also an open source project that I use, called rclone, that is designed for cloud storage sync to many (almost 50) different cloud providers. It can easily be configured to access this hidden part of Google drive. It may help get this feature going in the right direction.
rclone.org

They have arm binaries, here's a link to an unofficial Android rclone project, https://github.com/x0b/rcx

[agrogers]

Yes, something is better than nothing for sure. It is one feature that is noticeably absent. If on opening AndBible it could check if there is a latter DB backup and prompt to restore it automatically that would be enough in most cases.

…

On Sat, 6 Aug 2022, 11:32 pm mattzab, ***@***.***> wrote: That's how FitNotes works, as I mentioned above <#154 (comment)> . There's also a project that I use, called rclone, that is designed for cloud storage sync to many (almost 50) different cloud providers. It can easily be configured to access this hidden part of Google drive. It may help open the door for better solutions. rclone.org — Reply to this email directly, view it on GitHub <#154 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKGTJWUBH5EINWW4LKM3U3VX2LCHANCNFSM4GFOROSQ> . You are receiving this because you commented.Message ID: ***@***.***>

[tuomas2]

Unfortunately all cloud services have their own APIs and no generic approach to support (all) cloud storage services seem to exist. So, I'm planning to implement only Google Drive support.

[tuomas2]

Hi Tuomas

I think this is becoming even more important now that we can use BSA (Bible Study App) for so much.

I think it is worth considering a really simple solution. Just backup the DB to google drive automatically (perhaps when the app loses focus) and check that file when starting BSA gets focus. If the DBs are out of date prompt the user to restore it with some info on when it was last backed up and from what device.

I have been playing around with this a little in #2584. I made some optimization to the db filesize and gzipped it and my db currently is about 500 kB, gzipped. And I know that some people have much larger databases. This approach that we upload / download whole db when focusing/unfocusing feels quite wasteful for bandwidth. Need to think about different options...

[tuomas2]

Do you think there might be some interest for self-hosted solution? It would be possible to implement support for minio (s3 compatible) as an "free alternative". https://github.com/minio/minio
I'm just thinking about all the de-googlers out there...

[timbze]

Do you think there might be some interest for self-hosted solution?

How about just a directory? I use Syncthing, or even Nextcloud etc could be used like that. Haven't looked at minio yet

[tuomas2]

Do you think there might be some interest for self-hosted solution?

How about just a directory? I use Syncthing, or even Nextcloud etc could be used like that. Haven't looked at minio yet

Does not sound a reliable method. Implementation requires that files are uploaded in order, immediately. If AB would write one patch first, then after one minute, another patch, and then Syncthing would do its job in wrong order, that would make things messy.