Skip to content

Local restore and backup.#725

Merged
blackforestboi merged 83 commits intodevelopfrom
feature/local-storage-restore-tmp
Feb 28, 2019
Merged

Local restore and backup.#725
blackforestboi merged 83 commits intodevelopfrom
feature/local-storage-restore-tmp

Conversation

@digi0ps
Copy link
Copy Markdown
Contributor

@digi0ps digi0ps commented Feb 21, 2019

Forked off #709.

@oliversauter Ticked off most of the things in the list you made in the previous PR.

Remaining:

if user makes their first backup, no matter if they restored before or not, it should give them the choice to setup the path again.

Don't exactlly get this. Do you mean change the choice of backend or change the local backup? Both are possible right now, if you click the "change" button.

I don't think I deleted any restore code as such. But a rebase error might have happened. @ShishKabab can you check when you are free?

ShishKabab and others added 30 commits February 21, 2019 18:22
# Conflicts:
#	src/options/backup/screens/overview.tsx
Added a temporary icon for it.
Created a new reusable button component called SmallButton.
# Conflicts:
#	src/util/ui-logic.ts
@blackforestboi
Copy link
Copy Markdown
Member

From a UX and UI perspective this is ready to go into production.

However there are still some oddities: I did a full restore with my 2gb data library from G Drive too and all worked well without errors. 👍 However there is

  1. some data missing. It does not show me pages from the last 5 days, and a lot of screenshots are missing too.
  2. more data in my restored version than in my original one headscratch

I feel like before we start implementing the syncing problem (or as part of it), we should make sure that we figure out those indexedDB oddities somehow, and if those are not the cause then make really sure that all data is properly backed up and restored.

Will now do a backup to my local hard drive to compare and can more tomorrow.

@blackforestboi
Copy link
Copy Markdown
Member

blackforestboi commented Feb 26, 2019

Ok some more data:

Good news:

  • It seems as if all pages and other documents are properly backed up and restored

Bad news:

  • It seems like that is not the case for screenshots. A lot of them get lost in the process.
  • There are still some discrepancies between GDrive and local backup. I can't figure out why I have this hole of 600 pages missing, but just in the last week. There was no data loss when using the local app to backup & restore. Will do another Backup on Google Drive and see what the result is.

Setup:

  1. Main Extension 24100 pages
  2. Google Drive Account with (allegedly 3gb) of data
  3. New extension used for restore & Backup: 23599 pages
  • Backup Size calculation: 1800mb images, 400mb rest
  • Actual Backup on disk: 700mb images, 400mb rest

@ShishKabab
Copy link
Copy Markdown
Member

ShishKabab commented Feb 27, 2019

before we start implementing the syncing problem (or as part of it), we should make sure that we figure out those indexedDB oddities somehow.

Yup, that's one thing we've learned from this whole adventure. The new sync tech sits in front of Dexie, captures all modifications, and turns them into transactions that contain the original write and the write to the sync log. That we we're always 100% sure each and every change is logged.

(Right now we're using Dexie hooks, which are more unpredictable it seems. Plus, they're not portable to different databases.)

ShishKabab and others added 6 commits February 27, 2019 13:05
Fix logical errors in prev commit's implementation

Comment out chunking loop for now

Revert "Comment out chunking loop for now"

This reverts commit 3af0e67.

Set up objectChange PKs as inc PKs rather than timestamps

Fix order of pagination operations in dexie query

- apparently that's important....
@poltak poltak force-pushed the feature/local-storage-restore-tmp branch from 47e2b24 to 3cb6ac4 Compare February 28, 2019 03:24
@blackforestboi blackforestboi merged commit 217a2ef into develop Feb 28, 2019
@blackforestboi
Copy link
Copy Markdown
Member

Thanks @digi0ps for the awesome work to finish this one up :) ready to launch baby!

@ShishKabab ShishKabab deleted the feature/local-storage-restore-tmp branch April 9, 2019 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants