Failed Daily Backup to Google Drive Sync

[isaacsu]

Describe the bug
My Daily Backup to Google Drive has stopped working. I received an email with the subject "Failed Daily Backup to Google Drive Sync" and content:

The usual cause is an expired or revoked token from your sync provider. Please follow these instructions to use CloudLink on the web or desktop Standard Notes application to uninstall then reinstall this sync provider.

To Reproduce
I've tried uninstalling and reinstalling the Google Drive Third Party integration (did the whole flow, authenticated, copied/pasted the new token) but when I click "Perform Backup" it still fails and I get another email saying "Failed Daily Backup to Google Drive Sync".

Expected behaviour
The backup should work. I should see a recent backup of my SN in the relevant google drive folder.

Desktop (please complete the following information):

• OS: macOS 10.15.5
• Standard Notes version: 3.4.1

[rasoir-sanglant]

Mine is failing in the same way to both Dropbox and Google

OS: Windows 10
Standard Notes version: 3.4.1

[Abrantedr]

Same here with Google Drive and OneDrive services.

• OS: Windows 10
• Standard Notes version: 3.4.1

[JaspalSuri]

@isaacsu Thank you for providing a detailed report and for taking the time to reproduce the issue.

The dev team is aware of this issue and will be working on a fix starting tomorrow morning (CST / UTC-6). I apologize for the interruption and delay; the team hopes to get this fixed ASAP.

Please download manual backups of your notes in the meantime.

[moughxyz]

Sorry about that, this issue should now be fixed. Any missed backups were re-run so you should see the backups in your folders now.

[s75uiFbxnykgkQMC7NQh]

I confirm the issue still persisting with Dropbox. On a random period I receive an email that Dropbox Sync is failing due to revoked token from the sync provider. Re-installing the third party integration seems to fix the issue temporarily and backups are performed daily. After a week or so I receive the email again and so on.

[moughxyz]

Before the recent issues, how long have you had Dropbox backups enabled with no issues?

Can anyone else confirm they are or are not seeing issues?

[clyso]

same here. issue was gone but reappeared yesterday. before the first apperances of this issue (about 1 month ago) backup to dropbox was working flawlessly without any issues
I'm also doing the backup to onedrive without any failures - so it looks like this is regarding dropbox only?

[s75uiFbxnykgkQMC7NQh]

My Dropbox used to sync fine until about a month ago. This morning sync broke again and the last time was 4 days ago. Before that a week and two weeks prior to it. It seems that the issues are getting more and more frequent.

[JaspalSuri]

Thank you both for letting us know! I can confirm that I haven't had any issues with OneDrive so far, either.

[JaspalSuri]

According to @karolsojko (on Slack, yesterday):

We’ve adjusted a couple of scaling parameters so hopefully this will be fixed from tomorrow on August 6th; if not, we will investigate further.

Please let us know if it still isn't working!

[moughxyz]

We had an issue with yesterday's email + cloud backups. We've made a few updates and are reattempting a send shortly.

[clyso]

no problems any more so far. thanx for fixing :-)

[s75uiFbxnykgkQMC7NQh]

Things are stable for now, thank you, will let you know if the issue re-occurs.

[s75uiFbxnykgkQMC7NQh]

The issue is here again, no backup since September 11th and today I got the email that Dropbox Sync is failing due to revoked token from the sync provider.

[karolsojko]

I'm afraid that we had a separate issue that was introduced on Friday with: standardnotes/syncing-server@a87dcf0

but fixed on Tuesday with: standardnotes/syncing-server@d7b2f6a

As well as some fixes applied on production infrastructure today. Tomorrow's backup should run as expected. 🙂

[karan]

I followed the steps in https://standardnotes.org/help/27/how-do-i-enable-dropbox-google-drive-or-onedrive-backups and retried the backup but it failed again.

Will it automatically work tomorrow now?

[karolsojko]

We had an issue with the new infra setup for Google Drive backups. Reverted to old setup now - but we will be working on this during the upcoming days so bear with us :)

[s75uiFbxnykgkQMC7NQh]

And again today.

[JaspalSuri]

Thank you for letting us know. We will have someone from our back-end team look into it.

[s75uiFbxnykgkQMC7NQh]

There's a new twist to the issue though, I don't know if it would be helpful to mention it. In the past, when the problem occured, deleting and then re-adding the Dropbox sync permissions used to solve the matter, at least for a couple of days. Now, doing so fails immediately - right after granting new tokens and trying to perform a sync, the latter triggers a SN help e-mail that warns about sync failure.

[karolsojko]

We're working on fixing the issue

[DBCerigo]

Now, doing so fails immediately - right after granting new tokens and trying to perform a sync, the latter triggers a SN help e-mail that warns about sync failure.

Just to confirm, I'm also experiencing this behaviour - for Dropbox sync.

Good luck with the fix! :)

[moughxyz]

This issue should be fixed now. So you shouldn't see any issues on the next round of automated backups, in about 6-12 hours. You can also push an immediate backup via the CloudLink extension.

[karan]

I got this again, do I need to do anything or will this be fixed automatically?

Is there a public postmortem for why these backups fail every couple of months?

[karolsojko]

@karan the issue was found and fixed. First reported on Slack: https://standardnotes.slack.com/archives/C48MQLA79/p1603348813282700

At the moment we are doing Postmortems internally. Nevertheless we have a "Status Page" on our to-do list.

[DBCerigo]

Now, doing so fails immediately - right after granting new tokens and trying to perform a sync, the latter triggers a SN help e-mail that warns about sync failure.

Just to confirm, I'm also experiencing this behaviour - for Dropbox sync.

Good luck with the fix! :)

This is back for me. Intermittent since around 2 Feb. From just now, when granting new tokens I get help e-mail that warns about sync failure immediately.

[karolsojko]

We are aware of the issue and investigating it

[karolsojko]

We’ve applied a temporary fix, which should solve the problem for now.

At the same time we’re working on some new infra setup for Q1 2021 which will solve the issue permanently. Hopefully your backups will continue from now on smoothly :)