Test Settings Sync features

[sandy081]

Testing settings sync features

• windows @alexr00
• mac @eamodio
• linux @weinand

Complexity: 5

---

Set Up

• Follow instructions here
• To test, have two insiders open with different user data directories and extension directories
• Make sure to back up your settings & keybindings since some test scenarios involve deleting data from server.

Reset

Reset Completely

To reset completely Turn off sync and select Turn off sync on all devices checkbox. This will remove all your data on the server and also remove the sync state on your local machine.

Reset Machine

You can reset a machine and remove the sync state using following two commands

• Turn off (do not check the Turn off sync on all devices)
• Developer: Reset local (Sync) - This will remove the sync state on your local machine.

Exploratory Testing

Getting Started Experience

• Test setting up Settings Sync on a machine for the first time ever. (Reset completely first to test)
• Test setting up Settings Sync on another machine for the first time. You can always reset your local machine using Reset machine step.
• Test turning off sync completely also turn off sync on other machines

Synchornising Settings

• Test settings file (including settings positions and comments) is synchronised across machines as expected.
• Test showing and resolving conflicts flow and UX when there are conflicts while synchronising settings and keybindings.
• For settings, conflicts should be detected not only with settings values but also with their positions and comments.

Synchronising Extension enablement

• Test extension's enablement is synchronised across devices.

Synchronising Display Language (UI State)

• Test synchronising display language across machines

Back ups

• Test local settings and keybindings file are backed up when ever updated with remote content. Local data is backed up in .sync folder under the same directory as settings and keybindings files.

Error handling & logging

• Test errors are logged (in sync log channel) and reported. Errors are reported and sync is disabled in following cases
  • When the file to sync is too large (1MB)
  • If sync fails consecutively 5 times
• Auto sync interval increases exponentially every time there is an error. Regular interval is 5 mins. It increases by number of sync failures.
• Test Sync Output channel logs are having useful information when set to different log levels (eg: trace).

[alexr00]

Works very nicely and is a great feature!

[eamodio]

@sandy081 I'm still digging into extension syncing, but for extension state sync (enable/disable) should it be syncing workspace/folder enable/disable? Because it doesn't seem like it is.

[sandy081]

Only global enablement is synced.

Workspace/Folder enablement state is not yet considered because currently there is no way of mapping workspaces/folders yet.

[eamodio]

@sandy081 I haven't been able to thoroughly test the extension sync, but I think when sync is running it is mostly fine. But at first sync of a 2nd machine, I believe there is an issue where the extensions sync, but are all enabled until you restart vscode. But I haven't been able to reproduce that, because I keep running into sync errors, not authorized connection refused, etc. A couple of times, I've had sync turn off (forced me to sign-in again) when I woke my pc up from sleep. And on the mac, I tried turning sync back on and it keeps erroring with not authorized.

[eamodio]

I also did encounter one very strange case.

On my PC I turn off sync and had it delete all of the cloud data and turn off sync for all my machines.
I then went over to my mac, and there were sync errors in the logs (not authorized), and sync thought it was still on. So I restarted vscode to see if it would change, no change and still sync errors (not authorized). So I turned sync off, and then uninstalled all of the disabled extensions (I believe there were 4 of them). Then I went back to my PC, and turned sync back on and the turn it on on the mac. But instead of it syncing the disabled extensions from my PC to the mac, it uninstalled the extensions from my PC!

I assume this was some timing issue because of the network errors, but it was not good. Especially since I had chose the option to turn off sync everywhere and deleted the cloud data... but somehow it still either left data or sync'd after that.

[sandy081]

@eamodio Thanks for the feedback. Can you please file issues for above two so that they can be tracked?

[sandy081]

@eamodio Regarding extension enablement syncing issue is this what you are seeing #89462?

[eamodio]

@sandy081 No that isn't the same

[sandy081]

Any reason why you added verification found label?

[eamodio]

I thought that was the procedure when there are bigger issues. Sorry.

[sandy081]

No worries. Generally we added this tag to bugs or feature requests those needs to be verified.