Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sync tests for all platforms #113

Merged
merged 2 commits into from Apr 12, 2019
Merged
Changes from all commits
Commits
File filter...
Filter file types
Jump to…
Jump to file
Failed to load files.

Always

Just for now

@@ -1,38 +1,44 @@

## Installer

- [ ] Check that installer is close to the size of last release.
- [ ] Check the Brave version in About and make sure it is EXACTLY as expected.
- [ ] Check that installer is close to the size of last release
- [ ] Check the Brave version in About and make sure it is EXACTLY as expected

## Visual look

- [ ] Make sure there is no words of Chromium after any merge.
- [ ] Make sure there is no words of Chromium after any merge

## Data
Pre-Requisite: Put previous build shortcut on home screen. Also have several sites 'Added to home screen' (from 3 dots menu). Then update previous build to test build.
Pre-Requisite: Put previous build shortcut on home screen. Also have several sites 'Added to home screen' (from 3 dots menu). Then update previous build to test build
- [ ] Verify that data from the previous build appears in the updated build as expected (bookmarks, etc)
- [ ] Verify that the cookies from the previous build are preserved in the updated build
- [ ] Verify shortcut is still available on home screen after updating Brave
- [ ] Verify sites added to home screen are still visible and able to be used after updating Brave.
- [ ] Verify sites added to home screen are still visible and able to be used after updating Brave
- [ ] Verify sync chain created in previous version is still retained on upgrade

## Bookmarks

- [ ] Test that creating a bookmark in the right well works.
- [ ] Test that clicking a bookmark in the right well loads the bookmark.
- [ ] Test that deleting a bookmark in the right well works.
- [ ] Test that created a bookmark folder works.
- [ ] Test that creating a bookmark in the right well works
- [ ] Test that clicking a bookmark in the right well loads the bookmark
- [ ] Test that deleting a bookmark in the right well works
- [ ] Test that created a bookmark folder works

## Custom tabs

- [ ] Make sure Brave handles links from gmail, slack.
- [ ] Make sure Brave works as custom tabs provide with Chromer browser.
- [ ] Make sure Brave handles links from gmail, slack
- [ ] Make sure Brave works as custom tabs provide with Chromer browser
- [ ] Ensure custom tabs work even with sync enabled/disabled

## Context menus

- [ ] Make sure context menu items in the URL bar work
- [ ] Make sure context menu items on content work with no selected text.
- [ ] Make sure context menu items on content work with selected text.
- [ ] Make sure context menu items on content work inside an editable control (input, textarea, or contenteditable).
- [ ] Make sure context menu items on content work with no selected text
- [ ] Make sure context menu items on content work with selected text
- [ ] Make sure context menu items on content work inside an editable control (input, textarea, or contenteditable)

## Developer Tools

- [ ] Verify you are able to inspect sublinks via dev tools

## Find in page

@@ -43,7 +49,7 @@ Pre-Requisite: Put previous build shortcut on home screen. Also have several sit

## Site hacks

- [ ] Test https://www.twitch.tv/adobe sub-page loads a video and you can play it
- [ ] Verify https://www.twitch.tv/adobe sub-page loads a video and you can play it

## Downloads

@@ -57,33 +63,45 @@ Pre-Requisite: Put previous build shortcut on home screen. Also have several sit

## Autofill tests

- [ ] Test that autofill works on http://www.roboform.com/filling-test-all-fields
- [ ] Test that autofill works on https://srirambv.github.io/formfiller.html

## Zoom

- [ ] Test zoom in / out gestures work
- [ ] Test that navigating to a different origin resets the zoom

## Sync

- [ ] Verify you are able to join sync chain by scanning the QR code
- [ ] Verify you are able to join sync chain using code words
- [ ] Verify you are able to create a sycn chain on the device and add other devices to the chain via QR code/Code words
- [ ] Verify that bookmarks from other devices on the chain show up on the mobile device after sync completes
- [ ] Verify newly created bookmarks gets sync'd to all devices on the sync chain
- [ ] Verify existing bookmarks before joining sync chain also gets sync'd to all devices on the sync chain
- [ ] Verify sync works on a upgrade profile and new bookmarks added post upgrade sync's across devices on the chain
This conversation was marked as resolved by srirambv

This comment has been minimized.

Copy link
@kjozwiak

kjozwiak Mar 27, 2019

Member

Same as the above suggestion, maybe this will be better under the data section where we handle the upgrades?

- [ ] Verify add a bookmark on custom tab gets sync'd across all devices in the chain
- [ ] Verify you are able to create a standalone sync chain with one device

## Bravery settings

- [ ] Check that HTTPS Everywhere works by loading https://https-everywhere.badssl.com/
- [ ] Check that HTTPS Everywhere works by loading http://https-everywhere.badssl.com/
- [ ] Turning HTTPS Everywhere off and shields off both disable the redirect to https://https-everywhere.badssl.com/
- [ ] Check that toggling to blocking and allow ads works as expected.
- [ ] Test that clicking through a cert error in https://badssl.com/ works.
- [ ] Check that toggling to blocking and allow ads works as expected
- [ ] Test that clicking through a cert error in https://badssl.com/ works
- [ ] Visit https://brianbondy.com/ and then turn on script blocking, nothing should load. Allow it from the script blocking UI in the URL bar and it should work.
- [ ] Test that about:preferences default Bravery settings take effect on pages with no site settings.
- [ ] Test that 3rd party storage results are blank at https://jsfiddle.net/7ke9r14a/7/ when 3rd party cookies are blocked.
- [ ] Test that about:preferences default Bravery settings take effect on pages with no site settings
- [ ] Test that 3rd party storage results are blank at https://jsfiddle.net/7ke9r14a/7/ when 3rd party cookies are blocked
### Fingerprint Tests
- [ ] Visit https://browserleaks.com/webrtc, ensure 2 blocked items are listed in shields
- [ ] Test that https://diafygi.github.io/webrtc-ips/ doesn't leak IP address when `Block all fingerprinting protection` is on

## Content tests

- [ ] Go to https://brianbondy.com/ and click on the twitter icon on the top right. Test that context menus work in the new twitter tab.
- [ ] Go to https://trac.torproject.org/projects/tor/login and make sure that the password can be saved. Make sure the saved password is auto-populated when you visit the site again.
- [ ] Open a github issue and type some misspellings, make sure they aren't autocorrected.
- [ ] Open an email on http://mail.google.com/ or inbox.google.com and click on a link. Make sure it works.
- [ ] Test that https://mixed-script.badssl.com/ shows up as grey not red (no mixed content scripts are run).
- [ ] Go to https://brianbondy.com/ and click on the twitter icon on the top right. Test that context menus work in the new twitter tab
- [ ] Go to https://trac.torproject.org/projects/tor/login and make sure that the password can be saved. Make sure the saved password is auto-populated when you visit the site again
- [ ] Open a github issue and type some misspellings, make sure they aren't autocorrected
- [ ] Open an email on http://mail.google.com/ or inbox.google.com and click on a link. Make sure it works
- [ ] Test that https://mixed-script.badssl.com/ shows up as grey not red (no mixed content scripts are run)

## Top sites view

@@ -96,12 +114,12 @@ Pre-Requisite: Put previous build shortcut on home screen. Also have several sit

## Session storage

- [ ] Test that tabs restore when closed, including active tab.
- [ ] Test that tabs restore when closed, including active tab

## Yet to be implemented

- Check that ad replacement works on http://slashdot.org
- Test that Safe Browsing works (https://www.raisegame.com/)
- Turning Safe Browsing off and shields off both disable safe browsing for https://www.raisegame.com/
- Test that turning on fingerprinting protection in about:preferences shows 3 fingerprints blocked at https://jsfiddle.net/bkf50r8v/13/. Test that turning it off in the Bravery menu shows 0 fingerprints blocked
- Test that audio fingerprint is blocked at https://audiofingerprint.openwpm.com/ when fingerprinting protection is on.
- Test that audio fingerprint is blocked at https://audiofingerprint.openwpm.com/ when fingerprinting protection is on
@@ -9,8 +9,9 @@
- [ ] Make sure that data from the last version appears in the new version OK
- [ ] Test that the previous version's cookies are preserved in the next version
- [ ] Test that saved passwords are retained upon upgrade
- [ ] Ensure stats are retained when upgrading from previous version
- [ ] Ensure per site settings are retained when upgrading from previous version
- [ ] Verify stats are retained when upgrading from previous version
- [ ] Verify per site settings are retained when upgrading from previous version
- [ ] Verify sync chain created in previous version is still retained on upgrade

## Bookmarks

@@ -40,7 +41,7 @@

## Find on page

- [ ] Ensure search box is shown when selected via the share menu
- [ ] Verify search box is shown when selected via the share menu
- [ ] Test successful find
- [ ] Test forward and backward find navigation
- [ ] Test failed find shows 0 results
@@ -78,8 +79,8 @@

## Gestures

- [ ] Test zoom in / out gestures work
- [ ] Test that navigating to a different origin resets the zoom
- [ ] Verify zoom in / out gestures work on https://www.homedepot.com/
- [ ] Verify that navigating to a different origin resets the zoom
- [ ] Swipe back and forward to navigate, verify this works as expected

## Password Managers
@@ -97,9 +98,14 @@

## Sync

- [ ] Ensure you are able to scan the QR code and sync with laptop
- [ ] Ensure the bookmarks from laptop shows up on the mobile after sync completes
- [ ] Add a bookmark on mobile and check if it gets synced to the laptop
- [ ] Verify you are able to join sync chain by scanning the QR code
- [ ] Verify you are able to join sync chain using code words
- [ ] Verify you are able to create a sycn chain on the device and add other devices to the chain via QR code/Code words
- [ ] Verify that bookmarks from other devices on the chain show up on the mobile device after sync completes
- [ ] Verify newly created bookmarks get sync'd to all devices on the sync chain
- [ ] Verify existing bookmarks before joining sync chain also gets sync'd to all devices on the sync chain
- [ ] Verify sync works on a upgrade profile and new bookmarks added post upgrade sync's across devices on the chain
- [ ] Verify you are able to create a standalone sync chain with one device

## Bravery settings

@@ -135,7 +141,7 @@
## Background
- [ ] Start loading a page, background the app, wait >5 sec, then bring to front, ensure splash screen is not shown
- [ ] Start loading a page, background the app, wait >5 sec, then bring to front, Verify splash screen is not shown
## Session storage
@@ -14,6 +14,7 @@
- [ ] opened tabs can be reloaded
- [ ] stored passwords are preserved
- [ ] unpinned tabs can be pinned
- [ ] Sync chain created in previous version is retained

### About pages

@@ -144,6 +145,20 @@ _Each start should take less than 7 seconds_
- [ ] Verify tips list is not lost after upgrade
- [ ] Verify wallet panel transactions list is not lost after upgrade

### Sync
This conversation was marked as resolved by srirambv

This comment has been minimized.

Copy link
@LaurenWags

LaurenWags Feb 18, 2019

Contributor

Saw you have upgrade tests for Android/iOS, do we want to test upgrade for laptop/desktop too?

This comment has been minimized.

Copy link
@srirambv

srirambv Feb 19, 2019

Author Contributor

Ah nice catch. I'll add the same upgrade tests to desktop.

This comment has been minimized.

Copy link
@LaurenWags

LaurenWags Feb 19, 2019

Contributor

sounds good. wasn't sure if it should be in Sync specific section like others or in the 'Update Tests' section. I guess either makes sense.


- [ ] Verify you are able to create a sync chain and add a mobile/computer to the chain
- [ ] Verify you are able to join an existing sync chain using code words
- [ ] Verify the device name is shown properly when sync chain is created
- [ ] Verify you are able to add a new mobile device to the chain via QR code/code words
- [ ] Verify newly created bookmarks get sync'd to all devices on the sync chain
- [ ] Verify existing bookmarks on current profile gets sync'd to all devices on the sync chain
- [ ] Verify folder structure is retained after sync completes
- [ ] Verify bookmarks don't duplicate when sync'd from other devices
- [ ] Verify removing bookmark from device gets sync'd to all devices on the sync chain
- [ ] Verify adding/removing a bookmark in offline mode gets sync'd to all devices on the sync chain when device comes online
- [ ] With only two device in chain, verify removing the other device resets the sync on b-c as well

### Tor Tabs

- [ ] Visit https://check.torproject.org in a Tor window, ensure its shows success message for using a Tor exit node
@@ -194,7 +209,7 @@ _Each start should take less than 7 seconds_

### Keyboard Shortcuts (WIP)

### Sync (WIP)

####Rewards Media (To be verified on YouTube and Twitch) (WIP)

#### Rewards Media (To be verified on YouTube and Twitch) (WIP)

ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.