Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove redundant test run and update template #139

Merged
merged 1 commit into from Aug 29, 2019
Merged
Changes from all commits
Commits
File filter...
Filter file types
Jump to…
Jump to file
Failed to load files.

Always

Just for now

Remove redundant test run and update template

  • Loading branch information
srirambv committed Aug 29, 2019
commit 3c7cde88e21887e2f67ac5cd991e00782e902e98
@@ -28,10 +28,9 @@
win32_checklist = []
win64_checklist = []
linux_checklist = []
iOS12_iPad5G_checklist = []
iPadMini_iOS12_checklist = []
iPhone6_iOS12_checklist = []
iPhone8_iOS12_checklist = []
iPad_checklist = []
iPhone_checklist = []
iPhoneX_checklist = []
android_x86_checklist = []
android_arm_checklist = []
android_tab_checklist = []
@@ -460,24 +459,20 @@ def ios_testruns():
str(issue.number) + "](" + issue.html_url + "))"
checklist.append(output_line)
checklist.append(issue.html_url)
if("QA Pass - iPad Pro" not in label_names and
"QA Pass - iPad 5th Gen" not in label_names and
if("QA Pass - iPad" not in label_names and
"iPhone" not in label_names and
"tests" not in label_names):
iOS12_iPad5G_checklist.append(output_line)
iPad_checklist.append(output_line)

if("QA Pass - iPad Mini" not in label_names and
if("QA Pass - iPhone" not in label_names and
"iPad" not in label_names and
"tests" not in label_names):
iPadMini_iOS12_checklist.append(output_line)
iPhone_checklist.append(output_line)

if("QA Pass - iPhone 6" not in label_names and
"QA Pass - iPhone SE" not in label_names and
"tests" not in label_names):
iPhone6_iOS12_checklist.append(output_line)

if("QA Pass - iPhone 8" not in label_names and
"QA Pass - iPhone 7" not in label_names and
if("QA Pass - iPhone X" not in label_names and
"iPad" not in label_names and
"tests" not in label_names):
iPhone8_iOS12_checklist.append(output_line)
iPhoneX_checklist.append(output_line)

print("Release Note:")
for line in release_notes:
@@ -489,78 +484,60 @@ def ios_testruns():
print(line)
print("")

print("iPad Pro/5G iOS12 Checklist:")
print("iPad Checklist:")
bigline = "## Per release speciality tests\n"
for line in iOS12_iPad5G_checklist:
for line in iPad_checklist:
bigline += line + "\n"
bigline = bigline + ios_template
print(bigline)
print("")
iPad5GiOS12Title = "Manual test run for " + ios_key +\
" on iPad Pro/5th Gen running iOS12"
iPad5GiOS12List = ["ipad", "release-notes/exclude", "tests", "QA/Yes"]
iPad_Title = "Manual test run for " + ios_key +\
" on iPad running iOS12"
iPad_List = ["ipad", "release-notes/exclude", "tests", "QA/Yes"]

if args.test is None:
ios_repo.create_issue(title=iPad5GiOS12Title,
ios_repo.create_issue(title=iPad_Title,
body=bigline,
assignee="LaurenWags",
milestone=ios_milestone[ios_key],
labels=iPad5GiOS12List)

print("iPad Mini iOS12 Checklist:")
bigline = "## Per release speciality tests\n"
for line in iPadMini_iOS12_checklist:
bigline += line + "\n"
bigline = bigline + ios_template
print(bigline)
print("")
iPadMiniiOS12Title = "Manual test run for " + ios_key +\
" on iPad Mini running iOS12"
iPadMiniiOS12List = ["ipad", "release-notes/exclude", "tests", "QA/Yes"]

if args.test is None:
ios_repo.create_issue(title=iPadMiniiOS12Title,
body=bigline,
assignee="kjozwiak",
milestone=ios_milestone[ios_key],
labels=iPadMiniiOS12List)
labels=iPad_List)

print("iPhone 6 iOS 12 Checklist:")
print("iPhone Checklist:")
bigline = "## Per release speciality tests\n"
for line in iPhone6_iOS12_checklist:
for line in iPhone_checklist:
bigline += line + "\n"
bigline = bigline + ios_template
print(bigline)
print("")
iPhone6iOS12Title = "Manual test run for " + ios_key +\
" on iPhone 6/SE running iOS12"
iPhone6iOS12List = ["iPhone", "release-notes/exclude", "tests", "QA/Yes"]
iPhone_Title = "Manual test run for " + ios_key +\
" on iPhone running iOS12"
iPhone_List = ["iPhone", "release-notes/exclude", "tests", "QA/Yes"]

if args.test is None:
ios_repo.create_issue(title=iPhone6iOS12Title,
ios_repo.create_issue(title=iPhone_Title,
body=bigline,
assignee="srirambv",
milestone=ios_milestone[ios_key],
labels=iPhone6iOS12List)
labels=iPhone_List)

print("iPhone 7/8 iOS 12 Checklist:")
print("iPhone X Checklist:")
bigline = "## Per release speciality tests\n"

for line in iPhone8_iOS12_checklist:
for line in iPhoneX_checklist:
bigline += line + "\n"
bigline = bigline + ios_template
print(bigline)
print("")
iPhone8iOS12Title = "Manual test run for " + ios_key +\
" on iPhone 7/8 running iOS12"
iPhone8iOS12List = ["iPhone", "release-notes/exclude", "tests", "QA/Yes"]
iPhoneX_Title = "Manual test run for " + ios_key +\
" on iPhone X running iOS12"
iPhoneX_List = ["iPhone", "release-notes/exclude", "tests", "QA/Yes"]

if args.test is None:
ios_repo.create_issue(title=iPhone8iOS12Title,
ios_repo.create_issue(title=iPhoneX_Title,
body=bigline,
assignee="GeetaSarvadnya",
milestone=ios_milestone[ios_key],
labels=iPhone8iOS12List)
labels=iPhoneX_List)

return

@@ -1,35 +1,35 @@

## Installer

- [ ] Check that installer is close to the size of last release
- [ ] Check that installer is close to the size of the last release
- [ ] Check the Brave version in About and make sure it is EXACTLY as expected

## Data

- [ ] Make sure that data from the last version appears in the new version OK
- [ ] Test that the previous version's cookies are preserved in the next version
- [ ] Test that saved passwords are retained upon upgrade
- [ ] Verify stats are retained when upgrading from previous version
- [ ] Verify per site settings are retained when upgrading from previous version
- [ ] Verify sync chain created in previous version is still retained on upgrade
- [ ] Verify stats are retained when upgrading from the previous version
- [ ] Verify per-site settings are retained when upgrading from the previous version
- [ ] Verify sync chain created in the previous version is still retained on upgrade

## Bookmarks

- [ ] Test that creating a bookmark in the left well works
- [ ] Test that clicking a bookmark in the left well loads the bookmark
- [ ] Test that deleting a bookmark in the left well works
- [ ] Test that creating a bookmark works
- [ ] Test that clicking a bookmark from bookmark manager loads the bookmark
- [ ] Test that deleting a bookmark works
- [ ] Test that creating a bookmark folder works
- [ ] Test that creating a bookmark inside the created folder works
- [ ] Test that you are able to add a bookmark directly inside a bookmark folder
- [ ] Test that you are able to delete a bookmark in edit mode
- [ ] Test that you are able to delete a bookmark folder with bookmarks inside
- [ ] Test adding a bookmark domain subpaths is retained and you are successfully able to visit the domain subpath in a new tab
- [ ] Test that you can add a bookmark directly inside a bookmark folder
- [ ] Test that you can delete a bookmark in edit mode
- [ ] Test that you can delete a bookmark folder with bookmarks inside
- [ ] Test adding a bookmark domain, subpaths is retained and you are successfully able to visit the domain subpath in a new tab

## Favourites

- [ ] Test editing favourite and chaning URL updates the favicons accordingly
- [ ] Test that you are able to remove favourites
- [ ] Test that you are able to add new favourites from share menu
- [ ] Test editing favourite and changing URL updates the favicons accordingly
- [ ] Test that you can remove favourites
- [ ] Test that you can add new favourites from the share menu

## Context menus

@@ -44,7 +44,7 @@
- [ ] Verify search box is shown when selected via the share menu
- [ ] Test successful find
- [ ] Test forward and backward find navigation
- [ ] Test failed find shows 0 results
- [ ] Test failed to find shows 0 results

## Private Mode

@@ -57,12 +57,12 @@

## History

- [ ] On youtube.com, thestar.com (or any other site using push state nav), navigate the site and verify history is added. Also note if the progress bar activates and shows progress
- [ ] Settings > Clear Private Data, and clear all. Check history is cleared and none of the favourites are cleared
- [ ] On youtube.com, thestar.com (or any other site using push state nav), navigate the site and verify history is added. Also, note if the progress bar activates and shows progress
- [ ] Settings > Clear Private Data, and clear all. Check only the history is cleared and favourites are retained

## Shields Settings

- [ ] Enable all switches in settings and visit a site and disable block scripts. Kill and relaunch app and verify if the site shield settings are retained
- [ ] Enable all switches in settings and visit a site and disable block scripts. Kill and relaunch the app and verify if the site shield settings are retained

## Site hacks

@@ -75,7 +75,7 @@

## Fullscreen

- [ ] Test that entering HTML5 full screen works. And pressing restore to go back exits full screen. (youtube.com)
- [ ] Test that entering HTML5 fullscreen works. And pressing restore to go back exits full screen. (youtube.com)

## Gestures

@@ -85,27 +85,27 @@

## Password Managers

- [ ] Test tapping on 1Password on the slide out keyboard launches 1Password App and able to select the stored credentials
- [ ] Test tapping on bitwarden password manager in the autofill field launches the app and autofills the stored data
- [ ] Test tapping on 1Password on the slide-out keyboard launches 1Password App and able to select the stored credentials
- [ ] Test tapping on bitwarden password manager in the autofill field launches the app and auto-fills the stored data

## Browser Lock

- [ ] Test enabling browser pin settings asks for pin confirmation followed by reconfirm
- [ ] Test swipe up/swip down with browser in focus doesn't ask for pin confirmation
- [ ] Test clicking on set pin asks for pin to unlock before setting a new pin
- [ ] Remove app from memory and relaunch, enter wrong pin, browser should not be unlocked
- [ ] Test cancel fingerprint confirmation shows enter pin window when fingerprint unlock is setup on device
- [ ] Test swipe up/swipe down with browser in focus doesn't ask for pin confirmation
- [ ] Test clicking on set pin asks for the pin to unlock before setting a new pin
- [ ] Remove the app from memory and relaunch, enter a wrong pin, the browser should not be unlocked
- [ ] Test cancel fingerprint confirmation shows enter pin window when fingerprint unlock is set up on the device

## Sync

- [ ] Verify you are able to join sync chain by scanning the QR code
- [ ] Verify you are able to join sync chain using code words
- [ ] Verify you are able to create a sycn chain on the device and add other devices to the chain via QR code/Code words
- [ ] Verify you are able to create a sync chain on the device and add other devices to the chain via QR code/Codewords

This comment has been minimized.

Copy link
@LaurenWags

LaurenWags Aug 29, 2019

Contributor

++ the transposed letters in sync were killing me for awhile and I kept forgetting to say anything 😆

This comment has been minimized.

Copy link
@srirambv

srirambv Aug 29, 2019

Author Contributor

It was bothering me as well but kept thinking in the next edit will fix it and finally happened

This comment has been minimized.

Copy link
@LaurenWags

LaurenWags Aug 29, 2019

Contributor

Also, should Codewords be two separate words? I think they are in the app.

- [ ] Verify that bookmarks from other devices on the chain show up on the mobile device after sync completes
- [ ] Verify newly created bookmarks get sync'd to all devices on the sync chain
- [ ] Verify newly created bookmarks get synced to all devices on the sync chain
- [ ] Verify existing bookmarks before joining sync chain also gets sync'd to all devices on the sync chain

This comment has been minimized.

Copy link
@LaurenWags

LaurenWags Aug 29, 2019

Contributor

sync'd to synced to match line above.

- [ ] Verify sync works on a upgrade profile and new bookmarks added post upgrade sync's across devices on the chain
- [ ] Verify you are able to create a standalone sync chain with one device
- [ ] Verify sync works on an upgrade profile and new bookmarks added post-upgrade sync's across devices on the chain
- [ ] Verify you can create a standalone sync chain with one device

## Bravery settings

@@ -116,11 +116,11 @@
- [ ] Test that clicking through a cert error in https://badssl.com/ works
- [ ] Test that Safe Browsing works (https://www.raisegame.com/)
- [ ] Turning Safe Browsing off and shields off both disable safe browsing for https://www.raisegame.com/
- [ ] Enable block script globally from settings, Visit https://brianbondy.com/, nothing should load. Tap on Shields and disable block script, page should load properly
- [ ] Enable block script globally from settings, Visit https://brianbondy.com/, nothing should load. Tap on Shields and disable block script, the page should load properly
- [ ] Test that preferences default Bravery settings take effect on pages with no site settings
- [ ] Test that 3rd party storage results are blank at https://jsfiddle.net/7ke9r14a/7/ when 3rd party cookies are blocked
### Fingerprint Tests
- [ ] Test that turning on fingerprinting protection in preferences shows 1 fingerprints blocked at https://browserleaks.com/canvas . Test that turning it off in the Bravery menu shows 0 fingerprints blocked
- [ ] Test that turning on fingerprinting protection in preferences shows 1 fingerprint blocked at https://browserleaks.com/canvas . Test that turning it off in the Bravery menu shows 0 fingerprints blocked
- [ ] Test that audio fingerprint is blocked at https://audiofingerprint.openwpm.com/ when fingerprinting protection is on
- [ ] Test that https://diafygi.github.io/webrtc-ips/ doesn't leak IP address when `Block all fingerprinting protection` is on

@@ -131,9 +131,9 @@
- [ ] Go to https://trac.torproject.org/projects/tor/login and make sure that the password can be saved. Make sure the saved password is auto-populated when you visit the site again
- [ ] Open an email on http://mail.google.com/ or inbox.google.com and click on a link. Make sure it works
- [ ] Test that PDF is loaded over https at https://basicattentiontoken.org/BasicAttentionTokenWhitePaper-4.pdf
- [ ] Test that PDF is loaded over http at http://www.pdf995.com/samples/pdf.pdf
- [ ] Test that PDF is loaded over HTTP at http://www.pdf995.com/samples/pdf.pdf
- [ ] Test that https://mixed-script.badssl.com/ shows up as grey not red (no mixed content scripts are run)
- [ ] Test that search results from https://startpage.com/ opens in a new tab (due to target being _blank)
- [ ] Test that search results from https://startpage.com/ open in a new tab (due to target being _blank_)

## App linker

@@ -145,4 +145,4 @@

## Session storage

- [ ] Test that tabs restore when closed, including active tab
- [ ] Test that tabs restore when closed, including active tab
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.