Skip to content
This repository has been archived by the owner on Dec 11, 2019. It is now read-only.

Manual tests for OS X 0.12.2 RC2 #4204

Closed
alexwykoff opened this issue Sep 22, 2016 · 7 comments
Closed

Manual tests for OS X 0.12.2 RC2 #4204

alexwykoff opened this issue Sep 22, 2016 · 7 comments
Assignees
@alexwykoff
Labels
OS/macOS QA/no-qa-needed tests
Milestone
0.12.2dev

Comments

@alexwykoff
Copy link
Contributor

alexwykoff commented Sep 22, 2016 •
edited

Manual Tests Template

Copy this to a new issue on each release.
The tests should run for OS X, Windows x64, Windows ia32, and a Linux distro for each release.

Installer

  1. Check that installer is close to the size of last release.
  2. Check signature: If OS Run spctl --assess --verbose /Applications/Brave.app/ and make sure it returns accepted. If Windows right click on the installer exe and go to Properies, go to the Digital Signatures tab and double click on the signature. Make sure it says "The digital signature is OK" in the popup window.
  3. Check Brave, electron, and libchromiumcontent version in About and make sure it is EXACTLY as expected.

Data

  1. Make sure that data from the last version appears in the new version OK.
  2. Test that the previous version's cookies are preserved in the next version.

About pages

  1. Test that about:bookmarks loads bookmarks
  2. Test that about:downloads loads downloads
  3. Test that about:preferences changing a preference takes effect right away
  4. Test that about:preferences language change takes effect on re-start
  5. Test that about:passwords loads
  6. Test that about:history loads

Bookmarks

  1. Test that creating a bookmark on the bookmarks toolbar works
  2. Test that creating a bookmark folder on the bookmarks toolbar works
  3. Test that moving a bookmark into a folder by drag and drop on the bookmarks folder works
  4. Test that clicking a bookmark in the toolbar loads the bookmark.
  5. Test that clicking a bookmark in a bookmark toolbar folder loads the bookmark.

Context menus

  1. Make sure context menu items in the URL bar work
  2. Make sure context menu items on content work with no selected text.
  3. Make sure context menu items on content work with selected text.
  4. Make sure context menu items on content work inside an editable control (input, textarea, or contenteditable).

Find on page

  1. Ensure search box is shown with shortcut
  2. Test successful find
  3. Test forward and backward find navigation
  4. Test failed find shows 0 results
  5. Test match case find

Geolocation

  1. Check that https://developer.mozilla.org/en-US/docs/Web/API/Geolocation/Using_geolocation works

Site hacks

  1. Test https://www.twitch.tv/adobe sub-page loads a video and you can play it

Downloads

  1. Test downloading a file works and that all actions on the download item works.

Fullscreen

  1. Test that entering full screen window works View -> Toggle Full Screen. And exit back (Not Esc).
  2. Test that entering HTML5 full screen works. And Esc to go back. (youtube.com)

Tabs and Pinning

  1. Test that tabs are pinnable
  2. Test that tabs are unpinnable
  3. Test that tabs are draggable to same tabset
  4. Test that tabs are draggable to alternate tabset

Zoom

  1. Test zoom in / out shortcut works
  2. Test hamburger menu zooms.
  3. Test zoom saved when you close the browser and restore on a single site.
  4. Test zoom saved when you navigate within a single origin site.
  5. Test that navigating to a different origin resets the zoom

Bravery settings

  1. Check that HTTPS Everywhere works by loading http://www.apple.com
  2. Turning HTTPS Everywhere off and shields off both disable the redirect to https://www.apple.com
  3. Check that ad replacement works on http://slashdot.org
  4. Check that toggling to blocking and allow ads works as expected.
  5. Test that clicking through a cert error in https://badssl.com/ works.
  6. Test that Safe Browsing works (http://excellentmovies.net/)
  7. Turning Safe Browsing off and shields off both disable safe browsing for http://excellentmovies.net/.
  8. Visit https://brianbondy.com/ and then turn on script blocking, nothing should load. Allow it from the script blocking UI in the URL bar and it should work.
  9. Test that about:preferences default Bravery settings take effect on pages with no site settings.
  10. Test that turning on fingerprinting protection in about:preferences shows 3 fingerprints blocked at https://jsfiddle.net/bkf50r8v/13/. Test that turning it off in the Bravery menu shows 0 fingerprints blocked.
  11. Test that 3rd party storage results are blank at https://jsfiddle.net/7ke9r14a/9/ when 3rd party cookies are blocked and not blank when 3rd party cookies are unblocked.
  12. Test that audio fingerprint is blocked at https://audiofingerprint.openwpm.com/ when fingerprinting protection is on.

Content tests

  1. Go to https://brianbondy.com/ and click on the twitter icon on the top right. Test that context menus work in the new twitter tab.
  2. Load twitter and click on a tweet so the popup div shows. Click to dismiss and repeat with another div. Make sure it shows.
  3. Go to http://www.bennish.net/web-notifications.html and test that clicking on 'Show' pops up a notification asking for permission. Make sure that clicking 'Deny' leads to no notifications being shown.
  4. Go to https://trac.torproject.org/projects/tor/login and make sure that the password can be saved. Make sure the saved password shows up in about:passwords. Then reload https://trac.torproject.org/projects/tor/login and make sure the password is autofilled.
  5. Open a github issue and type some misspellings, make sure they are underlined.
  6. Make sure that right clicking on a word with suggestions gives a suggestion and that clicking on the suggestion replaces the text.
  7. Make sure that Command + Click (Control + Click on Windows, Control + Click on Ubuntu) on a link opens a new tab but does NOT switch to it. Click on it and make sure it is already loaded.
  8. Open an email on http://mail.google.com/ or inbox.google.com and click on a link. Make sure it works.
  9. Test that PDF is loaded at http://www.orimi.com/pdf-test.pdf
  10. Test that https://mixed-script.badssl.com/ shows up as grey not red (no mixed content scripts are run).

Flash tests

  1. Turn on Flash in about:preferences#security. Test that clicking on 'Install Flash' banner on myspace.com shows a notification to allow Flash and that the banner disappears when 'Allow' is clicked.
  2. Test that flash placeholder appears on http://www.y8.com/games/superfighters

Autofill tests

  1. Test that autofill works on http://www.roboform.com/filling-test-all-fields

Per release specialty tests

  • Windows 7 - selection problems with first item in custom menu titlebar (#4142)
    • URL in the bar is replaced during load bug (#4191)
    • Unable to run flash content on SiriusXM.com (#4195)
    • change import dialog switch default from OFF to ON for available data types (#4187)
    • add confirmation dialog after successful import of data from other browser (#4188)
    • Regression: Google Adsense responsive ads no longer blocked (#4172)
    • Regression: http://excellentmovies.net/ cannot be blocked (Safe Browsing does not work) (#4173)
    • Fix the check for fullscreen window. (#4181)
    • Unable to delete autofill data (#3725)
    • Multiple issues with Autofill (#4155)
    • Import Browser Data is shown under Help menu importer (#4177)
    • Windows - Keyboard does not work with submenus help wanted titlebar (#4143)
    • Pasting into a input field causes a immediate crash (#4169)
    • Hitting a space after inputting a character inside input field causes a crash (#4176)
    • ledger notification should have the style of the update bar (#3257)
    • Browser crashes when signing into Facebook crash (#4153)
    • Import Bookmarks doesn't import data from other browsers (#4154)
    • Connection error tabs not showing error message (#4036)
    • chrome 53 ERR_CERTIFICATE_TRANSPARENCY_REQUIRED are not handled properly (#4158)
    • Unique Device ID's privacy (#1883)
    • ignore ERR_CERTIFICATE_TRANSPARENCY_REQUIRED until https://github.com… (#4150)
    • Add Importer support (#4049)
    • Import data from other browsers (#428)
    • Titlebar on Windows has a lot of wasted space (#3036)
    • Optimize titlebar space on Windows (#3854)
    • fix standardjs 8.1.0 linter errors (#4135)
    • ledger state files no longer need suffixes (#4132)
    • Delete bookmark folder won't delete its child items (#4124)
    • Deleted bookmarks under folder shows up in URL (#3763)
    • Upgrade to Chromium v 53.0.2785.116 (#1593)
    • Add option to clear site-specific Bravery settings (#2545)
    • add right-click to synopsis entries to delete them (#3829)
    • Stop recreating shortcut on my desktop after every update (#1181)
    • Swipe gestures are very touchy and require a very fast movement (#4064)
    • "Bring All to Front" does nothing on Windows (#3664)
    • Show "Bring All to Front" only on macOS (#4093)
    • Start searching bookmarks as soon as characters are typed bookmarks (#2778)
    • Search bookmarks as soon as characters are typed (#4097)
    • Fix sortableTable hover effect and searchTab onClick (#4104)
    • Changing default search provider no longer working (#4099)
    • Line spacing regression in preferences panel (#4100)
    • Line spacing regression in preferences panel (#4101)
    • Horizontal scroll with two-finger conflicts with swipe back/forward (#2548)
    • Geo Location doesn't detect user location (#3825)
    • npm run clean-session-store and similar clear commands no longer working (#3912)
    • View page source should respect private windows (#4077)
    • Check page status code before running ledger timer (#3759)
    • Don't register a visit if page returns a non-successful HTTP status code (#4032)
    • Page title getting picked from Bookmarks rather than the page itself history (#4029)
    • History now uses title (not custom title) (#4048)
    • Horizontal Scroll bar appears with just one entry in Brave Payments (#3792)
    • Horizontal Scroll bar appears with just one entry in Brave Payments (#4039)
    • Use language name for sort order instead of identifier. Fix a few outdated comments (#4055)
    • Add "note" text to the Brave Payments Welcome page for country-detection (#4067)
    • add note to payments welcome page (#4068)
    • QR Code Popup from Add funds dialog scrolls independently (#4071)
    • QR Code Popup from Add funds dialog scrolls independently from its parent bug (#4043)
    • Reorder Brave Payments buttons to improve 1st-time flow (#4040)
    • Reorder Brave Payments buttons to improve 1st-time flow, and sync up to auto-wallet creation (#3963)
    • System buttons on window should not be wrapped titlebar (#4185)
    • View Page Source don't work for SVG files (#4056)
    • sortabletable fixes, make ledger table sortable (#4057)
    • ledger synopsis table should be sortable (#4046)
    • about:history sort-by-time sorts in lexicographical order, not chronological history (#4047)
    • Updated to version Brave: 0.12.0 and my $5 is gone (#3781)
    • add network connection notifications (#4038)
    • Hide noscript and bookmarks in title mode and fill in bookmark star if bookmarked (#4042)
    • Blockscript/Favourite icons look faded (#4041)
    • Blockscript/Favourite icons look faded (#4035)

Ledger

  1. Create a wallet with a value other than $5 selected in the monthly budget dropdown. Click on the 'Add Funds' button and check that Coinbase transactions are blocked.
  2. Remove all ledger-*.json files from ~/Library/Application\ Support/Brave/. Go to the Payments tab in about:preferences, enable payments, click on create wallet. Check that the add funds button appears after a wallet is created.
  3. Click on add funds and verify that adding funds through Coinbase increases the account balance.
  4. Repeat the step above but add funds by scanning the QR code in a mobile bitcoin app instead of through Coinbase.
  5. Visit nytimes.com for a few seconds and make sure it shows up in the Payments table.
  6. Go to https://jsfiddle.net/LnwtLckc/5/ and click the register button. In the Payments tab, click add funds. Verify that the transfer funds button is visible and that clicking on transfer funds opens a jsfiddle URL in a new tab.
  7. Go to https://jsfiddle.net/LnwtLckc/5/ and click unregister. Verify that the transfer funds button no longer appears in the add funds modal.
  8. Check that disabling payments and enabling them again does not lose state.

Session storage

Do not forget to make a backup of your entire ~/Library/Application\ Support/Brave folder.

  1. Temporarily move away your ~/Library/Application\ Support/Brave/session-store-1 and test that clean session storage works. (%appdata%\Brave in Windows, ./config/brave in Ubuntu)
  2. Test that windows and tabs restore when closed, including active tab.
  3. Move away your entire ~/Library/Application\ Support/Brave folder (%appdata%\Brave in Windows, ./config/brave in Ubuntu)

Cookie and Cache

  1. Make a backup of your profile, turn on all clearing in preferences and shut down. Make sure when you bring the browser back up everything is gone that is specified.
  2. Go to http://samy.pl/evercookie/ and set an evercookie. Check that going to prefs, clearing site data and cache, and going back to the Evercookie site does not remember the old evercookie value.

Update tests

  1. Test that updating using BRAVE_UPDATE_VERSION=0.8.3 env variable works correctly.
@alexwykoff alexwykoff added this to the 0.12.2dev milestone Sep 22, 2016
@alexwykoff alexwykoff self-assigned this Sep 22, 2016
@alexwykoff alexwykoff mentioned this issue Sep 22, 2016
Closed
@alexwykoff
Copy link
Contributor Author

#4032 failed for me

@alexwykoff
Copy link
Contributor Author

#3759 failed for me

@alexwykoff
Copy link
Contributor Author

#3825 failed for me

@alexwykoff
Copy link
Contributor Author

Missing QA directions for #3257

@alexwykoff
Copy link
Contributor Author

Questionable behavior for #4181

@alexwykoff
Copy link
Contributor Author

#4220 created to track regression

@alexwykoff
Copy link
Contributor Author

all above issues have been tracked appropriately, closing this run

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@alexwykoff alexwykoff

Labels
OS/macOS QA/no-qa-needed tests
Projects
None yet
Milestone
0.12.2dev
Development

No branches or pull requests
2 participants
@alexwykoff @luixxiul