Release QA Update Testing Requirements

Coverage Requirements

Release

Channels	release-localtest, release-cdntest, release
Locales	ar, de, en-US, es-ES, fr, ja, ja-JP-mac, ko, pl, pt-PT, ru, tr, vi, zh-CN
Versions	<= 55.* ⭢ 56.0 ⭢ latest on Windows <= 55.* ⭢ 57.0.4 ⭢ latest on Mac OS and Linux >= 56.* ⭢ latest on all platforms
Platforms	Windows 10 Windows 7 Mac OS 10.13 Ubuntu 16.04 LTS
Tests	manual update background update what's new page

Beta

Channels	beta-cdntest, beta
Locales	ar, de, en-US, es-ES, fr, ja, ja-JP-mac, ko, pl, pt-PT, ru, tr, vi, zh-CN
Versions	<= 56.0b2 ⭢ 56.0b3 ⭢ latest >= 56.0b3 ⭢ latest
Platforms	Windows 10 Windows 7 Mac OS 10.13 Ubuntu 16.04 LTS
Tests	manual update background update

Developer Edition

Channels	aurora-cdntest, aurora
Locales	ar, de, en-US, es-ES, fr, ja, ja-JP-mac, ko, pl, pt-PT, ru, tr, vi, zh-CN
Versions	<= 56.0b2 ⭢ 56.0b3 ⭢ latest >= 56.0b3 ⭢ latest
Platforms	Windows 10 Windows 7 Mac OS 10.13 Ubuntu 16.04 LTS
Tests	manual update background update

ESR

Channels	esr-localtest, esr-cdntest, esr
Locales	ar, de, en-US, es-ES, fr, ja, ja-JP-mac, ko, pl, pt-PT, ru, tr, vi, zh-CN
Versions	<= ESR 45 ⭢ ESR 52 ⭢ latest <= ESR 45 ⭢ ESR 52 (Windows XP/Vista only) >= ESR 52 ⭢ latest
Platforms	Windows 10 Windows 7 Mac OS 10.13 Ubuntu 16.04 LTS Windows XP/Vista (ESR <=52 only)
Tests	manual update background update Windows XP/Vista EOL

Nightly

Updates for Nightly are not currently tested on a regular basis.

Windows XP/Vista EOL

Firefox ESR 52 is the end of the line for users on Windows XP and Windows Vista.

• ESR <= 45 should update to ESR 52 and present the Windows XP/Vista EOL KB article
• ESR 52 should not update further on Windows XP and Vista

Whats New Page

Any given release may receive a Whats New Page as defined in a release-specific bugzilla report and must be tested for the following conditions. Builds/locales not specified must also be tested to ensure the What's New Page does not appear.

Firefox Account	Build	Sync'd Mobile Device	Send-to Widget Support¹	What's New Page²
Signed Out	en-US	n/a	n/a	Pocket signup page
Signed Out	localized	n/a	n/a	Firefox Account signup page
Signed In	all	✘	✘	QR code page
Signed In	all	✘	✔	Send To widget page
Signed In	all	✔	✘	QR code page
Signed In	all	✔	✔	Send To widget page

¹ Send-to Widget support varies release to release as defined in bedrock/settings/base.py
² What's New Page support varies release to release as defined in a release-specific bug

Note: Not all releases have a WNP, this may need to be subject to manual testing in the short term

Reporting

Once update tests are completed a sign-off email is sent to the release sign-off team using one of these templates.

For more information on the update testing sign-off workflow please see the Build Updates Sign-off Flows document.

Functional Tests used for Manual Testing

• Media playback & support
• New Tab page / Activity Stream
• Plugin compatibility & support
• Crash reporting
• Session restore
• Printing
• Graphics compatibility & support
• Download manager
• Search
• Build installation & uninstallation
• Awesomebar & indicators
• Profile support & compatibility
• WebExtensions compatibility & support
• Content scroll, zoom & drag support
• Safe browsing
• Private browsing
• Geolocation
• In-browser PDF support
• Image compatibility & support
• Default & custom theme support
• File system interaction
• Data migration
• Privacy
• Bookmarks
• Browsing History
• Password manager & form autofill
• Toolbars & window controls