A comprehensive suite of BackstopJS scenarios to automate visual regression testing on CiviCRM
A CiviCRM site with sample data (check the "Load sample data" option when running the installer)
- The uk.co.vedaconsulting.mosaico extension should not be enabled on the site. In case it is, either momentarily disable it or remove the mailings_menu.json scenarios group
- Clean URLs enabled
- The user name for admin should be
admin
- Go to Actions Tab
- Click on 'Run Workflow' button.
- Enter required 'Reference CiviCRM Version', 'Test CiviCRM Version', 'Shoreditch Reference Branch/Tag Name' and 'Shoreditch Test Branch/Tag Name'.
- Wait for the Action to finish, and download the Artifacts to compare the screenshots.
- Create a
backstop_data/site-config.json
file with the following content.{ "url": "your_local_url", "root": "absolute_path_to_site" }
- Setup required data
npx gulp backstopjs:setup-data
- Create the reference screenshots
npx gulp backstopjs:reference
- Create the test screenshots and compare
npx gulp backstopjs:test
In case you want to run backstop only for the scenarios of a specific group (ie. administer-menu.json
), pass the --group
argument to the gulp task (without the .json
extension)
npx gulp backstopjs:test --group administer-menu
BackstopJS supports taking multiple screenshot at once. Change the value of asyncCaptureLimit
in backstop.tpl.json to decide how many screenshots you want to take in parallel
Note:Please be aware that BackstopJS performance is heavily dependent on the specs of the machine it runs on, so make sure to choose a value that the tool can handle on your machine (otherwise you will encounter random timeout errors)
This backstop test suite relies on session cookies to be present in the backstop_data/cookies/admin.json
file. The file is automatically generated by the gulp
task by using the drush uli
command.
If you want to skip the cookie creation phase (in case the cookies are already present in the folder and are not expired yet), pass a --skipCookies
argument to the task
npx gulp backstopjs:reference --skipCookies
- Find Contacts
- Advanced search
- Standard search results
- Advanced search results
- Full-text Search
- Full-text Search results
- Search Builder
- Find Contributions
- Find Mailings
- Find Membership
- Find Participants
- Find Participants results
- Find Pledges
- Find Activities
- Custom Searches
- New Individual
- New Household
- New Organization
- Contact Reports
- New Activity
- New Email
- Import Contacts
- Import Activities
- New Group
- Manage Groups
- New Tag
- Manage Tags (Categories)
- Find and Merge Duplicate Contacts
- Dashboard
- New Contribution
- Find Contribution
- Contribution Reports
- Import Contributions
- Pledges - Dashboard
- Pledges - New Pledges
- Pledges - Find Pledges
- Pledges - Pledges Reports
- Pledges - New Pledge Report
- Batch Data Entry
- New Batch Data Entry
- Accouting Batch - New Batch
- Accouting Batch - Open Batches
- Accouting Batch - Closed Batches
- Accouting Batch - Exported Batches
- New Contribution Page
- Manage Contribution Pages
- Personal Campaign Pages
- Premiums (Thank-you Gifts)
- New Price Set
- Manage Price Set
- Close Accounting Period
- Dashboard
- Register Event Participant
- Find Participants
- Event Reports
- Import Participants
- New Event
- Manage Events
- New Mailings
- Draft and Unscheduled Mailings
- Scheduled and Sent Mailings
- Archived Mailings
- Mailing Reports
- Headers, Footers, and Automated Messages
- Headers, Footers, and Automated Messages - Add Mailing Component
- Message Templates
- From Email Addresses
- New SMS
- Find Mass SMS
- New A/B Test
- Manage A/B Tests
- Dashboard
- New Membership
- Find Memberships
- Membership Reports
- Batch Data Entry
- Import Memberships
- System Status
- Configuration Checklist
- Custom Fields
- New Custom Field
- Profiles
- Name and Address - CiviCRM Profile Fields
- Tags
- Activity Types
- Relationship Types
- Contact Types
- Dropdown Options
- Gender Options
- Individual contact prefixes Options
- Individual contact suffixes Options
- Instant Messenger (IM) screen-names Options
- Location Types (Home, Work...)
- Mobile Phone Providers Options
- Phone Type Options
- Website Type Options
- Display Preferences
- Search Preferences
- Date Preferences
- Navigation Preferences
- World Replacements
- Manage Custom Searches
- Organization Address and Contact Info
- Message Templates
- New Message Template
- Schedule Reminders
- Preferred Communication Methods
- Label Formats
- Print Page (PDF) Formats
- Communication Style Options
- Email Greeting Type Options
- Postal Greeting Type Options
- Addressee Type Options
- Settings
- Addresses
- Date
- Languages Options
- Access Control
- Synchronize Users to Contacts
- Enable Components
- Connections
- Extensions
- Cleanup Caches and Update Paths
- Debugging and Error Handling
- Upload Directories
- Import/Export Mappings
- Mapping and Geocoding Providers
- Misc
- Option Groups
- Payment Processor
- Resource URLs
- Safe File Extension Options
- Scheduled Jobs
- SMS Provider
- All Reports
- Create New Report from Template
- Manage Templates
- Register Report
- Membership Types
- Membership Status Rules
- Component Settings
- Survey Types
- Campaign Types
- Campaign Status
- Engagement Index
- CiviCampaign Component Settings
- Financial Types
- Financial Accounts
- Payment Methods Options
- Accepted Credit Cards Options
- Soft Credit Types Options
- Payment Processor
- Payment Processor
- Event Templates
- Event Type Options
- Participant Status
- Participant Role Options
- Participant Listing Options
- Event Name Badge Layouts
- New Event Name Badge Layout
- CiviEvent Component Settings
- Mail Accounts
- CiviMail Settings
- CiviMail Component Settings