4. Running tests in console

Running tests in console

In order to run all tests for all pages in console file with list of all pages under test should be created. For example we have aplication named SampleApp and tests located in sc_website/sitecore/shell/client/Tests Also we have two pages with 3 test files:

/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage?sc_testrunnerjs=1  
/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage/Pages/SecondPage?sc_testfixture=additionaltests&sc_testrunnerjs=1  
/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage/Pages/SecondPage?sc_testrunnerjs=1

[Deprecated in 0.8.0]
In this case we have to create sampleApp.js file in test folder root and set to testPages variable array of urls to run under PhantomJS in a loop:

testPages = [
  "/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage?sc_testrunnerjs=1",
  "/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage/Pages/SecondPage?sc_testfixture=additionaltests&sc_testrunnerjs=1",
  "/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage/Pages/SecondPage?sc_testrunnerjs=1"
];

[Introduced in 0.8.0]
In this case we have to create sampleApp.json file in test folder root with array of urls to run under PhantomJS in a loop:

[
  "/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage?sc_testrunnerjs=1",
  "/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage/Pages/SecondPage?sc_testfixture=additionaltests&sc_testrunnerjs=1",
  "/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage/Pages/SecondPage?sc_testrunnerjs=1"
]

When config file with all pages is in place we have to perform following steps:

1. Install PhantomJS nuget package
2. Install Sitecore.TestRunnerJS nuget package
3. Copy TestRunnerJS files from package tools folder to website folder
4. Run test fixtures using PhantomJS.exe using following command:
   [Deprecated in 0.9.0]
   phantomjs.exe loader.js deployed_instance_name app_name report.js [TIMEOUT]

Where:

• phantomjs.exe - path to phantomjs executable from PhantomJS package
• loader.js - path to phantomjs test loader.js javascript file that is located in \sitecore\TestRunnerJS\assets folder on deployed instance
• deployed_instance_name - name of your deployed instance
• app_name - name of your SPEAK application
• report.js - path to console output report generator that is located in \sitecore\TestRunnerJS\assets folder on deployed instance
• TIMEOUT - optional value of test fixture load timeout in milliseconds. Can be big due to deployed instance warmup on first requests

[Introduced in 0.9.0]
phantomjs.exe loader.js deployed_instance_name app_name [ADDITIONAL PARAMETERS]

Where:

• phantomjs.exe - path to phantomjs executable from PhantomJS package
• loader.js - path to phantomjs test loader.js javascript file that is located in \sitecore\TestRunnerJS\assets folder on deployed instance
• deployed_instance_name - name of your deployed instance
• app_name - name of your SPEAK application

Additional parameters:

• -t "timeout" - test fixture load timeout in milliseconds,
• -g "grep" - mocha.js grep parameter for selecting tests to run,
• -i "invert" - inverts results of grep parameter,
• -r "reporter" - path to console output report generator that is located in \sitecore\TestRunnerJS\assets folder on deployed instance,
• -v "verbose" - show SPEAK debug output in console,
• -u "url" - runs tests only on specified by url page,
• -o "outputReportPath" - full path to generate resulting html output report

Powershell deploy script example:

$instanceName = "sitecore8"
$instancePath = "c:\inetpub\wwwroot\$instanceName"
$websitePath = "$instancePath\website"
$toolsPath = "$instancePath\tools"
    
# Install PhantomJS from nuget into tools folder
# Install Sitecore.TestRunnerJS from nuget into tools folder
    
$phantomPath = "$toolsPath\PhantomJS\tools\phantomjs\phantomjs.exe"
Copy-Item -Path "$toolsPath\Sitecore.TestRunnerJS\tools\*" -Destination "$websitePath" -recurse -Force
    
$phantomScriptsFolderPath = "$websitePath\sitecore\TestRunnerJS\phantom"
$loaderPath = "$phantomScriptsFolderPath\loader.js"
$reportPath = "$phantomScriptsFolderPath\report.js"

&$phantomPath $loaderPath $instanceName 'myapplication' $reportPath 180000

if (!$?) { throw "Failures reported in javascript integration tests." }`

Sample console output when running tests:

Installing 'PhantomJS 1.9.8'.
Successfully installed 'PhantomJS 1.9.8'.
Installing 'Sitecore.TestRunnerJS 0.6.3'.
Successfully installed 'Sitecore.TestRunnerJS 0.6.3'.
Working instance: sca
Application name: sampleApp
Loading test settings.
Test settings was loaded.
Login page loaded! Url: http://sca/sitecore/login
Launch pad page loaded! Url: http://sca/sitecore/client/Applications/Launchpad?sc_lang=en
Testing page: http://sca/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage?sc_testrunnerjs=1
Total tests on page: 1
[PASS] Another page Should load test from ActionTextBox into ResultTextBox on Action button click
Testing page: http://sca/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage/Pages/SecondPage?sc_testfixture=additionaltests&sc_testrunnerjs=1
Total tests on page: 1
[PASS] Second page Should process selected item
Testing page: http://sca/sitecore/shell/sitecore/client/Applications/SampleApp/FirstPage/Pages/SecondPage?sc_testrunnerjs=1
Total tests on page: 2
[PASS] Second page Should load list data from server
[PASS] Second page Should show message from server on item selection
Tests execution was finished.
Passed: 4 Failed: 0 Total: 4 Time: 30968ms

[Introduced in 0.7.0]
It is possible to run console version from Visual Studio. It order to do so just perform following steps:

1. Open Package Manager Console in Visual Studio
2. Run in console Execute-TestRunner <instance_name> <app_name>
   Where:
   <instance_name> - Your website instance name
   <app_name> - Speak application name (SampleApp)
   [Introduced in 0.9.0]
   Also several optional parameters could be useful:
3. <grep> - Mocha.js grep expression to filter running tests
4. <invert> - Inverts grep parameter
5. <outputReportPath> - Full path where to store test execution results html report. (Will not be generated if parameter is not specified)