2. Slim Fixtures

Fried Hoeben edited this page Nov 15, 2016 · 34 revisions

This project’s fixtures’ documentation can be found in the installation’s wiki, which also contains working test examples.

A single integration test (i.e. wiki page) can combine the usage of these fixtures either by defining separate tables for each, or by combining methods defined by multiple fixtures in a single script table. This, for instance, allows to create test using a single script that first places an online order using a browser and then checks the order’s content in a back-end system with a SOAP call.

A high level overview of the project’s fixtures

The main fixtures in the project are:

  1. BrowserTest: test web applications using Selenium
  2. (Xml/Json)HttpTest: test web services

BrowserTest

BrowserTest defines a language (a DSL of sorts) to interact with a website via a browser. Normally a heuristic is used to find elements using end-user terms, leading to readable wiki pages describing tests, robust to technical changes in the site’s HTML. But this heuristic may be bypassed in favor of technical selectors (e.g. ‘id=abc1’) to specify which element is meant. Furthermore BrowserTest retries finding elements (for a configurable time) if it cannot locate the immediately. This allows tests of dynamic web pages (where the page content is updated asynchronous using JavaScript) or using browsers where Selenium is not always able to determine whether a page load is complete (e.g. Safari).

The language supported by BrowserTest is defined by its public methods, which can be used in combination with Slim’s standard script verbs (e.g. check, ensure, reject, show).
BrowserTest’s language will suffice for many tests, but can be extended by creating a custom subclass.

A specific subclass (NgBrowserTest) is included to test AngularJs sites. This class has functionality to find elements based on ‘models’, ‘bindings’ and ‘repeaters’ (borrowed from Protractor).

Language

BrowserTest

The main commands/keywords (i.e. public methods) offered by BrowserTest are listed below.
The heuristic to find places tries a number of translations of end user terms to Selenium ‘find’ commands (e.g. is a full or partial label given, a link’s or button’s text, a placeholder). The first ‘find’ that results in an element on the page is used by BrowserTest.

open url opens the specified URL
click place performs mouse click at the designated place (e.g. link/button/checkbox/radio button)
click if available place performs mouse click at the designated place (e.g. link/button/checkbox/radio button). Does not throw an exception on timeout but returns false, so can be used with script table’s ‘reject’ verb to check something cannot be clicked.
enter value as input clears the current value of an input and types the value
select value for input selects a value in the drop down box (or radio button)
available options for input retrieves all possible values for the drop down box
type text enters the supplied text in the currently active element
press key sends press of a key (e.g. Tab, Enter, Escape, Space, M) to browser
clear input removes the content in the input
take screenshot name stores the current browser’s content as .png (used in combination with ‘show’ verb)
page source gets the current browser’s content (used in combination with ‘show’ verb, the html is escaped and added to the wiki table)
save page source stores the current browser’s content as .html file (used in combination with ‘show’ verb, a link to the created file is added to the wiki table)
value of input retrieves the current value of the input (for use with ‘check’ verb or variable assignment)
values of place retrieves all values of place, which is expected to be a ‘select’ or list
number for list item text retrieves the number of the item (in a numbered list) with the supplied text
page title retrieves the current page’s title (for use with ‘check’ verb)
is visible place whether specified place is visible in browser’s window (used in combination with ‘ensure’ and ‘reject’)
is enabled place whether specified place is enabled (i.e. can be clicked) (used in combination with ‘ensure’ and ‘reject’)
scroll to place scrolls the designated place into view
hover over place simulates hovering over the designated place with the mouse (so e.g. ‘mouseover’ behavior can be tested)
wait for page title waits until a page with the specified title is shown
wait for visible place waits until the specified place (i.e. element) is visible in the browser
seconds before page load timeout value sets how many seconds to wait for the browser to load the next page after a request before an exception is thrown
seconds before page load timeout retrieves how many seconds is waited when loading a new page before an exception is thrown
seconds before timeout value sets how many seconds to wait on the browser before an exception is thrown
seconds before timeout retrieves how many seconds is waited before an exception is thrown
wait seconds (milli)seconds waits the specified number of (milli)seconds (e.g. for an animation to complete)
location returns the current web address (i.e. URL of page)
back goes back 1 page in the browser’s history (i.e. clicks browser’s back button)
forward goes forward 1 page in the browser’s history (i.e. clicks browser’s forward button)
refresh refreshes the current page (i.e. clicks the brower’s refresh button)
alert text retrieves the text of the open JavaScript alert.
confirm alert confirms the open JavaScript alert.
dismiss alert dismisses the open JavaScript alert.
delete all cookies deletes all cookies(for the domain of the current open page in the browser)
clear local storage clears HTML5’s localStorage (for the domain of the current open page in the browser)
open in new tab url opens the specified URL in a new tab, and switches to that tab
switch to next tab Switches the browser’s window to the next tab
switch to previous tab Switches the browser’s window to the previous tab
close tab Closes the current tab and switches the browser’s window back to the previous one, if the browser has multiple tabs
tab count How many tabs are open in the browser currently?
current tab index Which tab is currently displayed in the browser’s window?
value of column number columnIndex in row number rowIndex retrieves the value from column number ‘columnIndex’ in row number ‘rowIndex’
value of columnName in row number rowIndex retrieves the value from the column with header ‘columnName’ in row number ‘rowIndex’
value of columnName in row where selectColumn is selectValue finds the first row where the column with header ‘selectColumn’ is ‘selectValue’ and retrieves the value from the column with header ‘columnName’
row exists where selectColumn is selectValue determines whether there is a row where the column with header ‘selectColumn’ is ‘selectValue’
click place in row number rowIndex clicks the button or link whose text matches ‘place’ in row number ‘rowIndex’
click place in row where selectColumn is selectValue finds the first row where the column with header ‘selectColumn’ is ‘selectValue’ and clicks the button or link whose text matches ‘place’
enter value as columnName in row where selectColumn is selectValue finds the first row where the column with header ‘selectColumn’ is ‘selectValue’ and enters ‘value’ in the column with header ‘columnName’
download link Downloads the target of the link (to the wiki’s files section).
download content from url Downloads the file at the specified url (to the wiki’s files section).
select file file for file input Selects the supplied file using the indicated input. The file may be in the wiki’s files section, or on the local file system.
download place from row number rowIndex Downloads the target of the link in column ‘place’, or with text ‘place’, in row number ‘rowIndex’.
download place from row where selectColumn is selectValue Finds the first row where the column with header ‘selectColumn’ is ‘selectValue’ and downloads the target of the link in column ‘place’, or with text ‘place’.
current browser width Retrieves the current width of the browser’s window (in pixels).
set browser width newWidth Sets the width of the browser’s window to ‘newWidth’ (in pixels).
current browser height Retrieves the current height of the browser’s window (in pixels).
set browser height newHeight Sets the height of the browser’s window to ‘newHeight’ (in pixels).
set browser size to newWidth by newHeight Sets the size of the browser’s window to ‘newWidth’ by ‘newHeight’ (in pixels).
set browser size to maximum Maximizes the size of the browser’s window.
set search context to container Limits searches for ‘places’ by other keywords/commands to inside ‘container’ (i.e. the child-elements inside the container element), instead of the whole page. This is useful in case the same ‘place’ occurs multiple times on the same page, to ensure which one is found. All subsequent searches are limited to ‘container’ until either the ‘clear search context’, ‘refresh’ or ‘open’ command is given. Many commands also offer an ‘in container’ version (e.g. ‘click place in container’) to limit searching only for that row.
clear search context Removes any search limits previously set by ‘set search context to container’ (i.e. subesquent searches will search the entire page).
set implicit wait for angular to true/false Controls whether BrowserTest should determine whether the site being tested uses AngularJs and if so, wait for Angular to finish rendering the page (default is ‘true’).
switch to default content Activates main/top-level (i)frame (i.e. makes it the current frame).
switch to frame technical selector Activates child frame of current (i)frame found using ‘technical selector’.
switch to parent frame Activates parent frame of current (i)frame. Does nothing if when current frame is the main/top-level one.
set implicit find in frames to true/false Controls whether BrowserTest should look for elements inside nested (i)frames if they cannot be found in the current frame (default is ‘true’).

NgBrowserTest

NgBrowserTest ensures Angular is finished processing the page before trying to find elements or interact with them (BrowserTest has basic functionality for that, NgBrowserTest ‘requires’ Angular to be present). Furthermore it adds finding elements by based on Angular bindings and models to the heuristic. Finally it adds some commands/keywords to BrowserTest’s, the main ones are listed below.

number of repeater Counts the number of elements created by the ‘repeater’ (e.g. ‘cat in cats’).
value of columnName in row where selectColumn is selectValue of repeater Retrieves the value of the column with the ‘columnName’ header in the first row where the column with header ‘selectColumn’ is ‘selectValue’ of a table generated by ‘repeater’.
value of columnName in row number rowIndex of repeater Retrieves the value of the column with the specified header in the specified row of a table generated by ‘repeater’.
value of column number columnIndex in row number rowIndex of repeater Retrieves the value of the specified column and row of a table generated by ‘repeater’.
wait for Angular requests to finish Explicitly wait for Angular to finish rendering the page (should not be needed normally, as the other commands ensure this is done, unless ‘set implicit wait for angular to’ has been called with ‘false’).
set Angular root element Defines the root element containing the Angular application.

HttpTest

HttpTest defines a language to interact with a server via HTTP requests (allowing SOAP (and REST) services to be tested). The XMLHttpTest subclass adds functionality to perform checks using XPath expressions on the response received. The JsonHttpTest subclass adds functionality to perform checks using JsonPath expressions.

Language

The main commands/keywords (i.e. public methods) offered by XmlHttpTest are listed below.

get from url Sends a GET to the specified url (parameters can be added using ‘set value(s) for’).
post body to url Sends a POST containing the specified body to the url.
post template to url Sends a POST to the specified url, determining the body by combining the configured template and values (see ‘set value for’ and ‘template’)
template location Configures the Freemarker template to use on next call using ‘post template to’.
set value value for name Stores the value for either a query or template parameter with the supplied name.
set values values for name Stores the a list of values (comma separated) for either a query or template parameter with the supplied name.
post values to url Sends a POST containing all (url-encoded) values to the url.
set value value for header name Stores the value for an HTTP header to add to subsequent requests.
put body to url Sends a PUT containing the specified body to the url.
put template to url Sends a PUT to the specified url, determining the body by combining the configured template and values (see ‘set value for’ and ‘template’)
delete url Sends a DELETE to the specified url (parameters can be added using ‘set value(s) for’).
clear values Clears the list of name/value pairs to be added to GET requests or template evaluation.
clear header values Clears the list of HTTP headers to be sent with requests.
request Returns the last url (including parameters) used for ‘get from’ or the body used for ‘post (template) to’. Intended to be used in combination with the ‘show’ command.
response Returns the last response body returned. Intended to be used in combination with the ‘show’ or ‘check’ commands.
html response Returns the last response body returned as HTML to be embedded in the test result table (in combination with the ‘show’ command).
response status HTTP status code of last response received.
response time Time (in milliseconds) it took to send the last request and receive its response.
response headers HTTP headers received with last response.
response is valid Whether the last request was successful, or an error.
set store cookies true/false Whether cookies received, which are not expired, should be stored (defaults to ‘false’) and sent with subsequent requests.
cookie values Names and values of all cookies stored.
cookie name value Value of stored cookie with name ‘name’.
cookie name domain Domain of stored cookie with name ‘name’.
cookie name path Path of stored cookie with name ‘name’.
cookie name is persistent Whether stored cookie with name ‘name’ is persistent.
cookie name is secure Whether stored cookie with name ‘name’ requires a secure connection (i.e. should only be sent over secured connections).
cookie name is http only Whether stored cookie with ‘name’ is http-only (not accessible to Javascript).
cookie name attribute attribute Value of attribute ‘attribute’ of stored cookie with name ‘name’.
clear cookies Clears the stored cookies.
copy browser cookies Adds all current Selenium (i.e. BrowserTest’s) cookies to the stored cookies. This will also ensure these will be sent with subsequent requests and all received cookies are stored.
set content type content-type Sets the content type to use when posting.
repeat until response status is value If the last response status is not ‘value’: repeats last request until it is.
repeat until response is value If the last response is not ‘value’: repeats last request until it is.
repeat until header name is value If HTTP header ‘name’ for the last response is not ‘value’: repeats last request until it does.
repeat count Returns the number of requests sent by the last ‘repeat until …’ command.
time spent repeating Returns the number of milliseconds the last ‘repeat until …’ command took.
repeat at most count times Sets the maximum number of requests sent by ‘repeat until …’ commands. The commands will return false if this number is reached without the condition being met.
repeat at most times Returns the maximum number of requests sent by ‘repeat until …’ commands.
set repeat interval to interval Sets the number of milliseconds between requests sent by ‘repeat until …’ commands.
repeat interval Returns the number of milliseconds between requests sent by ‘repeat until …’ commands.

JsonHttpTest

The commands/keywords added, to those of HttpTest, by JsonHttpTest are listed below.

post values as json to url Sends a POST containing all values (as JSON properties) to the url.
json path expression Evaluates the (definite) expression against the last response received.
json path count expression Evaluates the (indefinite) expression against the last response received, counting the number of results.
all json path matches expression Evaluates the expression against the last response received and returns all results. Intended to be used with ‘show’ command.
repeat until json path expression is value If ‘expression’ does not evaluates to ‘value’ for the last response: repeats last request until it does.

XmlHttpTest

The commands/keywords added, to those of HttpTest, by XmlHttpTest are listed below.

xPath expression Evaluates the expression against the last response received.
all XPath matches expression Evaluates the expression against the last response received and returns all results. Intended to be used with ‘show’ command.
register prefix prefix for namespace url Defines a prefix to be used to identify a namespace in subsequent ‘xPath’ evaluation.
xPath int expression Evaluates the expression against the last response received, returning a whole number (allowing relative value tests).
xPath double expression Evaluates the expression against the last response received, returning a floating point number (allowing relative value tests).
raw xPath expression Evaluates the expression against the last response received, ignoring any (SOAP) faults received (which cause ‘xPath expression’ to throw an exception).
create file fileName from base64 content of expression Evaluates the expression against the last response received, expecting the found element to contain a base64 encoded value. That value will be saved to a file and a link to the file returned.
repeat until xPath expression is value If ‘expression’ does not evaluates to ‘value’ for the last response: repeats last request until it does.