Syntax for creating JBehave stories
Tests are structured in story files, each file having one or more scenarios. Story files should each have a unique file name, ideally describing the context they're testing.
Test steps within a scenario follow the BDD naming approach using simple phrases.
Steps allow setting up of pipeline variables establishing mocks, invoking services, validating mock invocation and verifying pipeline contents. Tests are executed with Junit which runs all of the stories, with a green tick shown for each successful story step and overall green bar if all test scenarios pass
Each sequential step is a phrase based on:
- Given - This is the set up of the test where there can be one or more steps
- When - A single step to invoke the service you want to test
- Then - Multiple steps to verify that the Given and Invoke have the expected returns.
Taking a standard service, string concatenation, this service can be tested:
Scenario: Concatenate two strings
Given pipeline values inString1 = "hello "; inString2 = "world"
When invoke pub.string:concat without idata
Then pipeline has value == "hello world"
The Given is setting up two pipeline values which correspond to the inputs required by the service. The When is invoking a service without passing any additional values to the pipeline. Values are passed into services in the form of IData files. The Then performs a test on the pipeline looking for value to be the concatenated result. Running this test, it should pass with a green bar and green ticks against all the steps.
If the test fails then the unit test will fail, red bar and the step in error highlighted with a cross. It also shows what the problem could be in the Failure Trace. In the example below, the Then has failed when testing the string match:
The word And can be used as a synonym in place of Then to make the scenario more readable:
Scenario:
Given pipeline values inString1 = "hello "; inString2 = "world"
When invoke pub.string:concat without idata
Then pipeline has value == "hello world"
And pipeline has inString1 == "hello "
Within a Then, expected outcome is described as an expression to test the pipeline that something has happened. The expression is based on the JEXL syntax with full details of how to use it on the Pipeline Expressions - Using Jexl page
The scenarios rely on the setting up of data into the pipeline then testing that the pipeline has values. To do this there are two options: Direct set up using a Given pipeline values phrase or supplying data from a file.
A phrase exists that allows one or more simple values such as strings numbers, etc to be set in to the pipeline before the service is called. Using the phrase Given pipeline values followed by one or more name = value assignments sets those into the pipeline. Where there is more than one required, separate each on the same line with a semicolon followed by a space, e.g Given pipeline values inString1 = "hello"; inString2 = " world"
This method of assignment is not suitable for setting long strings of complex XML, lots of values, etc and where that is required, use an IData file instead.
The internal pipeline format of webMethods is known as IData and is a key-value map hierarchy of the pipeline and its documents. You can see this format if using the pub.flow:savePipeline to capture pipeline contents during a service execution. The format is well defined and can be easily edited with an XML tool. Using savePipeline will give examples and if you want to capture data without modifying any flow, the RESTful pipeline capture functionality in wmaop can do that.
Once you have IData files they can be used to set the pipeline contents when the service under test is invoked with the phrase When invoke xxx with yyy
This phrase will load the IData from the yyy specified file and set it in the pipeline before the service xxx is called. An example might be
When invoke org.wmaop.test.services:rootSvc with data/lorem.xml
This is loading the file lorem.xml from the data directory within the project (See Test project set up and options for more information on how the files are located and loaded) then the rootSvc is invoked.
When testing multiple conditions with many varied inputs, its sometimes required to alter some input parameters to a service. Creating lots of IData files where one value has changed is tedious and verbose. It is possible to create one IData file and have values overridden by using the Given pipeline values step in conjunction with the When invoke .. with .. step. An example of this is where we have a simple IData content:
<IDataXMLCoder version="1.0">
<record javaclass="com.wm.data.ISMemDataImpl">
<value name="lorem">ipsum</value>
<value name="dolor">sit</value>
</record>
</IDataXMLCoder>
And we invoke a service but first override lorem to have a different value. The test is looking for lorem being amet and not the ipsum value from the IData file
Given pipeline values lorem = "amet"
When invoke org.wmaop.test.services:rootSvc with data/lorem.xml
Then pipeline has lorem == "amet"
A story always has a When invoke line which indicates which service to call. There is a choice of whether to call the service and supply some IData to set up the pipeline or just to call the service with an empty pipeline. Using the 'with ...' suffix which indicates a file containg data in the IData format (The same XML format as that when you capture a pipeline). File references are relative from the src/test/resources directory so, for example, specifying data/pubresponse.xml as the name would pull the file from src/test/resources/data/pubresponse.xml
Alternatively the service can be invoked with no inputs. This is useful if simple pipeline data is already set with the Given pipeline values or no pipeline content is required.
Special attention must be made to the service and whether caching is enabled. Any service specified in the When invoke line that has caching enabled will still return cached values and therefore give unexpected values, particularly if you're trying to mock services. You may have to switch off the caching (Not recommended) or alternatively refactor your service to separate the caching from the service behaviour (recommended). Note that cached services elsewhere in the call stack (Ie called from within the 'invoke' service, etc) will always respond correctly to mocking and should need changing.]
Injecting data into your service and testing the output form the basis of the tests but an important part is eliminating service dependencies so that the test and the package it is deployed in becomes independent. Mocking out dependant services allows you to control what's returned so that the service under test is exercised without having to set up other systems or complex data flows.
Using Given mock, a service can be replaced with a mock such that it returns some pipeline values instead of calling the real service. There are two alternatives to this approach, always returning data or conditionally returning it.
When a fixed response is required where the same data is always supplied, the syntax looks like:
Given mock org.wmaop.test:aMockedService always returning data/mydata.xml where the service and the IData files are specified.
In some cases, the data needs to be varied, depending on the pipeline or presented a particular condition applies. The syntax in this case is Given mock org.wmaop.test:aMockedService returning data/option-a.xml when selection == "a" This mock will only fire when selection is set to 'a' otherwise the real service is invoked. This allows for a service to be overridden only when needed
Taking the conditional approach one step further by overloading the mock definition, the data can be varied depending on the pipeline content and by using an always returning mock as the last definition, a default can be provided:
Given mock org.wmaop.test.services:mockedOption returning data/option-a.xml when selection == "a"
Given mock org.wmaop.test.services:mockedOption returning data/option-b.xml when selection == "b"
Given mock org.wmaop.test.services:mockedOption always returning data/option-c.xml
When invoke org.wmaop.test.services:concatOptions without idata
Then pipeline has result == "abc"
Sometimes its necessary to return more than one value from a mock, for instance in a loop. You could define multiple Given mock statement checking for some counter but that's not easy, particularly when nothing indicates the position. Fortunately its possible to specify more than one file as a comma separated list:
Given mock org.wmaop.test.services:mockedOption always returning data/option-c.xml,option-a.xml,option-b.xml
In this example when the mock is called it first returns the option-c.xml content from the data directory. On the next call it returns the option-a.xml file, from the implicit data directory used by the first, then the option-b.xml file. If called again it loops back around to serve the option-c.xml file, etc.
If the directory is specified anywhere that becomes the default for that and subsequent files so:
Given mock org.wmaop.test.services:foo always returning data/a.xml,bar/b.xml,c.xml
Would server a.xml from data then both b.xml and c.xml from the bar directory.
Mocks are a form of assertion (See below for more details) and this allows for checking on the number of times the mock has been called. In the Then section of the scenario the syntax looks like: Then mock org.wmaop.test.services:aMock was invoked 2 times Because data is being supplied and the outcome should be known, the invoke check is for the exact number of times its been actioned.
Key to a number of statements is the use of expressions to specify the condition to be met before the mock, assertion, etc is actioned. If you haven't already, take a look at defining pipeline expressions using JEXL for more information about the syntax and options available.
In some cases, matching complex data structures with multiple Then pipeline may be overly verbose and some form of direct xml matching is required. While the idea of capturing a document and matching against the pipeline seems like the easiest way to match pipeline contents, it's not the most declarative and can lead to lazy matching - Ie, just dump and compare without thinking about or declaring what's important within the document. To this end, document matching should only be used where there's a lot of data to compare and not as the default means of testing the pipeline.
Documents can be matched with either XML or IData outputs of the document. The following two examples are the same document but in XML format:
<?xml version="1.0" encoding="UTF-8"?>
<locality>
<pcounty id="30" name="North Yorkshire">
<parea id="HG">
<ptown name="Harrogate"/>
<ptown name="Ripon"/>
<ptown name="Knaresborough"/>
</parea>
</pcounty>
<pcounty id="46" name="West Yorkshire">
<parea>
<ptown name="Bradford"/>
<ptown name="Shipley"/>
<ptown name="Skipton"/>
</parea>
</pcounty>
</locality>
And IData format as the pipleline document called 'document':
<?xml version="1.0" encoding="UTF-8"?>
<IDataXMLCoder version="1.0">
<record javaclass="com.wm.data.ISMemDataImpl">
<record name="document" javaclass="com.wm.data.BasicData">
<value name="@version">1.0</value>
<value name="@encoding">UTF-8</value>
<record name="locality" javaclass="com.wm.data.BasicData">
<array name="pcounty" type="record" depth="1">
<record javaclass="com.wm.data.BasicData">
<value name="@id">30</value>
<value name="@name">North Yorkshire</value>
<record name="parea" javaclass="com.wm.data.BasicData">
<value name="@id">HG</value>
<array name="ptown" type="record" depth="1">
<record javaclass="com.wm.data.BasicData">
<value name="@name">Harrogate</value>
</record>
<record javaclass="com.wm.data.BasicData">
<value name="@name">Ripon</value>
</record>
<record javaclass="com.wm.data.BasicData">
<value name="@name">Knaresborough</value>
</record>
</array>
</record>
</record>
<record javaclass="com.wm.data.BasicData">
<value name="@id">46</value>
<value name="@name">West Yorkshire</value>
<record name="parea" javaclass="com.wm.data.BasicData">
<array name="ptown" type="record" depth="1">
<record javaclass="com.wm.data.BasicData">
<value name="@name">Bradford</value>
</record>
<record javaclass="com.wm.data.BasicData">
<value name="@name">Shipley</value>
</record>
<record javaclass="com.wm.data.BasicData">
<value name="@name">Skipton</value>
</record>
</array>
</record>
</record>
</array>
</record>
</record>
</record>
</IDataXMLCoder>
To make a document match, the following syntax is looking for myDocument in the pipeline and is being matched against the snippet.xml: Then pipeline document myDocument matches data/snippet.xml
The data being offered for match can be either an exact match or partial. For an exact match, the example above where the document and the data are one for one in their elements and attributes.
A partial match takes the data being offered and as long as there's a corresponding element and attribute where specified it assumes a match. In this case you dont have to specify every element, just the ones of interest that are to be validated. So, for the IData shown above we could partially match using:
<locality>
<pcounty id="30">
<parea id="HG">
<ptown name="Ripon"/>
</parea>
</pcounty>
<pcounty id="46">
<parea>
<ptown name="Skipton"/>
</parea>
</pcounty>
</locality>
Note that key attributes are specified to identify the elements, such as id and that not all child elements are specified, ie only one ptown element per parea. Where more than one child element exists, say two ptown elements, then as long as both exist, in any order, the match is successful.
When using an IData file as the data to match, the same principal applies in that the data can be trimmed to match just key elements (Represented as record elements). It will also perform partial match where the IData is represented as an array of records. In this case, the match is successful if the records match within the array, regardless of order.
Firing mocks and testing pipeline values might be enough to satisfy you that the test has worked, but sometimes its useful to know that a particular service has been invoked and how many times (Eg, if the service has iterated some data, did it action the correct number?). Its also useful to know if a service has produced the expected output but it might not be possible to check the eventual pipeline contents due to values being dropped, etc.
Using Assertions allows you to 'peek' at the pipeline and check that something has happend.
Given MyAssertionName assertion before service org.wmaop.test.services:svcB always
When invoke org.wmaop.test.services:rootSvc with data/lorem.xml
Then assertion MyAssertionName was invoked 1 times
In this case, before svcB is actioned, the assertion called MyAssertion is triggered to prove that svcB was called. The Then checks for the assertion name and the number of invocations.
Just like mocks, assertions can have conditions applied to them so that the assertion only registers as invoked if the condition is met.
In the above examples, the invocation point has been before a service. By specifying after instead its possible to verify that a service has returned data or check at the end of a flow. If you specify invoke as the intercept point this will effectively replace the service with the assertion, effectively creating a blank, featureless mock.
Always ensure there is a Then assertion... statement for every Given ... assertion to verify the assertion - Simply creating an assertion without checking whether its fired wont give you a meaningful result!
Note that you only need assertions when checking flow through a service, usually before or after a particular service of interest is invoked. If you want to check that your mocks have fired, use the Then mock syntax as shown above in Did my mock fire?
The assertion (And mock) invoke count lookup uses the serviceName as a prefix rather than an exact match. This allows a combined count for all mocks or assertions within a package or a combined count where they begin with the name. For example, a serviceName of "pub.foo:ba" will give a count for "pub.foo:bar" and "pub.foo:bat". Likewise just using "pub.foo" gives a combined count for all mocks or assertion in that package. Note that the combined count is only for the mocks and assertions declared in the Given for the scenario, not the whole story (ie just that particular test)
Testing of behaviour should include negative paths where exceptional circumstances need handling and exeception handling requires execution. Instead of a service executing, its possible to raise an exception instead thereby simulating a failure. An example of the syntax is Given exception java.lang.RuntimeException thrown calling service foo:bar always The exception class name must be any java class that is Throwable and with a String constructor (Ie one where the exception message is passed in; This is the norm for most exceptions).
When testing within a loop you may want an exception to be thrown when a pipeline variable has a particular value (Eg, a piece of data from a service or a point within the loop counter). Like the mocking of a service, an exception can have a condition applied: Given exception java.lang.RuntimeException thrown calling service pub.foo:bar when loopCounter == 5
- Defining exceptions to be thrown in place of a service or before/after to interrupt flow
When an exception is raised in a test, be it from using a Given exception step or from having a service throw an exception, the exception must have a check applied to catch it and verify its existence. Without a check, the exception is treated as unexpected and the test will fail. To check for an exception use the syntax of:
Then exception com.wm.app.b2b.server.ServiceException was thrown
Its also possible to check the message and match if it contains a value, etc. After a Then exception... statement the pipeline has the message inserted as exceptionMessage which allows for a regular pipeline check such as:
And pipeline has exceptionMessage.contains("Service interface name required")
A variety test options can be seen in the stories used to test the framework. Browse through https://github.com/wmaop/OrgWmAOPTest/tree/master/src/test/resources/stories for examples of how to use the framework. Its also possible to deploy the OrgWmAOPTest package and run all of the tests if you want to experiment with what's there.
Each line in the story is made up from a step statement based on set phrases. The phrases are shown below in the table. Each step must appear on its own line so ignore the word wrapping in the table due to formatting limitations within the wiki space.
- Parameter place holders are shown as
$...$ and should be replaced with actual values-
$jexVariableExpression$ one or more expressions to set a pipeline value in the form of name = value such as myVar = "hello" Where there is the need to set more than one variable, separated with a semicolon, e.g: a = 1; b = "two" -
$serviceName$ The fully qualified webMethods namespace of a service, eg: org.wmaop.pub.connectivity:doSomething -
$idataFile$ A file containing data in the IData format (The same XML format as that when you capture a pipeline) File references to IData are relative from the src/test/resources directory so, for example, specifying data/pubresponse.xml as the name would pull the file from src/test/resources/data/pubresponse.xml -
$jexlPipelinExpression$ An expression in JEXL format that tests the contents of the pipeline and returns a boolean value of true to indicate the expression has passed its test. See the page detailing JEXL Expressions for more information about the format. -
$assertionId$ A name used to identify an assertion. This name is used to define and refer to the assertion and will be displayed at test failure. Should be something meaningful to the test, unique, camelcase for easy reading and be alphanumeric with no spaces, such as BeforeSendInvoice or hasExpiredPolicy -
$interceptPoint$ The point at which the mock or assertion will be triggered. Values can be either: before to be triggered before a service; after to trigger after a service; or invoke to be triggered in-place of a service. -
$exception$ The fully qualified Java class name of an exception that needs to be raised or tested for, such as java.lang.RuntimeExcpetion
-
| Given | Purpose | Description |
|---|---|---|
| Given pipeline values $jexlVariableExpression$ | To set up pipeline variables prior to a test execution | Sometimes setting up complete IData structure might seem overkill, such as the simple test above. This step allows the declaration of one or more simple variables to be inserted into the pipeline. It is a JEXL statement expression so JEXL syntax applies. Where more than one variable is defined, each must be separated by a semi-colon. E.g: Given pipeline values inString1 = "hello "; inString2 = "world" |
| Given mock $serviceName$ always returning $idataFile$ | To create a mock that always returns the same IData content | Replace serviceName$ with the name of the service and specify |
| Given mock $serviceName$ returning $idataFile$ when $jexlPipelineExpression$ | Returns content only when the pipeline matches the JEXL expression | This step allows content to be conditionally returned if the $jexlPipelineExpression$ expression matches the content of the pipeline that is submitted by the invoke. When matching, the contents of the |
| Given $assertionId$ assertion $interceptPoint$ service $serviceName$ always | Creates an assertion to unconditionally count the number of invocations | For the $interceptPoint$ specify either before or after E.g: Given EnrichAssertion assertion before service org.wmaop.pub.functinality:doSomething always |
| Given $assertionId$ assertion $interceptPoint$ service $serviceName$ when $jexlPipelineExpression$ | Defines an assertion to occur on a condition | This is useful for testing that the pipeline had a specific value before or after a particular service was called. For the $interceptPoint$ specify either before or after |
| Given exception $exception$ thrown calling service $serviceName$ always | Raise an exception in place of a service | Always throw an exception whenever the service is called. Can be used to interrupt service flow |
| Given exception $exception$ thrown calling service $serviceName$ when $jexlPipelineExpression$ | Raise and exception on a condition | This can be useful to raise an exception in loops or ensuring that the fault condition has been met by interrogating the pipeline. The Then exception step can be used to test this has occurred. Given exception java.lang.RuntimeException thrown calling service org.wmap.pub.invoice:createInvoice when lineCount = 2 |
| Given exception $exception$ thrown $interceptPoint$ calling service $serviceName$ when $jexlPipelineExpression$ | Raise an exception before or after a service when a condition occurs | |
| Given exception $exception$ thrown $interceptPoint$ calling service $serviceName$ always | always raise an exception before or after a service |
| When | Purpose | Description |
|---|---|---|
| When invoke $serviceName$ with $idataFile$ | Invoke a service with data | Used to invoke a service and pass in data to the pipeline. Normally used to comply with the service inputs and supply data to initiate the test condition |
| When invoke $serviceName$ without idata | Invoke a service without any input | A straight invoke without using data from a file. Can be useful if simple pipeline data is already set with the Given pipeline values and no further pipeline content is required |
| Then | Purpose | Description |
|---|---|---|
| Then assertion $assertionId$ was invoked $invokeCount$ times | Verification that an assertion was executed. | Used to check that an assertion set up with a Given xxx assertion has been called a number of times. This helps to verify that a loop or iteration through some data for instance has been executed correctly |
| Then mock $serviceName$ was invoked $invokeCount$ times | Verify mock was called | Useful to check that a mock was invoked a number of times. Only works with mocks, not normal services. The lookup uses the |
| Then pipeline has $jexlPipelineExpression$ | Verify pipeline contents | |
| Then exception $exception$ was thrown | Check exception occurred | Verifies that the service exited having thrown the exception specified |
| Then pipeline document $document$ matches $idataOrXmlFile$ | Match XML or IData against a pipeline document | Locates the documented named $document$ and verify its match against the supplied $idataOrXmlFile$ data file. The data file can be IData or XML and either full or a snippet of data with the correct element hierarchy requried |
| Others | Purpose | Description |
|---|---|---|
| Then show pipeline in console | See the state of the returned pipeline | When debugging failing tests, using this step will display in the console the values returned from the service called in the When. Remove this step once the test passes otherwise it causes unnecessary output during test runs. Ensure you have a log4j properties file, as noted in the creating a test project page, and it is set to output on INFO otherwise the pipeline will not be visible. |