Simple dialogue tests
Unit testing is one of Rivr most notable feature. It allows the developer to perform non-regression testing with automated tools. Using this technique, you can simulate hundreds of call in seconds.
Suppose we have a simple dialogue and we want to write unit tests for it:
public class Dialogue implements VoiceXmlDialogue {
@Override
public VoiceXmlLastTurn run(VoiceXmlFirstTurn firstTurn, VoiceXmlDialogueContext context)
throws Exception {
playMessage(context, "welcome", "Welcome");
String number = null;
for (int tryCount = 0; tryCount < 3 && number == null; tryCount++) {
VoiceXmlInputTurn inputTurn = askNumber(context);
if (inputTurn.getRecognitionInfo() != null) {
JsonArray recognitionResult = inputTurn.getRecognitionInfo().getRecognitionResult();
number = recognitionResult.getJsonObject(0).getString("interpretation");
playMessage(context, "feedback", "You have entered: " + number);
} else if (hasEvent(VoiceXmlEvent.NO_INPUT, inputTurn.getEvents())) {
playMessage(context, "no-input", "You haven't entered anything.");
} else if (hasEvent(VoiceXmlEvent.NO_MATCH, inputTurn.getEvents())) {
playMessage(context, "no-match", "This is not a number.");
}
}
JsonObjectBuilder resultBuilder = createObjectBuilder();
if (number == null) {
resultBuilder.add("status", "error");
} else {
resultBuilder.add("status", "success");
resultBuilder.add("number", number);
}
//end of dialogue
return new Exit("exit", VariableList.create(resultBuilder.build()));
}
private VoiceXmlInputTurn askNumber(VoiceXmlDialogueContext context)
throws Timeout, InterruptedException {
GrammarItem dtmfGrammar = new GrammarReference("builtin:dtmf/digits");
DtmfRecognition dtmfRecognition = new DtmfRecognition(dtmfGrammar);
Interaction turn = interaction("ask-number")
.addPrompt(new SpeechSynthesis("Type a number."))
.build(dtmfRecognition, Duration.seconds(5));
return doTurn(turn, context);
}
private void playMessage(VoiceXmlDialogueContext context, String name, String messageText)
throws Timeout, InterruptedException {
Message message = new Message(name, new SpeechSynthesis(messageText));
doTurn(message, context);
}
}We first need to write a unit test class in which we will initialize a VoiceXmlTestDialogueChannel which is an implementation of DialogueChannel used for testing. This implementation allows us to simulate responses from the VoiceXML platform such as recognition results and no-match events.
public class DialogueTest {
private final VoiceXmlTestDialogueChannel mChannel =
new VoiceXmlTestDialogueChannel("test", Duration.seconds(15));This dialogue channel must be set in the dialogue context we will use in our test cases:
private final VoiceXmlDialogueContext mDialogueContext =
new VoiceXmlDialogueContext(mChannel,
mChannel.getLogger(),
"testDialogueId",
"/context",
"/servlet");The rest of the class is a mix of test methods (annotated with @org.junit.Test annotation) and helper methods.
Let's take a look at our first test case:
@Test
public void testNormal() {
startDialogue();
assertWelcomeMessage();
mChannel.processNoAction();
assertPromptForNumber();
mChannel.processDtmfRecognition("1 2 3 4 5 6", JsonUtils.wrap("123456"), null);
assertMessage(new SpeechSynthesis("You have entered: 123456"));
mChannel.processNoAction();
VariableList expectedVariables = new VariableList();
expectedVariables.addWithExpression("status", "\"success\"");
expectedVariables.addWithExpression("number", "\"123456\"");
assertExit(expectedVariables);
}The first thing to do is to start the dialogue.
startDialogue();We have used the startDialogue() helper method to do so:
private void startDialogue() {
mChannel.startDialogue(new Dialogue(), new VoiceXmlFirstTurn(), mDialogueContext);
}We then need to perform some kind of check to validate that the dialogue is executing correctly. We know that the first thing the dialogue should do is to play the welcome message, so let's verify that:
assertWelcomeMessage();Since other test cases also require to perform this check, we have extracted the logic in the assertWelcomeMessage() helper method:
private void assertWelcomeMessage() {
assertMessage(new SpeechSynthesis("Welcome"));
}The assertMessage() method ensure that the dialogue is generating a Message turn and that the audio items are as expected:
private void assertMessage(AudioItem... expectedAudioItems) {
VoiceXmlOutputTurn turn = mChannel.getLastStepAsOutputTurn();
assertEquals(Message.class, turn.getClass());
Message actualTurn = (Message) turn;
assertEquals(Arrays.asList(expectedAudioItems), actualTurn.getAudioItems());
}Note that in this method, we use the getLastStepAsOutputTurn method on our test dialogue channel to query the last step that
occured in the channel. The step can be one of:
- Output turn
- Last turn
- Error
Depending on the expected type of step at this point in the dialogue, we can invoke respectively
So the assertMessage() method will ensure
- that we have, at this point in the dialogue, an
OutputTurnfor which the type isMessage - that the audio items of this message turn are as expected
Note that we are only asserting some aspects of the Message. We didn't test neither the language nor barge-in properties.
Thus it's possible to make the test cases as sophisticated as needed. There's always a trade-off between what is actually asserted in
unit tests and the effort required to develop those test cases. You have control over what you depth of assertion for each turn.
Geting back at our example, now we'll simulate the VoiceXML platform response. In this case, since we have a message, the only possible response
from the platform (beside an error event) is what we call a no-action, i.e. an input turn without any property set. In order to simulate this platform response, we invoke the
processNoAction method on the dialogue channel:
mChannel.processNoAction();The dialogue will then progress to the next turn.
assertPromptForNumber();The assertPromptForNumber() validates that we have an interaction turn, parameterized for DTMF recogition and having some specific messages:
private void assertPromptForNumber() {
assertDtmfInteraction(Duration.seconds(5),
"builtin:dtmf/digits",
new SpeechSynthesis("Type a number."));
}This method uses two additionnal helper methods assertDtmfInteraction and assertInteraction:
private void assertDtmfInteraction(Duration expectedTimeout,
String expectedGrammar,
AudioItem... expectedAudioItems) {
DtmfRecognition dtmfRecognition = new DtmfRecognition(new GrammarReference(expectedGrammar));
assertInteraction(new FinalRecognitionWindow(dtmfRecognition, expectedTimeout),
new Prompt(expectedAudioItems));
}
private void assertInteraction(FinalRecognitionWindow expectedRecognition, Prompt... expectedPrompts) {
VoiceXmlOutputTurn turn = mChannel.getLastStepAsOutputTurn();
assertEquals(Interaction.class, turn.getClass());
Interaction actualTurn = (Interaction) turn;
assertEquals(Arrays.asList(expectedPrompts), actualTurn.getPrompts());
assertEquals(expectedRecognition, actualTurn.getRecognition());
}Again, we see that we can build many reusable helper methods that will make each test case very terse.
To simulate DTMF processing, including semantic result, we do:
mChannel.processDtmfRecognition("1 2 3 4 5 6", JsonUtils.wrap("123456"), null);The first argument is the raw input (DTMF recognizer will sometimes separate digits with spaces), the second one is the semantic result, the third one is a mark info (not covered here). Next step is to validate the feedback message:
assertMessage(new SpeechSynthesis("You have entered: 123456"));At this point, you may have noticed that we have always follow this pattern:
- perform some action
- assert something
Using a white line between each block will help you remain organized in you test case method code.
To assert the result of the dialogue, we only need to assert the LastTurn:
VariableList expectedVariables = new VariableList();
expectedVariables.addWithExpression("status", "\"success\"");
expectedVariables.addWithExpression("number", "\"123456\"");
assertExit(expectedVariables);With assertExit() defined as:
private void assertExit(VariableList expectedVariables) {
VoiceXmlLastTurn turn = mChannel.getLastStepAsLastTurn();
assertEquals(Exit.class, turn.getClass());
Exit exitTurn = (Exit) turn;
VariableList actualVariables = exitTurn.getVariables();
assertEquals(expectedVariables, actualVariables);
}If we only run JUnit for the test case testNormal() we do not completely every outcome of the callflow. This can be verified by running JUnit in conjunction
with a coverage tool. I will be using Emma with the Eclipse plug-in:
Adding 3 other test cases, we can achieve complete coverage:
@Test
public void testNoInput() {
startDialogue();
assertWelcomeMessage();
mChannel.processNoAction();
assertPromptForNumber();
mChannel.processNoInput();
assertNoInputMessage();
mChannel.processNoAction();
assertPromptForNumber();
mChannel.processNoInput();
assertNoInputMessage();
mChannel.processNoAction();
assertPromptForNumber();
mChannel.processNoInput();
assertNoInputMessage();
mChannel.processNoAction();
VariableList expectedVariables = new VariableList();
expectedVariables.addWithExpression("status", "\"error\"");
assertExit(expectedVariables);
}
@Test
public void testNoMatch() {
startDialogue();
assertWelcomeMessage();
mChannel.processNoAction();
assertPromptForNumber();
mChannel.processNoMatch();
assertNoMatchMessage();
mChannel.processNoAction();
assertPromptForNumber();
mChannel.processNoMatch();
assertNoMatchMessage();
mChannel.processNoAction();
assertPromptForNumber();
mChannel.processNoMatch();
assertNoMatchMessage();
mChannel.processNoAction();
VariableList expectedVariables = new VariableList();
expectedVariables.addWithExpression("status", "\"error\"");
assertExit(expectedVariables);
}
@Test
public void testOtherEvent() {
startDialogue();
assertWelcomeMessage();
mChannel.processNoAction();
assertPromptForNumber();
mChannel.processEvent("unexpectedevent");
assertPromptForNumber();
mChannel.processEvent("unexpectedevent");
assertPromptForNumber();
mChannel.processEvent("unexpectedevent");
VariableList expectedVariables = new VariableList();
expectedVariables.addWithExpression("status", "\"error\"");
assertExit(expectedVariables);
}
You can download or browse the complete code for this example at GitHub.This is a complete working application that you can build and run for yourself.
You can also clone the Rivr Cookbook repository and checkout this example:
git clone -b junit-dialogue-test git@github.com:nuecho/rivr-cookbook.git
Then, to build and run it:
cd rivr-cookbook
./gradlew jettyRun
The VoiceXML dialogue should be available at http://localhost:8080/rivr-cookbook/dialogue
To stop the application, press Control-C in the console.