BDD framework for iOS development.
Switch branches/tags
Nothing to show
Pull request Compare This branch is 27 commits ahead of mobilehub:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
App with tests
Test Navigation App


(simple) Simon BDD framework

Simon is a BDD framework for testing apps on iOS simulators and devices. I’ve written it to be as simple to use and easy to add to your projects as possible. In fact, it’s so simple that all you need to do is to write a story file, include a single line macro to run Simon, and then include another single line macro on each step implementation.

Why use a BDD framework?

Customes don’t understand code. So showing then a Unit test run or raw code means little to them. That’s ok, that’s why we write the code and they don’t. However, customers do understand their own requirements and they usually can express those requirements quite well.

Behaviour Driven Development is an agile technique where we get the customer to express their requirements in simple story like terns. Then using a framework like Simon, we match executable code to the stories, thus proving that they are implemented. By doing this we can prove that the app meets the customers expectations.

Why did I write Simon?

There are other BDD style frameworks for iOS. But after looking at them, they each had something I wasn’t that fond of. Everyone required me to modify my Mac for accessibility. Some placed the code in the same files as the steps. And other things. So I sat down and thought about what I wanted as a developer. I came up with this list:

  • I wanted to use Objective C because that’s what every iOS developer already knows.
  • I wanted it to be easy to include and run without requiring modifications to the system.
  • I wanted to use as little as possible “Glue” code to relate story steps to Objective C methods.
  • I wanted the stories to be stored in seperate files from the code so they would be easier to modify and update and the customer could read them.
  • I wanted it to run on the simulator.
  • I wanted to be easily able to locate and tap controls on a UI.
  • I wanted it – most of all – to be dead simple to run.

Where did the name come from?

When I was thinking about keeping things simple, I thought of the old nursery rhyme Simple Simon and the name stuck.

Ok how does it really work?

Here’s the basic sequence:

  1. The Customer and Developer work together to write stories in the form “As/Given …. then ….” For example “Given Simon is running Then I expect it to execute the stories”.
  2. You create a new target which runs your app as though you where going to manually test it.
  3. You add the text files containing the stories to your code base and the new target.
  4. You add the Simon framework to this target.
  5. You then open the startup code and add 1 line to get Simon launched in the background.
  6. You write one or more classes containing the code that executed each step in the stories.
  7. You add one line macros to the classes for each step, mapping the story step to a method.
  8. You lanuch the target on your simulator or device. Simon will automatically startup, read the story files, match the steps to the methods and run each story. At the end Simon will produce a report detailing successful executions of stories or failures.

How do I install it?

First there are two other frameworks you need to download. They are:

Download the DMG files for each. Open each one and drag and drop the framework files to a directory. Open XCode again, select your target and go to Build Phases and Link Binary With Libraries. Then click [+] to add library. Select Add Other… and then the framework directory you just copied. Do the same for the other frameworks.

Adding Simon to your project

Simon is designed to run on a background thread in your app. To save you from having the headache of working this out, there is a simple macro you can use to install and run stories. This macros:

  1. Sets up Simon on a background thread.
  2. Waits for your app to finish loading by watching the UIApplication notifications.
  3. Once your app is up, starts running the stories.

You place this macro into the main.m file of your app. Here’s a simplified example so show you what it looks like:

#import <UIKit/UIKit.h>

#import "SISimon.h"

int main(int argc, char *argv[]) {
   NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];

   // Add Simon.

   int retVal = UIApplicationMain(argc, argv, nil, @"YourAppDelegate");
   [pool release];
   return retVal;

There are two versions of the macros

  • SIRun() which searches the app for story files (*.stories) and runs them all.
  • SIRunFile(filename-without-extension) which runs stories from the specified file only.

How to I write a story?

Stories are written in plan text files with a “stories” extension Here’s an example:


Story: Example story showing the basic syntax.
Given Simon is working
Then I should be able to see abc and 5 in the log
and say goodby at the end.

Thats pretty simple. The Story: … line gives the story a name. The rest of the lines outline the story and what is expected. Simon does impose some rules about the first word on each line. Ultimately it doesn’t matter, but by imposing some simple rules, it helps to better understand the stories. Yes you can have more than one per file. Here’s an outline f the syntax rules:

  • Each story must start with a line that begins with the word story or story:.
  • The structure of the story lines must conform to:
    • As … (optional)
    • Given …
    • And … (zero or more)
    • Then … (zero or more)
    • And … (zero or more)
  • Leading and trail blanks are ignored on each line.
  • Lines that begin with # are treated as comments and ignored.
  • Blank lines are ignored.

How do I relate a story to the executable code?

Firstly Simon regards each line in a story as a seperate item to be executed. So lets look at the Given Simon is working line from the above story. Lets say we have a class called StepImplementations and we want to map the step to a method in it. Here’s what it would look like:

#import <SISimon/SISimon.h>

@interface StepImplementation : NSObject {

@implmentation StepImplementation

   SIMapStepToSelector(@"Given Simon is working", simonIsRunning) 
   -(void) simonIsRunning {
      NSLog(@"Woo hoo, Simon is running");


The first thing you wil notice is the inclusion of the SISimon.h header file. A necessary evil unfortunately. The second thing you will notice is the

SIMapStepToSelector(@"Given Simon is working", simonIsRunning)

line. That’s Simon. This preprocessor directive instructs Simon to map the given step to the selector.

Now it’s not obvious from the above, but that first parameter is a regular expression as specified in the doc for the NSRegularExpression class in Apple’s documentation. This means that we can use it to pick up a wide variety of steps and also (using regular expression groups) pick out arguments to pass to the selector. Oh, and the second argument is the selector to execute. You don’t need to use objective C’s selector(...) with this as Simon needs it as text. Plus it’s less to type in :-)

How do I map arguments in a step?

Lets look at the Then … step

Then I should be able to see abc and 5 in the log

As you can see there are two possible arguments in this step – abc and 5. Simon can map these quite simply. The only rule is that the order of arguments in the method that will be executed matches the order of arguments in the step. Of course Simon also expects that the data types are a match as well. Here’s the mapping:

SIMapStepToSelector(@".* I should be able to see (.*) and (.*) in the log", verifyText:andNumber:)

-(void) verifyText:(NSString *) text andNumber:(int) number {
   NSLog(@"Text: %@", text);
   NSLog(@"Number: %i", number);

Simon will automatically scan the data types of the arguments and attempt a matching conversion. Also notice that in the regular expression I set the start to match any characters. This means that, for example, I could use this same expression with both a Then … and a And … step.

Note that each step in a story can matched to only one selector. But that a selector can be mapped to any number of story steps. This gives you the ability to share code across many steps, saving you from repeating yourself. Also know as the DRY principle.

But wait, what if the step implementations are in different classes?

This is where Simon gets a bit smarter than the original Simon.

When steps are in the same class

Simon instantiates each class that contains mapped steps when it needs to run those steps. In addition it also caches and reuses these objects, so transferring data between story step mappings in the same class is as simple as using class properties.

When they are in different classes

But when dealing with step mappings in different classes, Simon has to be a little smarter. Each story has a local storage area your code can use to transfer objects between classes without these classes even being aware of each other. Accessing this storage is done through two macros. Here is an example:


Story: Should be able to pass objects between step classes.
Given this class stores abc in the story storage using key def
then this class should be able to retrieve abc from storage with key def


@implementation CommsASteps

   SIMapStepToSelector(@"Given this class stores (.*) in the story storage using key (.*)", storesString:withKey:)

   -(void) storesString:(NSString *) aString withKey:(NSString *) key{
      SIStoreInStory(key, aString);

CommsAStep reads a key and a value from a step and then stores the value in the story cache so that other objects can access it.


@implementation CommsBSteps

   SIMapStepToSelector(@"then this class should be able to retrieve (.*) from storage with key (.*)", retrieveString:withKey:)

   -(void) retrieveString:(NSString *) aString withKey:(NSString *) key{
      NSString *value = SIRetrieveFromStory(key);
      GHAssertEqualStrings(value, aString, @"Strings do not match");


CommsBSteps then reads that value back from the cache and verifies it.

Just in case you are wondering

After each story has finished, Simon clears out the story’s caches – releasing the step mapping implementations. So you cannot transfer data between stories. That may come later.

How do I interact with my app?

In addition to the simple steps above for creating stories, Simon also includes a framework for accessing the components that make up a UI in an app. Simon does this by mapping out the tree structure of a UI and allowing you to use dNodi’s xpath like queries to drill down through the interface, locate the controls you want, and perform various actions upon them. Tapping for example. Simon implements the dNodi DNNode protocol via a category on UIView. This keeps you from having to work through the mapping of dNodie expressions to the UI object graph.

A lot of this code is wrapped by preprocessor macros which can be found in the SISimon.h file.

Viewing the UI tree

A good way to understand this is to include this code in a step. It will print out the current tree structure showing each UIView’s class and index number in the hirachy. Some classes are further suplimented with attributes which Simon has identified as searchable ones.

SIMapStepToSelector(@"then print the UI tree", printTheUITree)
-(void) printTheUITree {

Getting a reference to a control on the UI

There are two other useful preprocessor macros. One which provides a reference to a single control using a dNodi path to locate it, and the other returning a list of zero or more controls from a path. They look like this:

SIMapStepToSelector(@"then get the button", getTheButton)
-(void) getTheButton {
	NSError *error = nil;
	UIButton *aButton = (UIButton *) SIFindView(@"/UIView/UIRoundedRectButton", &error);
	if (aButton == nil) {
		// Error!
SIMapStepToSelector(@"then get all the controls", getAllControls)
-(void) getAllControls {
	NSError *error = nil;
	NSArray *controls = SIFindViews(@"/UIView", &error);
	if (controls == nil) {
		// Error!

Tapping a control

This in an initial cut of this and I expect to expand it as I gain understanding of the best wasy to maniplulate a UI.

Tapping a control such as a button can also be accessed through a preprocessor macro which takes a path to the control as a parameter. Here is some example code:

SIMapStepToSelector(@"then tap the button", tapButton)
-(void) tapButton {
	NSError *error = nil;
	SITapControl(@"/UIView/UIRoundedRectButton", &error);
	if (error != nil) {
		// Error!

Accessing view properties using KVC

When Simon encounters a request to select based on the attribute of a UIView, it passes the attribute value to iOS’s KVC API. This means that any property of a UIView which is KVC complient can be used in a query. For example, on a UIButton [labelText.text='hello'] works just fine to find a button which has the text “hello” on it. Funny enough, frame.size does not work. Apparently CGRect is not understood by KVC although various other primitive types are.

Exception handling – Bugger!

Whilst coding for exceptions from the step implementations I came across a known Apple bug (I reported it as well as others). The bug is that NSInvocation (Which I use to call the implementation of a step) cannot handle NSExceptions properly and triggers a SIGABRT crash.

I looked into this quite a bit and ultimately was able to develope some code that worked around it, however it will not work whilst the XCode debugger (GDB) is running because GDB intercepts signals. It can be switched off, but requires some knowledge of GDB to get it working and having it working all the time would impact SIGABRTs across your program as well as Simon’s step implementations.

So the upshot is that if your code triggers a NSInvocation, it will abort Simon’s run. When Apple fixes the bug, I can change this.

Test Assert Macros

Unfortunately the assert macros in both Sen Kit and GHUnit require that they are called from within an appropriate test case class. So I’m coding up a set of macros which are basically the same, but don’t require the test code to be in any particular class. I’m also coding them so that rather than throwing NSExceptions, they pass and exception back to the step mapping via a static variable. This will help with working around the NSInvocation bug previously mentioned.

Note that with these macros, passing nil as the message results in a default message being displayed. This aleviates the need to always enter a message when most of the time, the default message serves the purpose.

Macro Description
SIFail(message, …) Indicates that the test should fail.
SIAssertTrue(expr, message, …) Fails if the expression results in NO.
SIAssertFalse(expr, message, …) Fails if the expression results in YES.

I’d like to say thanks to several developers for developing the following tools: