Testing Grails with Cucumber and Geb

Introduction

Did you ever want to test your Grails Web Application with Cucumber? And you did not have much fun using Cucumber with JRuby & cuke4duke?

Then you should try the cucumber plugin. I will introduce the plugin using a simple example that will show the setup, folder organization, step implementation and how to run the cucumber features.

the plugin

The cucumber plugin is based on Cucumber-JVM. Cucumber-JVM is the JVM implementation of cucumber with support for many JVM languages (thanks Aslak!). Using the groovy backend of cucumber-jvm we can implement the steps in groovy and run the features directly from groovy.

This makes integration of cucumber into grails a lot easier. The plugin runs cucumber inside grails which allows us to call the grails api. For example it is possible to call dynamic finders in the step implementations. You can populate the database with test data or check domain objects written to the database.

The plugin integrates cucumber as a grails test type into the test infrastructure. That means you can run the cucumber features with test-app and that the results will be included in the usual grails test reports.

Currently the plugin registers the cucumber test type only to the functional test phase. To run the cucumber features you call grails by one of the following commands:

grails test-app functional:cucumber
grails test-app :cucumber

Note that the cucumber plugin has still a few rough edges. The plugin is at an early stage and does not yet support all cucumber features. I will work on it :-) If you have any issues use the plugins issue tracker on github.

Example

We will walk through a very simple example that will use cucumber to test two scenarios against a grails web application. The two scenarios will be implemented as end-to-end tests that will remote control the application from a web browser.

The scenario steps will use Geb to remote control the web browser and to extract information from the DOM. We will see a lot of geb code used in the step implementations but we will not look at it in detail. You do not need to understand all of it to get the general idea how to use the cucumber plugin. You can learn more about geb on http://www.gebish.org. The documentation is very good.

You can also use the plugin to test your grails application below the ui using grails integration test support. You can read about this in Automating Specification with Cucumber and Grails.

To take the easy path on the views we will use grails scaffolding and we will use grails dynamic finders to check a result and we will pre-populate the database with a grails domain object.

The example is based on grails 2.0.x but it will work 1.3.7 too. The scaffolding of grails does not create the same html for 2.0 and 1.3 so we will have two versions for a small number of geb expressions.

Let's get started...

Setup

First create a new grails app:

grails create-app Books

Then we install two grails plugins. The first one is geb. As mentioned above we will use it to remote control the web browser from the the cucumber steps.

We use the usual geb setup. See the geb documentation for the installation and integration into grails. Basically it's just adding some dependencies to BuildConfig.groovy and creating a GebConfig.groovy file in the test/functional folder. A minimal version configured to use the Chrome browser looks like this:

test/functional/GebConfig.groovy:

import org.openqa.selenium.chrome.ChromeDriver

driver = {
    new ChromeDriver()
}

A longer version can be found in gebs grails example: GebConfig.groovy example.

The second plugin is the grails cucumber plugin.

After adding the plugins and gebs dependencies to BuildConfig.groovy it should look more or less like this (reduced to the interesting part):

grails.project.dependency.resolution = {
    ....
    def gebVersion = "0.7.0"
    def seleniumVersion = "2.22.0"

    dependencies {
        test ("org.codehaus.geb:geb-junit4:$gebVersion")
        test ("org.seleniumhq.selenium:selenium-chrome-driver:$seleniumVersion")
    }

    plugins {
        ....
        test ":geb:$gebVersion"
        test ":cucumber:0.5.0"
    }
}

Folder layout

To keep the geb and cucumber stuff seperate we will place them in seperate folders. Cucumber will depend on geb but geb itself will know nothing about cucumber so it makes sense to keep the files in seperate folders. If we already have existing geb tests, they work as usual, independent of cucumber.

The geb files will got to

test/functional/GebConfig.groovy
test/functional/pages/..
test/functional/modules/..

(pages and modules will be used later to organize our geb sources) and the cucumber files to:

test/cucumber/steps/..
test/cucumber/support/..
test/cucumber/..

(note that cucumber and geb do not require a specific folder layout. We could do without the sub folders or we could use different folder to organize our files)

The cucumber plugin is a functional test plugin by itself and by default expects the files in test/functional too, so we have to tell it to use test/cucumber. To do this we create the configuration file grails-app/conf/CucumberConfig.groovy with the following content:

cucumber {
    features = ["test/cucumber"]
    glue = features
}

features configures the path were cucumber will look for the features and glue configures the path were it will look for step and hook code. For this example we will simply put everything in test/cucumber.

We don't have any tests yet but by running grails test-app we can check that the plugins are installed without error.

Seeing a "no feature found" error is ok here. It tells us that the plugin runs and doesn't find any features, which of course is correct because we did not yet create one.

| Error Error executing script TestApp: cucumber.runtime.CucumberException: No features found at [test/cucumber]

Specification

Now we are ready to create our specification as cucumber features.

Our creative ;-) example is to keep track of our IT books. For this example we just have two simple requirements:

creating new book entries
showing existing book entries

To keep it simple we only have a single scenario for each features:

NewBook.feature:

Feature: new book entry
    As a book owner
    I want to add books I own to the book tracker
    so that I do not have to remember them by myself

Scenario: new book
   Given I open the book tracker
    When I add "Specification by Example"
    Then I see "Specification by Example"s details

ListBooks.feature:

@ignore
Feature: list owned books
    As a book owner
    I want to list my books
    so that I can look up which books I own

Scenario: list existing books
   Given I have already added "Specification by Example"
     And I open the book tracker
    Then my book list contains "Specification by Example"

We store the features as

test/cucumber/NewBook.feature and test/cucumber/ListBooks.feature.

We will implement feature by feature, and to get the second feature out of the way for now we add an @ignore tag to the feature and tell cucumber to ignore it by adding a tags setting to the cucumber configuration file we have used above:

cucumber {
    tags = ["~@ignore"]

    features = ["test/functional"]
    glue = features
}

Implementing NewBook.feature

Let's run the tests for the first time using the following command:

grails test-app functional:cucumber

You should see an output similar to this:

| Server running. Browse to http://localhost:8080/Books
| Running 1 cucumber tests...

You can implement missing steps with the snippets below:

Given(~'^I open the book tracker$') { ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

When(~'^I add "([^"]*)"$') { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

Then(~'^I see "([^"]*)"s details$') { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
| Completed 1 cucumber tests, 1 failed in 379ms
| Server stopped
| Tests FAILED  - view reports in target/test-reports

The cli output of grails is only a "simplified" output. If you like to see the usual cucumber output, take a look into target/test-reports/plain. (The test report does not lists the failure in this "nothing there yet" situation. It will when we have failing steps.)

Next, let us start implementing the steps. Create a new file test/cucumber/steps/Book_Steps.groovy (you can name it differently if you like, it just hast to be below test/cucumber) and copy the snippets for our first feature from the test output to the new file.

The next thing is to add the mixin for the gherkin keywords. Using import static will work too and has the advantage that the IDE will "recognize" cucumbers functions (in IDEA it is no longer underlined).

Finally it should look like this (I have added the assertions to make sure we get a test failure when running grails test-app):

test/cucumber/steps/Book_Steps.groovy:

import static cucumber.api.groovy.EN.*

Given (~'^I open the book tracker$') { ->
    assert false
}

When (~'^I add "([^"]*)"$') { String bookTitle ->
    assert false
}

Then (~'^I see "([^"]*)"s details$') { String bookTitle ->
    assert false
}

Running grails test-app functional:cucumber will obviously still fail but it now fails because of the "assertions" in the step implementation.

Before we finally start implementing the first step we have to create the link between cucumber and geb.

Create a new file test/cucumber/support/env.groovy:

import geb.binding.BindingUpdater
import geb.Browser

import static cucumber.api.groovy.Hooks.*

Before () {
    bindingUpdater = new BindingUpdater (binding, new Browser ())
    bindingUpdater.initialize ()
}

After () {
    bindingUpdater.remove ()
}

The Before() and After() hooks set up and clean up gebs environment (like the to and at methods we will see in a few seconds) from our steps binding. This code is a minimally modified version of the same file in gebs cuke4duke example.

Given(~'^I open the book tracker$')

The first step, finally! :)

Cucumber adds another level of abstraction and to make it as easy as possible to maintain we will try to minimize the code used to implement the steps and make the step layer as thin as possible.

Here is the implementation of the first step:

import pages.book.ListPage

Given(~'^I open the book tracker$') {
    to ListPage
    at ListPage
}

Hopefully easy to understand: navigate to the ListPage and check that we are at it. This is standard geb code, we just have to create the ListPage page object:

test/functional/pages/book/ListPage.groovy:

package pages.book

import geb.Page


class ListPage extends Page {
    static url = "book/list"

    static at = {
        title ==~ /Book List/
    }

    static content = {
    }
}

Let's run it:

Note: I choose chrome to run the tests because it is significantly faster than firefox when running the geb/WebDriver code (the difference is not so important for general web stuuf so I usually use Firefox for the web and Chrome for development). To run geb/WebDriver with chrome we need an additional chromedriver binary. You can put it in the path or provide its path via the command line, like below.

$ grails -Dgeb.env=chrome -Dwebdriver.chrome.driver=/Users/hauner/bin/chromedriver test-app functional:cucumber --stacktrace

| Server running. Browse to http://localhost:8080/Books
| Running 1 cucumber test...

Assertion failed: 

assert at (ListPage)
       |
       false

	at Book_Steps$_run_closure1.doCall(Book_Steps.groovy:7)
	at ✽.Given I open the book tracker(NewBook.feature:7)

| Completed 1 cucumber test, 1 failed in 10657ms
| Server stopped
| Tests FAILED  - view reports in target/test-reports

Ups, did not work. Any ideas? I know ;-) We did not implemented anything...

Let's create a minimal grails domain class and controller:

The domain class (grails-app/domain/books/Book.groovy):

package books

class Book {
    String author
    String title
}

and the controller, using scaffolding so that grails will automatically create the views for us (grails-app/controller/books/BookController.groovy):

package books

class BookController {
    def scaffold = Book
}

Next run:

$ grails -Dgeb.env=chrome -Dwebdriver.chrome.driver=/Users/hauner/bin/chromedriver test-app functional:cucumber --stacktrace

Before it fails again, you should see chrome opening and navigating to the ListPage

| Server running. Browse to http://localhost:8080/Books
| Running 1 cucumber test...

Assertion failed: 

assert false


	at Book_Steps$_run_closure2.doCall(Book_Steps.groovy:11)
	at ✽.When I add "Specification by Example"(NewBook.feature:8)

| Completed 1 cucumber test, 1 failed in 8271ms
| Server stopped
| Tests FAILED  - view reports in target/test-reports

It is still failing but we have made some progress, our first is passing and it is now the second step that is failing.

This first is passing now:

import pages.book.ListPage

Given(~'^I open the book tracker$') {
    to ListPage
    at ListPage
}

and the second step is failing:

When(~'^I add "([^"]*)"$') { String bookTitle ->
    assert false
}

When(~'^I add "([^"]*)"$')

We want to add a new book, so let's implement it to make it pass:

import pages.book.NewPage

When(~'^I add "([^"]*)"$') { String bookTitle ->
    page.toNewPage ()
    at NewPage

    page.add (bookTitle)
}

First we navigate to the New page and if we are at the correct page we add the new book.

Cucumber Note: The group selection of the steps regular expression will be passed as the element parameter to its closure. With the step: When I add "Specification by Example", this would be the book title "Specification by Example".

We will move all the search and match code on the DOM tree into the page objects. This has the advantage of keeping technical details out of the cucumber steps and second, we could easily re-use it in pure geb tests.

Another point worth mentioning is, that we will not list all properties of the book we add in the scenario description. We just use a single prominent property to identify it and hide the details in the lower level code. Here in the page.add () method.

We try to use a high abstraction level in the feature description to avoid as much detail as possible. This makes our feature description more robust against changes in the Book class. If any property is added or removed from Book we do not have to change it.

But of course, how much detail you include depends on what you want to test. ;-)

Here is the new version of the ListPage (test/functional/pages/book/ListPage.groovy):

package pages.book

import geb.Page


class ListPage extends Page {
    static url = "book/list"

    static at = {
        title ==~ /Book List/
    }

    static content = {
        create (to: NewPage) {
            $ ('a.create')
        }
    }

    def toNewPage () {
        create.click ()
    }
}

and here is the NewPage (test/functional/pages/book/NewPage.groovy):

package pages.book

import geb.Page
import data.Data


class NewPage extends Page {
    static at = {
        title ==~ /Create Book/
    }

    static content = {
        save {
            $ ('input.save')
        }
    }

    /*
    def add (String bookTitle) {
        $ ("form").title = bookTitle

        if (bookTitle == "Specification by Example") {
            $ ("form").author = "Gojko Adzic"
        }
        save.click ()
    }
    */
    
    def add (String bookTitle) {
        def book = Data.findByTitle (bookTitle)
        
        if (book.title == bookTitle) {
            $ ("form").author = book.author
        }
        $ ("form").title = bookTitle

        save.click ()
    }    
}

The add method will fill the form and submit (save.click ()) it. It is using a small helper class to look up the book properties by its title so we can easily handle other books without modifying the code. Initially the values were hardcoded. (Commented version of add()) But we will skip that small refactoring and move on with the final code ;-).

Data looks like this (test/functional/data/Data.groovy):

package data

class Data {

    static def books = [
        [title: "Specification by Example", author: "Gojko Adzic"]
    ]

    static public def findByTitle (String title) {
        books.find { book ->
            book.title == title
        }        
    }
}

Let's try if it works by running the test again:

grails -Dgeb.env=chrome -Dwebdriver.chrome.driver=/Users/hauner/bin/chromedriver test-app functional:cucumber --stacktrace

You should see the browser adding the new book, entering "Gojko Adzic" and "Specification by Example" into the form and submitting it before it fails again:

| Server running. Browse to http://localhost:8080/Books
| Running 1 cucumber test...

Assertion failed: 

assert false


	at Book_Steps$_run_closure3.doCall(Book_Steps.groovy:20)
	at ✽.Then I see "Specification by Example"s details(NewBook.feature:9)

| Completed 1 cucumber test, 1 failed in 15933ms
| Server stopped
| Tests FAILED  - view reports in target/test-reports

Progress :-)

The When step passed and we can move on to the Then step, checking the result of the add() operation.

Then(~'^I see "([^"]*)"s details$')

import page.books.ShowPage

Then(~'^I see "([^"]*)"s details$') { String bookTitle ->
    at ShowPage

    page.check (bookTitle)
}

The When step changed the page again when it submitted the form, so we check that we are at the new page and then validate that it shows the details of the created book.

The validation is where the cucumber plugin really gets interesting.

Here is the ShowPage (test/functional/pages/book/ShowPage.groovy):

package pages.book

import geb.Page
import data.Data


class ShowPage extends Page {
    static at = {
        title ==~ /Show Book/
    }

    static content = {
        row { val ->
            $ ('td.name', text: val).parent ()              // grails 1.3
            $ ('span.property-label', text: val).parent ()  // grails 2.0
        }

        value { val ->
            row (val).find ('td.value').text ()             // grails 1.3
            row (val).find ('span.property-value').text ()  // grails 2.0           
        }

        id {                                                // grails 1.3 only
            value ('Id')
        }

        btitle {
            value ('Title')
        }

        author {
            value ('Author')
        }
    }

    def check (String bookTitle) {
        def book = Data.findByTitle (bookTitle)

        assert id.number                                   // grails 1.3 only
        assert book.author == author
        assert book.title == btitle
    }
}

Running again:

-------------------------------------------------------
Running 1 cucumber test...
Running test new book entry...PASSED

Tests Completed in 15018ms ...
-------------------------------------------------------
Tests passed: 1
Tests failed: 0
-------------------------------------------------------

Great, it passed finally. :-)

Note the grails 1.3 and grails 2.0 comments in the code. The html created by the scaffolding is different in 1.3 and 2.0. You only need one version of the line, depending on our grails version.

Using GORM

Let's take a look at the page code of the ShowPage. It contains mostly geb stuff to extract values from the page. Interesting is the check () method. Let's take a few second to review it.

check () is called from the step we just made pass. It checks that the book we have entered is displayed with identical values. It uses the Data class introduced above and checks that the data shown is the data we have entered.

We could also do something like this:

def check (String bookTitle) {
    Book book = Book.findByTitle (bookTitle)

    assert book.id == id.toLong ()
    assert book.author == author
    assert book.title == btitle
}

See the difference? It does not get the book data from the Data helper class. It is using a dynamic finder to read it from the database and then compares it with the data from the page.

It is not exactly the same as the first version, which checks that the "entered" data is shown, but it shows that we can use standard grails code in the cucumber steps. This is very useful because it allows us to use normal code for setup and validation without having to run it through the gui.

I think you've got the basic idea here, so we will quickly run over the second feature that will use a little bit GORM to pre populate the database.

Implementing ListBooks.feature

First we remove the @ignored tag from the ListBooks.feature so that cucumber will run it. Here is the feature again:

ListBooks.feature (test/cucumber/ListBooks.feature):

Feature: list owned books
    As a book owner
    I want to list my books
    so that I can look up which books I own

Scenario: list existing books
   Given I have already added "Specification by Example"
     And I open the book tracker
    Then my book list contains "Specification by Example"

We want to check that our book gets listed if it was already entered.

Running test-app (again) will provide us with the basic step definitions.

Given(~'^I have already added "([^"]*)"$') { String arg1 ->
    ....
}

Then(~'^my book list contains "([^"]*)"$') { String arg1 ->
    ....
}

Given(~'^I have already added "([^"]*)"$')

We copy them to (cucumber/step_definitions/Book_Steps.groovy) and modify the first one to look like this:

Given(~'^I have already added "([^"]*)"$') { String bookTitle ->
    Data.createBookByTitle (bookTitle)
}

Next, we call another helper method on the Data class that will create the Book domain object and write it to the database:

package data

import books.Book


class Data {

    static def books = [
        [title: "Specification by Example", author: "Gojko Adzic"]
    ]

    static public def findByTitle (String title) {
        books.find { book ->
            book.title == title
        }        
    }
    
    static void createBookByTitle (String title) {
        new Book (findByTitle (title)).save ()      // create book and write it to the db
    }

    static void clearBooks () {
        Book.findAll()*.delete (flush: true)
    }
}

This version of the Data class has another new method: clearBooks (). It will delete all Book rows from the database so the next scenario will run on a clean database. The plugin is not capabable of handling rollback like functionality. We have to handle this ourself via the Beforeand After hooks.

We just have to add it to the setup (Before ()) or tear down code (After ()) in env.groovy. That is the file where we added gebs setup and tear down code.

cucumber/support/env.groovy:

import geb.binding.BindingUpdater
import geb.Browser
import data.Data

import static cucumber.api.groovy.Hooks.*

	
Before () {
    Data.clearBooks ()  // added to cleanup the database

    bindingUpdater = new BindingUpdater (binding, new Browser ())
    bindingUpdater.initialize ()
}

After () {
    bindingUpdater.remove ()
}

Running test-app (same parameters as above) again should show the Book on the list page and then fail with

assert false


	at Book_Steps$_run_closure5.doCall(Book_Steps.groovy:32)
	at ✽.Then my book list contains "Specification by Example"(ListBooks.feature:9)

The Given step passed and we can move on to the final Then step.

Then(~'^my book list contains "([^"]*)"$')

The implementation for our last step will be:

Then(~'^my book list contains "([^"]*)"$') { String bookTitle ->
    at ListPage
    
    page.checkBookAtRow (bookTitle, 0)
}

Straight forward: check that we are on the right page and check the Book in the first row of the table is the expected one.

Here is the new ListPage (test/functional/pages/book/ListPage.groovy):

package pages.book

import geb.Page
import modules.BookRow
import books.Book


class ListPage extends Page {
    static url = "book/list"

    static at = {
        title ==~ /Book List/
    }

    static content = {
        create (to: NewPage) {
            $ ('a.create')
        }

        bookTable {
            $ ("div.list table", 0)
        }

        bookRows {
            bookTable.find ('tbody').find ('tr')
        }

        row { row ->
            module (BookRow, bookRows[row])
        }
    }

    def toNewPage () {
        create.click ()
    }

    def checkBookAtRow (String bookTitle, int rowNumber) {
        def book = Book.findByTitle (bookTitle)

        assert book.id == row (rowNumber).id.toLong ()
        assert book.author == row (rowNumber).author
        assert book.title == row (rowNumber).btitle
    }
}

and here is the geb module we use to extract the row data from the table:

package modules

import geb.Module


class BookRow extends Module {
    static content = {
        cell { column ->
            $ ('td', column)
        }

        cellText { column ->
            cell (column).text ()
        }

        id {
            cellText (0)
        }

        author {
            cellText (1)
        }

        btitle {
            cellText (2)
        }
    }
}

The page adds some more geb stuff and in checkBookAtRow () it asserts that the book matches the one we have created and written to the database by getting it from the database using a dynamic finder.

If we run test-app again both scenarios pass and we are done. :-)

| Server running. Browse to http://localhost:8080/Books2
| Running 2 cucumber tests...
| Completed 2 cucumber tests, 0 failed in 59261ms
| Server stopped
| Tests PASSED - view reports in target/test-reports

Summary

We created a simple grails application, we specified it with features written in gherkin and automated them using cucumber and in the step implementations we were able to use standard grails code for setup and validation. We also used geb in the steps to run the features against the gui as end-to-end tests.

It is a simple example, but I think that it should give you enough information to get started with cucumber and grails. I hope that using geb in this example was not too distracting. :)

Thanks for reading :-)

The full source code of this example is available in the plugins github repository: Book example. The example is based on grails 2.0.x and there is also a 1.3.x based version (which, I fear, is probably not up to date).

cucumber code

At last, here is the cucumber related code put together (features and steps) to show that it is not so much code if you move all the details to helper classes in an automation layer that you can also use outside of cucumber.