Skip to content

Karma plugin for running Gherkin feature files against CucumberJS in end-to-end tests.

License

Notifications You must be signed in to change notification settings

bnowack/karma-cukes

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

84 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Karma-Cukes

Karma plugin for running Gherkin feature files against CucumberJS in end-to-end tests.

Idea

  • Karma is a developer-friendly tool for running tests in real browsers, with direct access to client-side markup and code (without needing a Selenium tunnel).
  • CucumberJS enables end-to-end tests and stakeholder-friendly, living documentation.

Bringing these two together could simplify the development workflow from story to spec through a unified set of tools.

Main features of this Karma plugin

  • Comes bundled with "Progress", "Pretty", and "JSON" reporters that are more in line with the official Cucumber reporters (no HTML reporter yet, this package is more targeted at command-line users).
  • The API and syntax for registering step definitions and other support code is compatible with CucumberJS (i.e. step definition files can use module.exports).
  • Native access to the browser, jQuery, etc.
  • A single toolkit for both Story BDD and Spec BDD.
  • Tested with Chrome, Firefox, PhantomJS

Existing alternatives

Selenium/Webdriver with built-in CucumberJS framework

While Webdriver (Selenium) offers near out-of-the-box support for Cucumber, the generated reports lack the level of detail that "native" Cucumber provides. They don't allow to auto-generate living documentation from report files (e.g. feature descriptions are missing and both feature and scenario names get normalized to underscored identifiers).

Other Karma-CucumberJS adapters

They basically have the same issues as Webdriver-Cucumber. There are good jUnit-reporters but I could not find proper "Progress", "Pretty", or "JSON" formatters that behave like the ones a Cucumber (or Behat in my case) developer is used to. The ones I tested also did not work with PhantomJS and Firefox.

Solution

The two main options were:

  • writing a custom adapter + custom Cucumber-compatible reporters for Webdriver, or
  • writing a custom adapter + custom Cucumber-compatible reporters for Karma.

This package goes for the second option, which –from looking at available adapters—, seemed to be the lower-hanging fruit. I'm not so sure about this anymore after writing the plugin, but here we are ;-)

Limitations compared to Webdriver

  • Only PhantomJS offers an API for taking screenshots through Karma (AFAIK), while Webdriver provides this feature for all browsers.

  • The Browser reports are not fully identical, e.g. PhantomJS and Firefox report step locations instead of step definition locations (due a patched Error.captureStackTrace).

  • There is only a minimal Browser API for visiting URLs in end-to-end tests. The rest is native browser access and up to you (which I personally think is actually an advantage).

  • No access to response headers (except for AJAX-based requests), but could be doable via Service Workers

  • For each non-CORS-enabled target server to be used in end-to-end tests, a proxy needs to be defined in the Karma configuration file.

Installation

npm install karma-cukes --save-dev

Peer dependencies: jquery (any version), cucumber (1.*), and of course karma (any version)

Configuration (karma.conf.js)

config.set({

    // activate framework
    frameworks: ['karma-cukes'],

    // specify feature files and support code
    files: [
        { pattern: 'dev/**/*.feature', included: false, watched: true, served: true },
        { pattern: 'dev/**/hooks.js', included: true, watched: true, served: true },
        { pattern: 'dev/**/step-definitions.js', included: true, watched: true, served: true }
    ],

    // forward command line arguments to CucumberJS
    client: {
        args: process.argv.slice(4),
        captureConsole: true
    },

    // activate reporters (kc-progress, kc-pretty, and/or kc-json)
    reporters: ['kc-pretty', 'kc-json'],

    // configure the JSON formatter
    kcJsonReporter: {
        outputDir: 'dev/reports/behaviour',
        outputFile: 'karma-cukes-{shortBrowserName}.json' // supported placeholders: `shortBrowserName`, `browserName`
    },

    // enable colors in the output
    colors: true,

    // proxy test server requests for end-to-end testing
    proxies: {
        "/": "http://localhost:8889/"
    },
    
    // use a root URL for the karma runner that does not interfere with proxied sites
    urlRoot: "/__karma__/",

    ...

})

Running features

// all scenarios
$ ./node_modules/.bin/karma start path/to/karma.conf.js

// tagged scenarios
$ ./node_modules/.bin/karma start path/to/karma.conf.js -- --tags @ui

Assertion utils

Karma-Cukes comes with 3 (basic) assertion methods that are accessible from step definitions.

  • this.assert.ok(condition, callback, message)
  • this.assert.equal(actual, expected, callback, message)
  • this.assert.contain(haystack, needle, callback, message)

Using the Browser object

The World context provides a Promise-based browser interface for loading local (proxied) paths or CORS-enabled URIs:

var world = this;
this.browser
    .url('/my/path')
    .then(function () {
        var window = world.browser.window;      // native browser window (an iframe)
        var document = world.browser.document;  // native browser document object
        var $title = $(document).find('title'); // just use jQuery from here on
        var pageContent = $(document).text();
        ...
    })
    // wait for an element to appear
    .then (function() {
        var $form = $(world.browser.document).find('#my-form');
        $form
            .find('input#search').val('cucumber').end()
            .find('button[type="submit"]').click();
        ;
        return world.browser.waitFor('#search-results');
    });

AJAX requests can be issued as well:

var world = this;
this.browser
    .http('my/api', 'GET', { search: 'cucumber' })
    .then(function(response) {
        var $xhr = world.browser.$ajax;
        if ($ajax.status === 200 && $ajax.getResponseHeader('Content-Type').match(/application\/json/)) {
            return JSON.parse(response);
        }
    })

Built-in Step Definitions

Karma-Cukes provides basic step definitions to get you started:

  • I go to "$path"
    • Calls browser.url
    • Example: When I go to "/test"
  • I should see "$html" in the "$element" element
    • Calls assert.contain
    • $element can be a tag name or a CSS selector
    • Example: Then I should see "Hello" in the "body" element
  • the "$element" element should be "$expected"
    • Calls assert.equal
    • $element can be a tag name or a CSS selector
    • Example: And the "title" element should be "Hello World"

Formatters

Karma-Cukes has 3 built-in reporters:

  • "kc-progress"

    Progress Reporter Output

  • "kc-pretty"

    Pretty Reporter Output

  • "kc-json"

      [{
          "id": "test-feature",
          "uri": "path/to/test.feature",
          "name": "Test feature",
          "description": "    In order to test the library\n    As a developer\n    I can test a few dummy features",
          "line": 3,
          "keyword": "Feature",
          "tags": [
              {
                  "name": "@active",
                  "line": 2
              }
          ],
          "browser": "Chrome 51.0.2704 (Mac OS X 10.11.5)",
          "elements": [
              {
                  "id": "test-feature;open-page",
                  "name": "Open Page",
                  "line": 8,
                  "keyword": "Scenario",
                  "type": "scenario",
                  "tags": [
                      {
                          "name": "@active",
                          "line": 2
                      }
                  ],
                  "examples": [],
                  "steps": [
                      {
                          "keyword": "When ",
                          "name": "I go to \"/\"",
                          "line": 9,
                          "match": {
                              "location": "/path/to/KarmaCukesStepDefinitions.js:9"
                          },
                          "result": {
                              "status": "passed",
                              "error_message": "",
                              "duration": 276000000
                          },
                          "log": []
                      },
                      ...
                  ]
              }
              ...
          ]
      }
      ...
      ]
    

License

The MIT License (MIT) / Copyright 2016 Benjamin Nowack

About

Karma plugin for running Gherkin feature files against CucumberJS in end-to-end tests.

Resources

License

Stars

Watchers

Forks

Packages

No packages published