Skip to content
BDD style unit testing framework for POSIX compliant shell script
Branch: master
Clone or download

README.md

shellspec

BDD style unit testing framework for POSIX compliant shell script.

Let’s test the your shell script!

demo

Travis CI Cirrus CI Circle CI Kcov Coveralls Codecov CodeFactor GitHub top language GitHub release License

bash busybox dash ksh mksh posh yash zsh

Project status: Implementation of practical features has been completed. I will add more tests and improve the documentation.

Table of Contents

Introduction

Features

  • Support POSIX compliant shell (dash, bash, ksh, busybox, etc...)
  • Specfile is BDD style syntax with shell scripts compatible
  • Implemented by shell script with Minimal dependencies (use only a few basic POSIX compliant command)
  • Nestable block with scope like lexical scope
  • Mocking and stubbing in the scope (temporary function override)
  • The before/after hook and the skip/pending of the examples
  • Execution filtering (line number, id, focus, tag and example name)
  • Parallel execution, random ordering execution, dry-run executions
  • Modern reporting (colorize, line number, progress/documentation/TAP/JUnit formatter)
  • Coverage (kcov integration) and Profiler
  • Built-in simple task runner
  • Extensible architecture (custom matcher, custom formatter, etc...)
  • shellspec is tested by shellspec

Supported shells

dash, bash, ksh, mksh, oksh, pdksh, zsh, posh, yash, busybox (ash), bosh, pbosh

Tested Platforms (See tested shells .travis.yml, .cirrus.yml)

Platform Test
Ubuntu 12.04, 14.04, 16.04, 18.04 Travis CI
macOS 10.10, 10.11, 10.12, 10.13, 10.14, 10.14 (Homebrew) Travis CI
FreeBSD 10.x, 11.x, 12.x Cirrus CI
Windows Server 2019 (Git bash, msys2, cygwin) Cirrus CI
Windows 10 1903 (Ubuntu 18.04 on WSL) manual
Solaris 10, 11 manual

Tested version details

Requires

shellspec is implemented in a pure shell script with a shell built-in commands and few basic POSIX compliant commands. (except kcov for optionally coverage).

Currently used external (not shell built-in) commands.

  • cat, date, ls, mkdir, od, rm, sleep, sort, time
  • ps (used on systems without procfs, but not required.)
  • kill, printf (used but almost shell built-in.)

Tutorial

Installation

$ curl -fsSL https://git.io/shellspec | sh

or

$ wget -O- https://git.io/shellspec | sh
Upgrade / Switch version

Upgrade the latest release version.

$ curl -fsSL https://git.io/shellspec | sh -s -- --switch

Switch to the specified version.

$ curl -fsSL https://git.io/shellspec | sh -s 0.18.0 --switch
Advanced installation
$ curl -fsSL https://git.io/shellspec | sh -s -- --help
Usage: [sudo] ./install.sh [VERSION] [OPTIONS...]
  or : wget -O- https://git.io/shellspec | [sudo] sh
  or : wget -O- https://git.io/shellspec | [sudo] sh -s -- [OPTIONS...]
  or : wget -O- https://git.io/shellspec | [sudo] sh -s VERSION [OPTIONS...]
  or : curl -fsSL https://git.io/shellspec | [sudo] sh
  or : curl -fsSL https://git.io/shellspec | [sudo] sh -s -- [OPTIONS...]
  or : curl -fsSL https://git.io/shellspec | [sudo] sh -s VERSION [OPTIONS...]

VERSION:
  Specify install version and method

  e.g
    1.0.0           Install 1.0.0 from git
    master          Install master from git
    1.0.0.tar.gz    Install 1.0.0 from tar.gz archive
    .               Install from local directory

OPTIONS:
  -p, --prefix PREFIX   Specify prefix          [default: $HOME]
  -b, --bin BIN         Specify bin directory   [default: <PREFIX>/bin]
  -d, --dir DIR         Specify directory name  [default: .shellspec]
  -s, --switch          Switch version (requires installed via git)
  -l, --list            List available versions (tags)
      --pre             Include pre-release
      --fetch FETCH     Force command to use when install from archive (curl or wget)
  -y, --yes             Automatic yes to prompts
  -h, --help            You're looking at it
Manual installation

Just get the shellspec and create a symlink in your executable PATH!

From git

$ cd /SOME/WHERE/TO/INSTALL
$ git clone https://github.com/shellspec/shellspec.git
$ ln -s /SOME/WHERE/TO/INSTALL/shellspec/shellspec /EXECUTABLE/PATH/
# (e.g. /EXECUTABLE/PATH/ = /usr/local/bin/, $HOME/bin/)

From tar.gz

$ cd /SOME/WHERE/TO/INSTALL
$ wget https://github.com/shellspec/shellspec/archive/{VERSION}.tar.gz
$ tar xzvf shellspec-{VERSION}.tar.gz

$ ln -s /SOME/WHERE/TO/INSTALL/shellspec-{VERSION}/shellspec /EXECUTABLE/PATH/
# (e.g. /EXECUTABLE/PATH/ = /usr/local/bin/, $HOME/bin/)

If you can't create symlink (like default of Git for Windows), create the shellspec file.

$ cat<<'HERE'>/EXECUTABLE/PATH/shellspec
#!/bin/sh
exec /SOME/WHERE/TO/INSTALL/shellspec/shellspec "$@"
HERE
$ chmod +x /EXECUTABLE/PATH/shellspec
Uninstall
  1. Delete the shellspec installation directpry [default: $HOME/opt/shellspec].
  2. Delete the shellspec executable file [default: $HOME/bin/shellspec].

Getting started

Just create your project directory and run shellspec --init to setup to your project

# Create your project directory. for example "hello".
$ mkdir hello
$ cd hello

# Initialize
$ shellspec --init
  create   .shellspec
  create   spec/spec_helper.sh
  create   spec/hello_spec.sh # sample

# Write your first specfile (of course you can use your favorite editor)
$ cat<<'HERE'>spec/hello_spec.sh
Describe 'hello.sh'
  Include lib/hello.sh
  It 'says hello'
    When call hello shellspec
    The output should equal 'Hello shellspec!'
  End
End
HERE

# Create lib/hello.sh
$ mkdir lib
$ touch lib/hello.sh

# It goes fail because hello function not implemented.
$ shellspec

# Write hello function
$ cat<<'HERE'>lib/hello.sh
hello() {
  echo "Hello ${1}!"
}
HERE

# It goes success!
$ shellspec

shellspec command

Usage

Usage: shellspec [options] [files or directories]

  -s, --shell SHELL                   Specify a path of shell [default: current shell]
      --[no-]fail-fast[=COUNT]        Abort the run after a certain number of failures [default: 1]
      --[no-]fail-no-examples         Fail if no examples found [default: disabled]
      --[no-]fail-low-coverage        Fail on low coverage [default: disabled]
                                        The coverage threshold is specified by the coverage option
  -r, --require MODULE                Require a file
  -e, --env NAME=VALUE                Set environment variable
      --env-from ENV-SCRIPT           Set environment variable from script file
      --random TYPE[:SEED]            Run examples by the specified random type
                                        [none]      run in the defined order [default]
                                        [specfiles] randomize the order of specfiles
                                        [examples]  randomize the order of examples (slow)
  -j, --jobs JOBS                     Number of parallel jobs to run (0 jobs means disabled)
      --[no-]warning-as-failure       Treat warning as failure [default: enabled]
      --dry-run                       Print the formatter output without running any examples
      --keep-tempdir                  Do not cleanup temporary directory [default: disabled]

  **** Output ****

      --[no-]banner                   Show banner if exist 'spec/banner' [default: enabled]
  -f, --format FORMATTER              Choose a formatter to use for display
                                        [p]rogress      dots [default]
                                        [d]ocumentation group and example names
                                        [t]ap           TAP format
                                        [j]unit         JUnit XML
                                                        (Require --profile for time attributre)
                                        [null]          do not display anything
                                        [debug]         for developer
                                        custom formatter name
  -o, --output GENERATOR              Choose a generator(s) to generate a report file(s)
                                        You can use the same name as FORMATTER
                                        Multiple options can be specified [default: not specified]
      --force-color, --force-colour   Force the output to be in color, even if the output is not a TTY
      --no-color, --no-colour         Force the output to not be in color, even if the output is a TTY
      --skip-message VERBOSITY        Mute skip message
                                        [verbose]  do not mute any messages [default]
                                        [moderate] mute repeated messages
                                        [quiet]    mute repeated messages and non-temporarily messages
  -p  --[no-]profile                  Enable profiling of examples and list the slowest examples
      --profile-limit N               List the top N slowest examples (default: 10)

  **** Ranges / Filters ****

    You can select examples range to run by appending the line numbers or id to the filename

      shellspec path/to/a_spec.sh:10:20     # Run the groups or examples that includes lines 10 and 20
      shellspec path/to/a_spec.sh:@1-5:@1-6 # Run the 5th and 6th groups/examples defined in the 1st group

    You can filter examples to run with the following options

      --focus                         Run focused groups / examples only
                                        To focus, prepend 'f' to groups / examples in specfiles
                                        e.g. Describe -> fDescribe, It -> fIt
      --pattern PATTERN               Load files matching pattern [default: "*_spec.sh"]
      --example PATTERN               Run examples whose names include PATTERN
      --tag TAG[:VALUE]               Run examples with the specified TAG
      --default-path PATH             Set the default path where shellspec looks for examples [defualt: "spec"]

  **** Coverage ****

      --[no-]kcov                     Enable coverage using kcov [default: disabled]
                                        Note: Requires kcov and bash, parallel execution is ignored.
      --kcov-path PATH                Specify kcov path [default: kcov]
      --kcov-common-options OPTIONS   Specify kcov common options [default: see below]
                                        --path-strip-level=1
                                        --include-path=.
                                        --include-pattern=.sh
                                        --exclude-pattern=/spec/,/coverage/,/report/
      --kcov-options OPTIONS          Specify additional kcov options
                                        coverage limits, coveralls id, etc...

  **** Utility ****

      --init                          Initialize your project with shellspec
      --count                         Count the number of specfiles and examples
      --list LIST                     List the specfiles / examples
                                        [specfiles]       list the specfiles
                                        [examples]        list the examples with id
                                        [examples:id]     alias for examples
                                        [examples:lineno] list the examples with lineno
                                        [debug]           for developer
                                        affected by --random but TYPE is ignored
      --syntax-check                  Syntax check of the specfiles without running any examples
      --translate                     Output translated specfile
      --task [TASK]                   Run task. If TASK is not specified, show the task list
  -v, --version                       Display the version
  -h, --help                          You're looking at it

Configure default options

To change default options for shellspec command, create options file. Read files in the order the bellows and overrides options.

  1. $XDG_CONFIG_HOME/shellspec/options
  2. $HOME/.shellspec
  3. ./.shellspec
  4. ./.shellspec-local (Do not store in VCS such as git)

Special environment variable

Special environment variable of shellspec is starts with SHELLSPEC_. It can be overridden with custom script of --env-from option.

Todo: descriptions of many special environment variables.

Parallel execution

You can use parallel execution for fast test with --jobs option.

Random execution

You can randomize the execution order to detect troubles due to the test execution order. If SEED specified, you can execute same order.

Reporter / Generator

You can specify one reporter (output to stdout) and multiple generators (output to file). Currently builtin formatters are progress, documentation, tap, junit, null, debug.

Ranges / Filters

You can execute specified spec only. It can be specified by line number, example id, example name, tag and focues. To focus, prepend f to groups / examples in specfiles. (e.g. Describe -> fDescribe, It -> fIt)

Coverage

shellspec is integrated with coverage for ease of use (Requires to install kcov. Sample of coverage report). It works with the default settings, but you may need to adjust kcov's options to make it more accurate.

Be aware of the shell can be used for coverage is bash only.

Profiler

When specified --profile option, profiler is enabled and list the slow examples.

Task runner

You can run the task with --task option.

Project directory

Typical directory structure.

Project directory
├─ .shellspec
├─ .shellspec-local
├─ report/
├─ coverage/
│
├─ bin/
│   ├─ executable_shell_script
│              :
├─ lib/
│   ├─ your_shell_script_library.sh
│              :
├─ libexec/
│   ├─ executable_utilities
│              :
├─ spec/
│   ├─ banner
│   ├─ spec_helper.sh
│   ├─ support/
│   ├─ your_shell_script_library_spec.sh
│              :

.shellspec

Project default options for shellspec command.

.shellspec-local

Override project default options (Do not store in VCS such as git).

report/

Directory where the generator outputs reports.

coverage/

Directory where the kcov outputs coverge.

spec/

Directory where you create specfiles.

banner

If exists spec/banner file, shows banner when shellspec command executed. To disable shows banner with --no-banner option.

spec_helper.sh

The spec_helper.sh loaded by --require spec_helper option. This file use to preparation for running examples, define custom matchers, and etc.

support/

This directory use to create file for custom matchers, tasks and etc.

Specfile

Example

The best place to learn how to write specfile is sample/spec directory. You must see it! (Those samples includes failure examples on purpose.)

Describe 'sample' # example group
  Describe 'bc command'
    add() { echo "$1 + $2" | bc; }

    It 'performs addition' # example
      When call add 2 3 # evaluation
      The output should eq 5  # expectation
    End
  End

  Describe 'implemented by shell function'
    Include ./mylib.sh # add() function defined

    It 'performs addition'
      When call add 2 3
      The output should eq 5
    End
  End
End

Translation process

The specfile is a valid shell script syntax, but performs translation process to implements the scope and line number etc. Each example group block and example block are translate to subsshell. Therefore changes inside the block do not affect the outside of the block. In other words it realize local variables and local functions in the specfile. This is very useful for describing a structured spec. If you are interested in how to translate, use the --translate option.

DSL

Describe / Context (example group)

You can write structured example using by Describe, Context. Example groups can be nested. Example groups can contain example groups or examples. Each example groups run in subshell.

It / Example / Specify (example)

You can write describe how code behaves using by It, Example, Specify. It constitute by maximum of one evaluation and multiple expectations.

When (evaluation)

Defines the action for verification. The evaluation is start with When It can be defined evaluation up to one for each example.

When call echo hello world
 |    |    |
 |    |    +-- The rest is action for verification
 |    +-- The evaluation type `call` is call a function or external command.
 +-- The evaluation is start with `When`

The (expectation)

Defines the verification. The expectation is start with The

Verify the subject with the matcher.

The output should equal 4
 |    |           |
 |    |           +-- The `equal` matcher verify a subject is expected value 4.
 |    +-- The `output` subject uses the stdout as a subject for verification.
 +-- The expectation is start with `The`

You can reverses the verification with should not.

The output should not equal 4

You can use the modifier to modify the subject.

The line 2 of output should equal 4
    |
    +-- The `line` modifier use specified line 2 of output as subject.

The modifier is chainable.

The word 1 of line 2 of output should equal 4

You can use ordinal numbers.

The second line of output should equal 4

shellspec supports to improve readability language chains like chai.js. It is only improve readability, does not any effect the expectation.

  • a
  • an
  • as
  • the

The following two sentences are the same meaning.

The first word of second line of output should valid number
The first word of the second line of output should valid as a number

Skip / Pending

You can skip example by Skip. If you want to skip only in some cases, use conditional skip Skip if. You can also use Pending to indicate the to be implementation. You can temporary skip Describe, Context, Example, Specify, It block. To skip, add prefixing x and modify to xDescribe, xContext, xExample, xSpecify, xIt.

Before / After (hook)

You can define hooks called before/after running example by Before, After. The hook is called for each example.

Data (input for evaluation)

You can use Data Helper that input data from stdin for evaluation. After #| in the Data block is the input data.

Describe 'Data helper'
  Example 'provide with Data helper block style'
    Data
      #|item1 123
      #|item2 456
      #|item3 789
    End
    When call awk '{total+=$2} END{print total}'
    The output should eq 1368
  End
End

subject / modifier / matcher

There is more subject, modifier, matcher. please refer to the References

Custom matcher

shellspec has extensible architecture. So you can create custom matcher, custom modifier, custom formatter, etc...

see sample/spec/support/custom_matcher.sh for custom matcher.

Directive

%const (constant definition)

%const (% is short hand) directive is define constant value. The characters that can be used for variable name is upper capital, number and underscore only. It can not be define inside of the example group or the example.

The timing of evaluation of the value is the specfile translation process. So you can access shellspec variables, but you can not access variable or function in the specfile.

This feature assumed use with conditional skip. The conditional skip may runs outside of the examples. As a result, sometime you may need variables defined outside of the examples.

%text (embedded text)

You can use %text directive instead of hard-to-use heredoc with indented code. After #| is the input data.

Describe '%text directive'
  It 'outputs texts'
    output() {
      echo "start" # you can write code here
      %text
      #|aaa
      #|bbb
      #|ccc
      echo "end" # you can write code here
    }

    When call output
    The line 1 of output should eq 'start'
    The line 2 of output should eq 'aaa'
    The line 3 of output should eq 'bbb'
    The line 4 of output should eq "ccc"
    The line 5 of output should eq 'end'
  End
End

%puts / %putsn (output)

%puts (put string) and %putsn (put string with newline) can be used instead of (not portable) echo. Unlike echo, not interpret escape sequences all shells. %- is an alias of %puts, %= is an alias of %putsn.

%logger

Output log to $SHELLSPEC_LOGFILE (default: /dev/tty) for debugging.

Mock and Stub

Currentry, shellspec is not provide any special function for mocking / stubbing. But redefine shell function can override existing shell function or external command. It can use as mocking / stubbing.

Remember to Describe, Context, It, Example, Specify block running in subshell. When going out of the block restore redefined function.

Describe 'mock stub sample'
  unixtime() { date +%s; }
  get_next_day() { echo $(($(unixtime) + 86400)); }

  Example 'redefine date command'
    date() { echo 1546268400; }
    When call get_next_day
    The stdout should eq 1546354800
  End

  Example 'use the date command'
    # date is not redefined because this is another subshell
    When call unixtime
    The stdout should not eq 1546268400
  End
End

Testing a single file script.

Shell scripts are often made up of a single file. shellspec provides two ways to test a single shell script.

Sourced Return

This is a method for testing functions defined in shell scripts. Loading a script with Include defines a __SOURCED__ variable. If the __SOURCE__ variable is defined, return in your shell script process.

#!/bin/sh
# hello.sh

hello() { echo "Hello $1"; }

${__SOURCED__:+return}

hello "$1"
Describe "sample"
  Include "./hello.sh"
  Example "hello test"
    When call hello world
    The output should eq "Hello world"
  End
End

Intercept

This is a method to mock/stub functions and commands when executing shell scripts. By placing intercept points in your script, you can call the hooks defined in specfile.

#!/bin/sh
# today.sh

test || __() { :; }

__ begin __

date +"%A, %B %d, %Y"
Describe "sample"
  Intercept begin
  __begin__() {
    date() {
      export LANG=C
      command date "$@" --date="2019-07-19"
    }
  }
  Example "today test"
    When execute ./today.sh
    The output should eq "Friday, July 19, 2019"
  End
End

For developer

If you want to know shellspec architecture and self test, see CONTRIBUTING.md

Version history

See Changelog

You can’t perform that action at this time.