Test Syntax

Config File

The top level config file that specifies resources to use for test. Here the scope, test, provider, and options for running the test are defined.

# endpoint where the docker daemon is being run.
# you can test this via 'docker -H 192.168.99.102:2376 info'
# if you are running on a mac then also 'check docker-machine ls' 
client:  https://192.168.99.100:2376

# scope file to use for test (see scope syntax below)
scope:  tests/simple/scope_small.yml

# test file to use (see test syntax below)
test: tests/simple/test_simple.yml

# provider of couchbase servers supported providers are
# (dev, docker, file)
provider: docker

# test options.
scale: 1   # passed to .Scale at runtime for scaling test
skip_setup: false 
skip_test: false
skip_teardown: false
repeat: 0  # set to -1 to run forever

Scope File

The scope file defines what resources to present to the test.

---
# buckets directive define bucket configuration.  
# multiple buckets can be created with same prefix name,
# this causes the name key to be extrapolated.
# i.e. other-1, other-2, resulting from count=2 below.
# Buckets are created during setup.  
buckets: 
  -
    name: default
    ram: 60%   # % of ram quota from server using this bucket
    replica: 1
    type: couchbase
  -
    name: other
    count: 2   # extrapolates to other-1, other-2
    ram: 10% 
    replica: 1
    type: couchbase

# here is a definition of a 4 node and 2 node cluster.
# the count modifier cases server names to be extrapolated.
# note these are just reference names, the provider has the
# backing IP addresses.
servers: 
  -
     name: local
     ram: 40%   # use 40% of the machines total memory
     index_ram: 10% 
     count: 4  # local-1, local-2, local-3, local-4
     rest_username: Administrator
     rest_password: password
     rest_port: 8091
     init_nodes: 4
     index_storage: forestdb
     services:   # services use a spread strategy 
        index: 1  # local-3 will be index node
        query: 1  # local-4 will be query node 
     buckets: default,other  # attaches default, other-1, other-2 to nodes

  -
     name: remote
     ram: 40% 
     count: 2  # remote-1, remote-2
     rest_username: Administrator
     rest_password: password
     rest_port: 8091
     init_nodes: 2
     buckets: other

As mentioned above the servers here are just reference names and not real hostnames. See Providers page for information about how the provider supplies the backing IP addresses to scopes.

The above scope can be referenced in the test using Scope Methods. For example, you could run tpcc container against the query node on local cluster:

-
   image: sequoiatools/tpcc
   command:  "./run.sh {{.QueryNode}}:8093 util/cbcrindex.sql"  # local-4

Or you can run pillow fight on the 2nd node of the remote cluster

-
   image: sequoiatools/pillowfight
   command: "-U  {{.Cluster 1 | net 1}} -I {{.Scale 1000}} -B {{.Scale 100}} -t 4 -c 10" # remote-2

More Scope Methods

Test File

Test file defines containers to run along with commands for writing end-to-end tests. A test defines a list of actions separated by a single '-' yaml char to denote list of actions.

# syntax
-
   image: <docker_image>
   command: <docker_command>
   entrypoint: [optional entry point for docker container]
   wait:  [true | false] # if execution should wait for container to compete, default = false
   requires:  [boolean expression]  # if false is returned container is not run
-
   image: ...

Image can be omitted if same command is running consecutively using same image. For example:

-
   image: sequoiatools/pillowfight
   command: "-U {{.Orchestrator}} -M 512 -p aaa -I {{.Scale 1000}} -B {{.Scale 100}} -t 4  --rate-limit {{.Scale 100}}"
-
   command: "-U {{.Orchestrator}} -M 512 -p bbb -I {{.Scale 1000}} -B {{.Scale 100}} -t 4  --rate-limit {{.Scale 1000}}"
-
   command: "-U {{.Orchestrator}} -M 512 -p ccc -I {{.Scale 1000}} -B {{.Scale 100}} -t 4  --rate-limit {{.Scale 1000}}"