End to end functional test and automation framework
Clone or download
Latest commit 13c0093 Oct 18, 2018
Permalink
Failed to load latest commit information.
bootstrap added http recorder option Oct 8, 2018
cli added ssh session to root context Oct 18, 2018
cloud merged pipeline into workflow Apr 24, 2018
criteria refomated Oct 8, 2018
deployment removed jre from jdk sdk May 3, 2018
doc updated doc Jun 22, 2018
endly updated etl workflow to follow best practice, added static/workflow Feb 12, 2018
example reformated Oct 11, 2018
gen added ssh/workflow runner option Oct 18, 2018
meta moved model to separate package Mar 23, 2018
model added variable evaluation for non structure output Oct 10, 2018
msg updated validation logic with runners, added docker automation exampl… Mar 31, 2018
notify patched defer multi task execuion Apr 24, 2018
server added default pipline variable, refactored example workflows Mar 27, 2018
shared refomated Oct 8, 2018
static updated version/changelog Oct 18, 2018
system updated doc Oct 18, 2018
test moved model to separate package Mar 23, 2018
testing added table name substitution Oct 18, 2018
udf refomated Oct 8, 2018
util refomated Oct 8, 2018
workflow patched sub workflow run Oct 3, 2018
CHANGELOG.md updated version/changelog Oct 18, 2018
LICENSE added changelog, notice, license and test Sep 18, 2017
NOTICE added changelog, notice, license and test Sep 18, 2017
README.md updated doc Oct 10, 2018
Version updated version/changelog Oct 18, 2018
api.go commit Apr 17, 2018
context.go updated comments Oct 10, 2018
context_test.go updated templates, shared workflows, workflow generator Apr 13, 2018
error.go moved model to separate package Mar 23, 2018
error_test.go instroduced ServiceActionMeta, refacotring service to reuse abstract … Feb 15, 2018
manager.go reformatted Oct 3, 2018
manager_test.go moved model to separate package Mar 23, 2018
project_generator.png updated image Oct 18, 2018
registry.go refomated Oct 8, 2018
service.go update shared workflow Apr 12, 2018
service_test.go reformatted Oct 3, 2018

README.md

Declarative end to end functional testing (endly)

Declarative funtional testing for Go. GoDoc

This library is compatible with Go 1.8+

Please refer to CHANGELOG.md if you encounter breaking changes.

Motivation

An end to end testing is a methodology which comprehensively tests an application in the environment closely imitating production system with all network communication, user, datastore and other dependencies interaction.

It takes great length to manually prepare an application and its data for testing. And it can be serious bottleneck if this process is manual or relies on the 3rd party. Complex regression test plan execution and data organization can be yet additional challenge with hundreds business use cases to test.

While there are many great frameworks selection helping with an integration testing:

web UI integration testing:

or database integration testing:

or application build and deployment:

None of these tools on its own have a comprehensive end to end testing and automation capabilities. What is worst some of these tools are coupled with a particular frameworks or specific language.

Endly takes declarative approach to test an application written in any language. The testing weight moves towards input and expected application output definition. This framework provides an ability to maintain and operate on hundreds of use cases effectively and cohesively, on top of that it can also automate system, data and application preparation related tasks. Endly can easily orchestrate e2e testing process of n-tier distributed application where backend, middleware and front-end each uses different stack.

Some typical e2e testing tasks supported by endly:

  • Local or remote system preparation including all services required by the application (with or without docker).
  • Checking out the application code
  • Building and deploying the application (with or without docker).
  • Database and data preparation
  • Setting application state
  • Regression test execution (HTTP, REST, Selenium)
  • Data and application output verification (UI changes, transaction logs, database changes)

Installation

  1. Download latest binary

     tar -xvzf endly_xxx.tar.gz
     cp endly /usr/local/bin
     endly -h
     endly -v
    
  2. Build from source a) install go 1.9+ b) run the following commands:

    mkdir -p ~/go
    export GOPATH=~/go
    go get -u  github.com/viant/endly
    go get -u  github.com/viant/endly/endly
    cd $GOPATH/src/github.com/viant/endly/endly
    go build endly.go
    cp endly /usr/local/bin
  3. Custom build, in case you need additional drivers, dependencies or UDF with additional imports:

@endly.go

package main

//import your udf package  or other dependencies here

import "github.com/viant/endly/bootstrap"

func main() {
	bootstrap.Bootstrap()
}

Getting Started

Create test project for your app.

test project generator

Endly uses various text data format (YAML, JSON, CSV), so here is some IDEs selection to consider:

a) Atom with tablr plugin (apm install tablr)

b) IntelJ with CSV plugin

Endly automate sequence of actions into reusable tasks and workflows, here are some examples:

a) System preparation

For instance: the following define inline workflowto prepare app system services:

@system.yaml

tasks: $tasks
defaults:
  target: $serviceTarget
pipeline:
  destroy:
    stop-images:
      action: docker:stop-images
      images:
        - mysql
        - aerospike
  init:
    services:
      mysql:
        workflow: "service/mysql:start"
        name: mydb3
        version: $mysqlVersion
        credentials: $mysqlCredentials
        config: config/my.cnf
      aerospike:
        workflow: "service/aerospike:start"
        name: mydb4
        config: config/aerospike.conf

b) Application build and deployment

For instance: the following define inline workflowto build and deploy a test app: (you can easily build an app for standalone mode or in and for docker container)

With Dockerfile build file and docker compose

@app.yaml

tasks: $tasks
init:
- buildPath = /tmp/build/myapp/
- version = 0.1.0
defaults:
  app: myapp
  version: 0.1.0
  useRegistry: false
pipeline:
  build:
    init:
      action: exec:run
      target: $target
      commands:
      - if [ -e $buildPath ]; then rm -rf $buildPath; fi
      - mkdir -p $buildPath
    checkout:
      action: version/control:checkout
      origin:
        URL: https://github.com/adrianwit/dstransfer
      dest:
        URL: scp://${targetHost}:22/$buildPath
        credentials: localhost
    download:
      action: storage:copy
      source:
        URL: config/Dockerfile
      dest:
        URL: $buildPath
        credentials: localhost
    build-img:
      action: docker:build
      target: $target
      path: $buildPath
      '@tag':
        image: dstransfer
        username: adrianwit
        version: 0.1.0
  stop:
    target: $appTarget
    action: docker:composeDown
    source:
      URL: config/docker-compose.yaml
  deploy:
    target: $appTarget
    action: docker:composeUp
    runInBackground: true
    source:
      URL: config/docker-compose.yaml

As Standalone app (with predefined shared workflow)

@app.yaml

tasks: $tasks
defaults:
  app: myApp
  sdk: go:1.8

pipeline:

  build:
    workflow: app/build
    origin:
      URL: ./../
    commands:
      - cd $buildPath/app
      - go get -u .
      - go build -o $app
      - chmod +x $app
    download:
      /$buildPath/app/${app}: $releasePath
      /$buildPath/endly/config/config.json: $releasePath

  deploy:
    workflow: app/deploy
    init:
      - mkdir -p $appPath
      - mkdir -p $appPath/config
      - chown -R ${os.user} $appPath
    upload:
      ${releasePath}/${app}: $appPath
      ${releasePath}/config.json: $appPath
    commands:
      - echo 'deployed'

  stop:
    action: process:stop-all
    input: ${app}

  start:
    action: process:start
    directory: $appPath
    immuneToHangups: true
    command: ./${app}
    arguments:
      - "-config"
      - "config.json"

c) Datastore creation

For instance: the following define inline workflowto create/populare mysql and aerospike database/dataset:

@datastore.yaml

pipeline:
  create-db:
    db3:
      action: dsunit:init
      scripts:
        - URL: datastore/db3/schema.ddl
      datastore: db3
      recreate: true
      config:
        driverName: mysql
        descriptor: "[username]:[password]@tcp(127.0.0.1:3306)/[dbname]?parseTime=true"
        credentials: $mysqlCredentials
      admin:
        datastore: mysql
        config:
          driverName: mysql
          descriptor: "[username]:[password]@tcp(127.0.0.1:3306)/[dbname]?parseTime=true"
          credentials: $mysqlCredentials
    db4:
      action: dsunit:init
      datastore: db4
      recreate: true
      config:
        driverName: aerospike
        descriptor: "tcp([host]:3000)/[namespace]"
        parameters:
          dbname: db4
          namespace: db4
          host: $serviceHost
          port: 3000
  populate:
    db3:
      action: dsunit:prepare
      datastore: db3
      URL: datastore/db3/dictionary
    db4:
      action: dsunit:prepare
      datastore: db4
      URL: datastore/db4/data

d) Testing

For instance: the following define inline workflowto run test with selenium runner:

@test.yaml

defaults:
  target:
     URL: ssh://127.0.0.1/
     credentials: localhost
pipeline:
  init:
    action: selenium:start
    version: 3.4.0
    port: 8085
    sdk: jdk
    sdkVersion: 1.8
  test:
    action: selenium:run
    browser: firefox
    remoteSelenium:
      URL: http://127.0.0.1:8085
    commands:
      - get(http://play.golang.org/?simple=1)
      - (#code).clear
      - (#code).sendKeys(package main

          import "fmt"

          func main() {
              fmt.Println("Hello Endly!")
          }
        )
      - (#run).click
      - command: output = (#output).text
        exit: $output.Text:/Endly/
        sleepTimeMs: 1000
        repeat: 10
      - close
    expect:
      output:
        Text: /Hello Endly!/

To see Endly in action,

In addition a few examples of fully functioning applications are included. You can build, deploy and test them end to end all with endly.

  1. Web Service

    • Reporter - a pivot table report builder.
      • Test with Rest Runner
      • Data Preparation and Validation (mysql)
  2. User Interface

    • SSO - user registration and login application.
      • Test with Selenium Runner
      • Test with HTTP Runner
      • Data Preparation and Validation (aersopike)
      • Web Content validation
      • Mocking 3rd party REST API with http/endpoint service
  3. Extract, Transform and Load (ETL)

    • Transformer - datastore to datastore transformer (i.e. aerospike to mysql)
      • Test with Rest Runner
      • Data Preparation and Validation (aersopike, mysql)
  4. Runtime - simple http request event logger

    • Logger
      • Test with HTTP Runner
      • Log Validation
  5. Automation - simple 3rd party echo app

    • Echo
      • Build 3rd party application binary in docker container
      • Build application docker image
      • Optionally publish app image to the docker registry
      • Deploy app to docker container
      • Test an app with REST and HTTP runner

Documentation

@run.yaml

target:
  URL: "ssh://127.0.0.1/"
  credentials: localhost
systemPaths:
  - /usr/local/go/bin
commands:
  - go version
  - echo $GOPATH

External resources

License

The source code is made available under the terms of the Apache License, Version 2, as stated in the file LICENSE.

Individual files may be made available under their own specific license, all compatible with Apache License, Version 2. Please see individual files for details.

Credits and Acknowledgements

Library Author: Adrian Witas