lightweight-rest-tester

A lightweight REST API testing framework written in Python (working with 2.7 ~ 3.6). It reads JSON Schema test-cases from JSON files and then generates and executes unittest of Python.

1. Getting Started

Install lightweight-rest-tester using setup.py:

python setup.py install

Write your test cases into JSON files and pass their locations (directory) as the argument:

lw_rest_tester ${JSON_FILE_DIRECTORY}

JSON File Format

Specify your api and how you test it. In the api part, you can set the target REST API by URL (url) with method (method). It supports five HTTP methods, GET, POST, PUT, PATCH and DELETE. You can also set the parameters (params). In the tests part, you can add three types of test cases, timeout (timeout) in seconds, HTTP status code (statusCode) and JSON Schema (jsonSchema).

The following example makes GET (get) API call to http://json-server:3000/comments with the postId=1 parameter. When receiving the response, it checks the follows:

• if the response is received within 10 seconds.
• if the status code is 200.
• if the returned JSON satisfies JSON Schema.

{
  "api": {
    "url": "http://json-server:3000/comments",
    "method": "get",
    "params": {
      "postId": 1
    }
  },

  "tests": {
    "timeout" : 10,
    "statusCode": 200,
    "jsonSchema": {
      "Write your JSON Schema in here."
    }
  }
}

You can find some samples in here and there. For the details, please read the below.

Options

lightweight-rest-tester provides some useful options.

Base URL

You may want to run identical test case on different servers (e.g., production and staging servers.). In this case, use --base_url option:

lw_rest_tester ${JSON_FILE_DIRECTORY} --base_url ${BASE_URL}

The base url will be added to the url in test cases.

Authentication

Your API server may require the authentication on API calls. It supports HTTP basic authentication. To add it, use --auth option:

lw_rest_tester ${JSON_FILE_DIRECTORY} --auth "username:password"

2. API

The api part consists of url, method, params and data. url and method are essential, but params and data are optional.

method

GET, POST, PUT, PATCH and DELETE methods are supported.

params

When parameter values are given as an array, multiple test cases with all possible parameter-sets are generated. They will show which parameter-set fails a test if exists (please see 5. Test Case Name). For example, the following parameters generate 9 different parameter sets. One of them could be {"p1": 1, "p2": "abc", "p3": "def"}.

"params": {
  "p1": [1, 2, 3],
  "p2": "abc",
  "p3": ["def", "efg", "hij"]
}

data

When you make POST, PUT and PATCH calls, you can send JSON format data using data. You can find an example in here.

3. Tests

The tests part validates the responses of API calls. It checks if the response is received within timeout. It also checks the received status code (statusCode) and JSON (jsonSchema). Either statusCode or jsonSchema should be provided.

timeout

Request's timeout in seconds. Its default value is 10 (seconds).

statusCode

The expected status code. When an array of status codes is given, the test checks if one of these codes is received. For example, the following statusCode checks if the received status code is either 200 or 201:

"statusCode": [200, 201]

jsonSchema

This framework uses jsonschema to validate the received JSON. jsonschema fully supports the Draft 3 and Draft 4 of JSON Schema.

4. Write-and-Read Test

Sometimes, it is necessary to check if modifications on a database work correctly. We call such test scenario as Write-and-Read test that has a particular test-execution-order like PUT-and-GET. This framework supports this feature. To use it, just put two api/tests pairs in a list when writing JSON file:

[
  {
    "api": {
      "url": "http://json-server:3000/posts",
      "method": "post",
      "data": {
        "title": "foo",
        "body": "bar",
        "userId": 1
      }
    },
    "tests": {
      "statusCode": 201
    }
  },
  {
    "api": {
      "url": "http://json-server:3000/posts",
      "method": "get",
      "params": {
        "userId": 1
      }
    },
    "tests": {
      "statusCode": 200
    }
  }
]

You can find more examples in here.

Unlike the single-method test, Write-and-Read test builds always one test case to preserve test-execution order. Even when arrays of parameter values are given, this framework executes all the test cases belonging to first api/tests pair and then runs the test cases of the rest pair.

5. Test Case Name

This framework uses URL query string format to make test case name. However, it starts with test_ to be recognized as a test case in unittest. It helps you understand which parameter-set fails a test if exists. For example, test_http://json-server:3000/comments?postId=1&id=2 is the name of the following test case:

{
  "url": "http://json-server:3000/comments",
  "method": "get",
  "params": {
    "postId": 1,
    "id": 2
  },
  "timeout" : 10
}

Unlike the single-method test, Write-and-Read test uses only url for its name.

6. Contributions to This Project

Please feel free to leave your suggestions or make PRs.