API-Automation: Using postman and newman with htmlextra reporting and azure pipeline integration

API Testing using Postman

Areas Covered:

• Create API Tests Using Postman Scripts
• Run tests using Postman Collection Runner and through Newman CLI tool
• Generate Report using Newman reporter
• Customized Newman Reporter -htmlextra
• Assertions using Chai Assertion Library
• Azure Pipeline Integration Azure CI/CD

Main Reference: DannyDainton: All-Things-Postman

Overview:

The solution tests APIs from,

1. Restful-Booker -Self Hosted
2. Restful-Booker -Public Link
3. Other API endpoints available are: Random User and Httpbin

Azure CI/CD integration working modal is available at: ajithgeorgethekkel0427

Prerequisites:

1. Postman Application installed.
2. Newman tool installed.
3. Node and Npm installed.
4. Htmlextra/Newman reporter node packages installed.

Execution Steps:

1. Clone the repo.
2. Import the collection and environment to postman tool.
3. Run the tests from either Postman Collection Runner or through Newman tool.
4. Use appropriate parameter for newman test execution to generate report.

Postman Sandbox API reference

Status Check:

        pm.response.to.be.info - Checks 1XX status code
        pm.response.to.be.success - Checks 2XX status code
        pm.response.to.be.redirection - Checks 3XX status code
        pm.response.to.be.clientError - Checks 4XX status code
        pm.response.to.be.serverError - Checks 5XX
        pm.response.to.be.error - Checks 4XX or 5XX
        pm.response.to.be.ok - Status code must be 200
        pm.response.to.be.accepted - Status code must be 202
        pm.response.to.be.badRequest - Status code must be 400
        pm.response.to.be.unauthorized - Status code must be 401
        pm.response.to.be.forbidden - Status code 403
        pm.response.to.be.notFound - Status code of response is checked to be 404
        pm.response.to.be.rateLimited - Checks whether response status code is 429

Ref: Extended Checks,Assertion Samples

Reuse Javascript code using Environment Variable

Setting as environment variable:

// Save common tests in a global variable

postman.setGlobalVariable("commonTests", () => {

  // The Content-Type must be JSON

  tests["Content-Type header is set"] = postman.getResponseHeader("Content-Type") === "application/json";

  // The response time must be less than 500 milliseconds

  tests["Response time is acceptable"] = responseTime < 500;

  // The response body must include an "id" property

  var data = JSON.parse(responseBody);

  tests["Response has an ID"] = data.id !== undefined;

});

Calling in Test/Pre-Script:

// First, run the common tests
eval(globals.commonTests)();

// Then run any request-specific tests
tests["Status code is 200"] = responseCode.code === 200;

Reference: Postman Tips

Schema as environment variable

Setting as environment variable:

const schema = {
    "required": [
        "bookingid",
        "booking"
    ],
    "properties": {
        "bookingid": {
            "type": "integer"
        },
        "booking": {
            "type": "object",
            "required": [
                "firstname",
                "lastname"
            ],
            "properties": {
                "firstname": {
                    "type": "string",
                },
                "lastname": {
                    "type": "string",
            }
        }
    }
}

pm.environment.set("testSchema",schema);; 

Comparing from environment variable:

var data = pm.response.json()

pm.test('Check Schema from Environment Variable', function () {
    pm.expect(tv4.validate(data, (pm.environment.get("testSchema")))).to.be.true;});

Note: The 'Required' field is mandatory else there won't be a strict match done for the field name.

Dynamic resource/data linking

Instead of hardcoding the resource url (like in case of id) use dynamic linking of the same to reduce error.

• Send a reuest in the pre-script to fetch in the list of resources ids.
• Select a random id from the list (enusre random selection to prevent the selection of the same id every run -displayed script doesn't randomize as the api return was inturn not in a particular set format)
• Set the same as an environment/global variable and consume in the API method.

Branching and looping of Scripts

Set the request to be executed next:

postman.setNextRequest("request_name");

Loop over the current request- use the same test in the setNextRequest() method. One should wrap setNextRequest in some logic so as to ensure that the request does not run indefinitely otherwise the collection runner would need to be force closed.

If postman.setNextRequest() is absent in a request, the collection runner defaults to linear execution and moves to the next request.

Stop workflow execution

postman.setNextRequest(null);

postman.setNextRequest() is always executed at the end of the current request. This means that if you put this function before other code blocks anywhere in pre-request or test script, these blocks will still execute.

postman.setNextRequest() has a scope, which is the source of your collection run. If you run a collection, you can jump to any request in the collection (even requests inside folders, using the same syntax). However, if you run a folder, the scope of postman.setNextRequest() is limited to that folder. So you can jump to any request in this folder, but not ones that are outside of the folder. It includes requests inside other folders, and also root-level requests in the collection.

Single test to iterate over a set of data

Instead of writing the same test script for different endpoints/ids the same test could be used to iterate over the data without any repetition of the code. (DRY -Don't Repeat Your code)

Pre-Script: Handles the initial gathering of ids and setting the variables.

Test Script: Handles the looping logic and teardown.

Variables: Through which the data is transferred for subsequent request

Tips#

1. Check presence of variables:

Have a folder Pre-Condition/Pre-Requisites which will check for the required variables in the environment/global space and if NOT will create them, thereby reducing the chance of error.

Sample: Setting a function to a variable if not already present.

if(!pm.variables.has("variableName")){}

if (pm.variables.get("variableName")===undefined){}

2. Endpoint availabiltiy check:

Have a folder for Health Check- to check the availability of the endpoint before commencing the remaining tests. There could be a logic added to prevent the execution of the remaining test if the endpoint is not alive.

if(!pm.response.to.be.success){

    postman.setNextRequest(null);
}

3. Integer validation:

To compare int values don't forget to do implicit conversion if the retrieved value is from any environment/global variable. Postman stores all variables in string format only.

pm.test("15 should equal 15", () => pm.expect(15).to.equal(parseInt(pm.globals.get('another_global_number'))))

4. CleanUp methods:

Different teardown approaches:

• Can specifically remove/unset the variables after the test.

function cleanup() {
    pm.environment.unset("id");
    pm.globals.unset("counter");
    pm.globals.unset("arrayLength");
    pm.globals.unset("bookingIds");
}

cleanup()

• Can remove/unset the varibales based on a preset prefix.

function cleanup() {

    const clean = _.keys(pm.environment.toObject())

    _.each(clean, (arrItem) => {

        if (arrItem.startsWith("test")) {

            pm.environment.unset(arrItem)

        }
    })

}

cleanup()

• Can remove an entire type of variables. Approach uses variables that are to be retained as one type(say global) and the others as a different type (say environment).

pm.globals.clear(); // This will remove all globals, use with caution.

• Remove items using a pre-defined array of variables

function cleanup() {
    const clean = ['first_name', 'last_name', 'total_price', 'depositPaid', 'check_in', 'check_out', 'additional_needs']
    for(let i = 0; i < clean.length; ++i){
        pm.environment.unset(clean[i])
    }
}
cleanup()

Sample Report

file: HtmlExtraReport.html