Design Test Plan

A test story often verify a small feature of the application. We can create many stories to cover all features need to be tested, and we also need to organize these stories and run them in control and in different ways. The Test Plan is the object Handow used to control a set of stories running.

Run a specific story is just required in local developing stage, Handow always consume a plan in real test running.

• A Test Plan is a JSON file - [planName].plan.json.
• A plan is composed with Stages, a stage is a multi-story container object.
• A plan can have its Config to customize Handow default behavior for this plan running.
• Handow can assign multiple 'workers' to run stories of a plan in parallel to speed up test running.
• Handow scheduler can dispatch plans running
• The report package is created for a plan after plan running.

Developers can add a plan file directly under the project/ root. The Editor in Edit dashboard can create, edit or delete a plan.

    {
        "title": "The key features plan",
        "stages": [],
        "config: {}
    }

Most stories couldn't run solely, e.g., we cannot run the test story to verify a secured page without finishing the Login story. It is recommended to run stories in a plan, even they are sole runnable and in local running.

Stages of a plan

A plan includes one or more stages.

• Stages are executed in sequential accouding to the index order of stages array.
• A stage contains a set of stories running randomly and in parallel if multiple workers are working on current plan.
• A stage includes its configuration properties, e.g., description and newBrowserSession
• Optionally the params object could be added to a stage, the parameters in params are available to all stories in current stage and override relevant parameters declared in global params file.

{
    "stage": "Navigation stage",
    "description": "Go through all the tabs to verify resouces visibility for current role. This stage is arranged after the Login stage",
    "newBrowserSession": false,
    "stories": ["NavHead", "HomePagePermission", "LibraryPagePermission", "BookPagePermission", "NotificationPagePermission"],
    "params": {
        "Login_Role": "Student_Role"
    }
}

stories - Stories in a stage shouldn't depend on each others. That means the testing features are not related with the running status and result of other stories in the stage. So that Handow can run them randomly and in pallel with muli-workers.

newBrowserSession - The browser will create a new context to evaluate this stage if it is true, the session cache is cleared.

params - Although any parameter could be defined in a stage, but mostly we add parameters here for running control refered by @skip: () expression.

The executing order is critical for a plan. Developers need design the stages carefully, implement preset and newBrowserSession to make the plan running as expected.

Config of a plan

Handow provide built-in configuration, and there is an config file config.js existed in handow-seed project which will override the built-in configuration. The config object in a plan is same as the Handow configuration but override the global value for this plan. That make it easy to config plans with different behaviors.

For example, the following config is added to a plan. That plan will always use headless browser for testing, and the max waiting time for element visible will be 5 seconds (shorter than the default 10s)

{
    "headless": true,
    "elementAppearTimeout": 5000
}

Read the Handow config document to know all the configurable items.

Multiple plans

A plan is complete test package, that doesn't mean you need one plan for your test project. We often create a lot of plans and run one or multiple of them on demand. For example, a full plan for everything, a oriented plan for some features, a plan just for local development test or a quick plan running after application deployed.

Above the plan, Handow provide scheduler to run multiple plans in one test cycle. So that users needn't define a huge plan for complex test project.

Scheduler can run multiple plans in one test cycle, but the reports are stilled generated for each plan.

Plan tree

When a plan is opened in Runner locally, it is rendered into a graphic tree to make the stages, stories, scenarios and looping instutive. Handow provide realtime monitor by watching the plan tree status updating dynamically.

Related documents

Deployment And Auto Run