Skip to content

Test Execution Engine tester support

Jump to bottom
ptwamley edited this page Jul 19, 2018 · 3 revisions

The Test Execution Engine is built to support multiple test frameworks support tester in mind. The initial implementation supports the following configurations:

bash tester

TEE support bash base script test. You need set test_type=bash to active the bash tester.

The bash tester will execute runner.sh in package root folder. Test result will be marked based on the script exit code:

  • none 0 value = FAIlED
  • 0 = SUCCESS

TTCN3 tester

The TTCN3 tester was a special type of bash tester. You need set test_type=ttcn3 to active the tester.

The TTCN3 tester will execute runner.sh in package root folder. You could use following sample command as template:

docker run --rm -v tee:/workspace registry.sonata-nfv.eu:5000/tng-vnv-tester-ttcn3 ttcn3_start ${workspace.absolutePath}/test-suite-binary ${workspace.absolutePath}/config

TTCN3 test will write the results in a text file. If the test finished without error, TEE will parse the results text file to get the test results. Test result will be marked based on following condition:

  • none 0 value = ERROR
  • has TTCN3 tests fail = FAILED
  • has TTCN3 tests success = SUCCESS
  • has NONE TTCN3 tests success = INVALID_TEST_RESULT

The test result summary will be stored at testSuiteResult.details e.g.

{
    "test_suite_result_id":"test_suite_result_id",
    "test_suite_id":"test_suite_id",
    "details":
        {
          "none":0,
          "pass":19,
          "inconc":0,
          "fail":3,
          "error":0
        },
    "status":"FAILED"
}

WRK tester

You need set test_type=wrk to activate the tester.

The WK tester will execute runner.sh in package root folder. You could use following sample command as template:

docker run --rm -v tee:/workspace registry.sonata-nfv.eu:5000/tng-vnv-tester-ttcn3 ttcn3_start ${workspace.absolutePath}/test-suite-binary ${workspace.absolutePath}/config

WRK test will deliver the results in two text files. If the test finished without error, TEE will parse the results text file to get the test results. Test result will be marked based on following condition:

  • none 0 value = ERROR
  • has WRK tests fail = FAILED
  • has WRK tests success = SUCCESS
  • has NONE WRK tests success = INVALID_TEST_RESULT

The test result summary will be stored at testSuiteResult.details e.g.

{
    "test_suite_result_id":"test_suite_result_id",
    "test_suite_id":"test_suite_id",
    "details":
        {
          "none":0,
          "pass":19,
          "inconc":0,
          "fail":3,
          "error":0
        },
    "status":"FAILED"
}
Clone this wiki locally