Automated Functional Testing

Description

Darwin developers uses a simple python tester for functional testing.

Note: Only tested on FreeBSD 12/HBSD 12.1

Dependencies

• Python 3.6.9 or above
  • Darwin client python
  • redis 3.3.8 or above
• Darwin dependencies
• Redis 4.0.14 or above
• Valgrind (if you plan to use it)

Running test

cd tests/
python3 test.py

If some errors occurs, you can find the logs in test_error.log.

Configuring test

You can configure the testing paths in tests/conf.py

Options:

• TEST_FILES_DIR (string): Root path for all files created during tests. This includes configuration files, sockets, pid files and some logging files (darwin_manager.log but not darwin.log)
• DEFAULT_MANAGER_PATH (string): Path to the manager.py
• DEFAULT_FILTER_PATH (string): Path to the darwin filter directory
• DEFAULT_PYTHON_EXEC (string): Path to the python executable used to launch the manager
• VALGRIND_MEMCHECK (bool): Enable or disable valgrind tests on filters (Note: disable on BSD systems)

Example:

TEST_FILES_DIR = '/tmp'
DEFAULT_MANAGER_PATH = '/home/darwin/manager/manager.py'
DEFAULT_FILTER_PATH = '/home/darwin/filters/'
DEFAULT_PYTHON_EXEC = 'python3'
VALGRIND_MEMCHECK = False

Developing test

Tools

The tools are generic methods used across tests. They are available in tests/tools/.

Available tools:

• output: methods used to print a test result
• darwin_utils: methods used to configure, start and stop darwin
• filter: Filter object used to manage filter execution and configuration
• redis_utils: class used to generate temporary Redis server instances (please use it for filter tests requiring Redis, see flogs tests for an example of usage)

Modules

Each test "family" MUST be in its own directory (Ex. manager_socket).

The directory MUST contain:

• __init__.py
• test.py

The file test.py MUST contain a method called run() that runs all the tests in the directory. We recommend to organize your tests in different files and to call them in the test.py of your directory.

Example:

tests/example/example.py

from tools.darwin_utils import darwin_configure, darwin_remove_configuration, darwin_start, darwin_stop
from tools.output import print_result


def run():
    tests = [
        example_test,
    ]

    for i in tests:
        print_result("Example: " + i.__name__, i())

def example_test():
    ret = True

    darwin_configure('{}')
    process = darwin_start()

    # Do something here

    darwin_stop(process)
    darwin_remove_configuration()
    return ret

tests/example/test.py

import example.example as example

def run():
    print('Example Results:')

    example.run()

    print()
    print()

The main

There is a test.py at the root of the tests directory. It runs the test.py of the directories.

You need to add your tests in it for them to be executed.

Example:

tests/test.py

import logging
from sys import stderr
import example.test as example


if __name__ == "__main__":
    logging.basicConfig(filename="test_error.log", filemode='w', level=logging.ERROR)

    example.run()

    print("Note: you can read test_error.log for more details", file=stderr)