Testing Policy

Testing Policy

Unit tests

Unit tests should be written for all engine features. This includes:

• Entity manipulation (adding/removing components, checking if components exist, etc.)
• System containers
• Resource containers
• System scheduling
• Resource loading and unloading

Unit tests should also be written for every plugin that does not implement interaction with graphics or sound. This includes:

• Utils (e.g. math, string manipulation, etc.)
• Physics (e.g. collision detection, rigid body simulation, etc.)
• Scripting
• Scene management
• ... and any other plugins that do not implement interaction with graphics or sound

Library used

GTest is used to write unit tests.

Unit tests structure

Unit tests should be stored in a subfolder named tests in the same folder as the source code. (engine / plugin / etc.) If a lot of unit tests are written, it is recommended to create a subfolder named tests in the src folder of the plugin / engine / etc. Each separate feature should have a cpp file made to test it. For example:

• tests/EntityTest.cpp for testing entity manipulation
• tests/StartupTest.cpp for testing system scheduling with the startup scheduler
• ... Each cpp file name should end with "Test.cpp" to be easily identified as a test file. Every other file should only be used for implementation.

Unit tests compilation

Unit tests are compiled using xmake. Each cpp file should be compiled as a separate executable file, to allow tests to continue even if one of them fails. A main.cpp file should be created in the tests folder as well, which will be used to run all the tests. This file should include the GTest header and call RUN_ALL_TESTS() at the end of the file. This boilerplate code can be added to the xmake.lua file when creating a new plugin:

for _, file in ipairs(os.files("tests/**.cpp")) do
    local name = path.basename(file)
    if name == "main" then
        goto continue
    end
    target(name)
        set_kind("binary")
        if is_plat("linux") then
            add_cxxflags("--coverage", "-fprofile-arcs", "-ftest-coverage", {force = true})
            add_ldflags("--coverage")
        end
        set_default(false)
        set_languages("cxx20")
        add_links("gtest")
        add_tests("default")
        add_packages("glm", "entt", "gtest", "spdlog", "fmt")

        -- add dependencies to tests
        -- add_deps("ExampleDependency")

        add_files(file)
        add_files("tests/main.cpp")
        if is_mode("debug") then
            add_defines("DEBUG")
        end
    ::continue::
end

Unit tests execution

Run all tests

To run all unit tests you can type:

xmake test

Run specific tests

If you want to run only 1 file of tests you can type:

xmake run [file without ".cpp"]

For example, if you want to run the test file "RegistryTest.cpp", you can type:

xmake run RegistryTest

Unit tests goals

Unit tests should cover as much cases as possible. The coverage goal is 90% for all testable features.

Functional tests

The best way to write functional tests is to use the engine to make small projects, either by contributing to EngineSquared's examples or by creating new projects specific to new features.

Functional tests goals

Functional tests aims to test the engine as a whole. Coverage is not tested here, but they should make sure that the engine works as expected. This includes:

• Plugins can be created and bound to the engine properly
• Plugins can be used in the engine properly
• (for graphical plugins) The engine can render what's wanted properly (without any graphical glitches)
• (for sound plugins) The engine can play sounds properly (without any sound glitches)
• (for physics plugins) The engine can simulate physics properly (without any physics glitches or unexpected behaviors)
• (for scripting plugins) The engine can run scripts properly

Examples

To have better use case of plugins, this project have some examples of usage of different plugins (one or multiple).

Setup

To inject one of the example to the build system, you must add a configuration to xmake by doing:

xmake f --[example_folder_name]=y

For example, if you want to integrate the minimal use case of core you could type:

xmake f --basic_core_usage=y

Some special cases exist:

All examples

You can inject all examples by running:

xmake f --all_examples=y

Executable examples

Some examples can be run without user interaction and in this case, they are specified with the file .ci_run_target in some of the example folder. You can inject them using this command:

xmake f --executable_examples=y

Run

To run an example, after injecting it, you could run them by basically run the target like this:

xmake run [TargetName]

You can find the target name inside xmake file of examples.

For example, for the usage of core, you can type:

xmake run BasicCoreUsage