Skip to content

Writing tests

Jump to bottom
Adrien Quillet edited this page Feb 16, 2023 · 5 revisions

To write efficient tests, one needs to understand how a test case is instanciated, what is its life cycle, what it can offers. This section will deal with those subjects.

What is a test case. What is TestCase ?

In STH, a test case is a set of tested functionnalities. Those functionnalities share some aspects (functional aspects, technical aspects), thus they are grouped into the same test case.

A test case is represented by the TestCase class. This class offers :

  • Methods that takes part of the test case lifecycle. Those methods can be redefined to customize test behavior,
  • Assertion framework access. A bunch of methods are available in all test cases ; those methods are called assertions, and allow to test specific things depending on the context. Most method names will begin with the assert_that prefix, or await_for depending of the required type of assertion. Those methods should never be redefined !

TestCase itself inherits from Node class. This allows test cases to be added to scene tree, and to be processed like any other nodes.

To create a test case, you simply have to create a GDScript file, that extends TestCase. 🎉 That's it !

TestCase structure and lifecycle

Customize test case name

In reports, test case name is, by default, the name of the associated GDScript file. You can customize displayed name by specifying the class_name directive in test case script.

Customize behavior

# Called once per test case, before any test
static func beforeAll() -> void:
    pass

# Called once per test case, after any test
static func afterAll() -> void:
    pass

# Called before each test
func beforeEach() -> void:
    pass

# Called after each test
func afterEach() -> void:
    pass

Test cases have four pre-defined methods that can be redefined :

  • beforeAll: this method is called only once during test case lifecycle, before any tests. This is a static function, so you can not interact with test case instance. It can be useful to initialize global states, like singletons,
  • afterAll: this method is called only once during test case lifecycle, after all tests. This is a static function, so you can not interact with test case instance. It can be useful to finalize global states, like singletons,
  • beforeEach: this method is called multiple times during a test case. This method is called once before each test method. You can initialize tested objects in this function,
  • afterEach: this method is called multiple times during a test case. This method is called once after each test method. You can finalized tested objects in this function.

Creating test methods

In TestCase, test methods are defined by convention : any methods whose name starts with test (case-sensitive) is considered to be a test method. Well, the method should not be static too !

# NOT a test
func TestTruc() -> void:
    pass

# NOT a test
static func test_something() -> void:
    pass

# IS a test
func test_a_thing() -> void:
    pass

# IS a test
func testAThing() -> void:
    pass

Lifecycle

Previous sections talk about predefined methods. As you can imagine, those methods are well-known by STH test runner, and are auto-magically called when needed. Here is the runner workflow :

  1. Check if test case script is valid and can be instantiated
  2. Create test case GDScript object
  3. Call beforeAll method from GDScript object
  4. For each test method in test case :
    1. Instantiate a new test case object by invoking new method on GDScript object
    2. Add instantiated test case instance into scene tree
    3. Call beforeEach method on instantiated test case object
    4. Call test method on instantiated test case object
    5. Call afterEach method on instantiated test case object
    6. Remove test case object from scene tree
    7. queue_free instantiated test case object
  5. Call afterAll method from GDScript object

Test method content

A test method is usually made of :

  • one or more functional instructions. Those instructions are related to objects taht are tested : function call, variable assignation, ...,
  • one or more assertions. Those assertions can verify tested object state, internval variables, side effects, ...

In rare cases, a test method can have zero assertion. One can think the associated test method will always reports success. That's not true. The STH testsuite runner also checks for errors (script errors, assertion errors, pushed errors) when running a test. As a result, a test can failed without assertion, because there is a pushed error in logs. So you can have a test with no assertion just to verify there is no pushed errors.

Given, when, then

That beeing said, a usual test case will contains assertions. A good practice when writing tests is to use the Given, When, Then approach :

  • Given a certain application state,
  • When I do something (function call, variable assignation, ...),
  • Then I expected a result.

This approch can be directly translated into code :

var _npc:NPC

func beforeEach() -> void:
    _npc = load("res://npc.tscn").instantiate()
    add_child(_npc)
    
func afterEach() -> void:
    remove_child(_npc)
    _npc.queue_free()
    _npc = null

func test_npc_attcks_if_player_is_near() -> void:
    # Given the NPC position, and player position
    _npc.global_position = Vector2(50, 50)
    _npc.set_player_position(Vector2(48, 49))
    
    # When NPC update state
    _npc.update_state()
    
    # Then it's new state is ATTACK PLAYER
    assert_equals(NPC.STATE_ATTACK_PLAYER, _npc.state)

Following Given When Then forces developpers to make short, simple and unit tests.

Assertions

Assertions are instructions that test a portion of code, a value, ... If an assertion fails, the enclosing test method will fails too, even if others assertions succedded. Enclosing test case will also fails.

STH provides two kinds of assertions :

  • synchronous assertions : those assertions can be directly called in your code in order to verify something,
  • asynchronous assertions : those assertions verify something that can happen in the future, i.e. not now, like signal emition.

Synchronous assertions

Synchronous assertions can be used anywhere in your test method. TestCase provides them as method, prefixed by assert_, like assert_equals or assert_that_string. Each assertion is specialised for testing a special kind of object : int, float, Vector2, Array, ... If a specific assertion is not available, you can always compose with existing one, and with the more generic ones : assert_true and assert_false.

See assertions reference for full assertion API documentation.

Asynchronous assertions

Asynchronous assertions are a little bit more complicated that synchronous assertions. Since they test asynchronous behaviors, TestCase provides them as method prefixed by await_for. Those assertions are coroutines.

A coroutine has the ability to pause execution before the end of a function, return to its caller, and be resumed where it left off. It can be useful where you want to perform a repetitive action but not on every frame. For example, checking for the proximity of an enemy.

(https://gdscript.com/solutions/coroutines-and-yield/)

For that reason, all asynchronous assertions must be used in await context.

Tip ! If you forget to use keyword await when calling an asynchronous assertion, the assertion will automatically failed !

await await_for("NPC dies").at_most(3).until_signal_emitted(npc.on_die) # SUCCESS
wait_for("NPC dies").at_most(3).until_signal_emitted(npc.on_die) # WILL ALWAYS FAIL

See assertions reference for full assertion API documentation.

Clone this wiki locally