y_testing | Implement spy & mock capabilities

[pedropapa]

I've seen some discussion around the topic at #157, but couldn't find a way to do it with the current documentation for y_testing.

In unit testing you want to obviously test the unit, a single method (for instance), everything else within the file should not be tested (given they have their own tests or will be tested implicitly like private functions) and external calls should be mocked. I see in y_testing a good solution for integration tests, but it still lack important features in order to serve as a unit testing library.

Example:

// my_implementation.inc
#include <a_samp>
...
forward CreateAccount(playerid);
forward ValidatePlayerName(name[]);
...
public CreateAccount(playerid) {
    new name[MAX_PLAYER_NAME + 1];
    GetPlayerName(playerid, name, sizeof(name));

    if(ValidatePlayerName(name) == 0) {
        SendClientMessage(playerid, 0xFFFFFFFF, "Your name is invalid. Try another one");
        return 0;
    }

    return 1;
} 

public ValidatePlayerName(name[]) {
    return (strlen(name) < 3 || strlen(name) > 20) ? 0 : 1;
}

// my_implementation_tests.inc
#include <my_implementation>
#include <YSI_Core\y_testing>

@test(.group = "CreateAccount") ShouldSendMessageIfInvalid()
{ // What I'd like to achieve with y_testing.
        MOCK_NATIVE_RETURN_VALUE("GetPlayerName", "ab");
	ASSERT_EQ(CreateAccount(1), 0);
	ASSERT_CALL("SendClientMessage", 1, 0xFFFFFFFF, "Your name is invalid. Try another one");
}

@test(.group = "CreateAccount") ShouldNotSendMessageIfValid()
{ // What I'd like to achieve with y_testing.
        MOCK_NATIVE_RETURN_VALUE("GetPlayerName", "abc");
	ASSERT_EQ(CreateAccount(1), 1);
	ASSERT_CALL_COUNT("SendClientMessage", 0);
}

@test(.group = "ValidatePlayerName") ShouldReturn0IfInvalid()
{
	new name[MAX_PLAYER_NAME + 1];

	format(name, sizeof(name), "ab");
	ASSERT_EQ(ValidatePlayerName(name), 0);

	format(name, sizeof(name), "ababababababababababa");
	ASSERT_EQ(ValidatePlayerName(name), 0);
}

@test(.group = "ValidatePlayerName") ShouldReturn1IfValid()
{
	new name[MAX_PLAYER_NAME + 1];

	format(name, sizeof(name), "abc");
	ASSERT_EQ(ValidatePlayerName(name), 1);

	format(name, sizeof(name), "abcdefg");
	ASSERT_EQ(ValidatePlayerName(name), 1);

	format(name, sizeof(name), "abababababababababab");
	ASSERT_EQ(ValidatePlayerName(name), 1);
}

The example shows the need to properly test my CreateAccount function. I need to mock the native GetPlayerName to return a specific value during my test and spy on SendClientMessage to check if it was called with the correct parameters.

I reckon MOCK_NATIVE_RETURN_VALUE can be achieved using the y_hooks api, but for ASSERT_CALL or ASSERT_CALL_COUNT it would require some sort of async check with timers.

I also took a read at open.mp's raknet mock solution, it facilitates integration tests by a margin, however it still isn't the solution for unit testing since for external calls we only need to check if the function was called with the correct parameters, we don't need to check if the external function is working properly (ideally the external function's implementation could even be mocked with dummy code).

I hope it makes sense 🙏

[pedropapa]

I was able to work around and create a PoC for implementing the outstanding functions:
https://gist.github.com/pedropapa/b5d1726fac9a0972a57cadb0dce3afa4

I'm not familiar with low-level pawn code so my code looks junky, the most important is exposing an API so I can move on with my codebase.

Now I'm able to do something similar to what I proposed:

#include <y_testing_mocks>
#include <my_implementation>
#include <YSI_Core\y_testing>

@test(.group = "CreateAccount") ShouldSendMessageIfInvalid()
{
	MockInit(); // Should be in BeforeAll ideally
	MockReset("SendClientMessage");
	ASSERT_EQ(CreateAccount(1, "ab"), 1);
	ASSERT_TRUE(MOCK_CALL_COUNT("SendClientMessage", 1));
	ASSERT_TRUE(MOCK_CALL("SendClientMessage", "1 0xFFFFFFFF Your name is invalid. Try another one"));
}

There still space for improvement:

• Rename MOCK_CALL_COUNT and MOCK_CALL to ASSERT_CALL_COUNTand ASSERT_CALL to make the assertion without wrapping it around ASSERT_TRUE.
• I couldn't find a way to make the api for MOCK_CALL using a variadic function like this: MOCK_CALL("SendClientMessage", 1, 0xFFFFFFFF, "Your name is invalid. Try another one");, tried to play around with getarg with no luck.

[Y-Less]

There was a vague skeleton for y_mock a long time ago for exactly this, but it never got very far in implementation. You're right that it should be completed.