21c Test: Checking Errors

[ngjunsiang]

Besides checking that our code successfully executes what should be executed, we also want to check that our code raises errors where it should. These are captured in the result object (which we'll have to turn into a class soon, before the result interface gets locked into a huge body of test code).

[ngjunsiang]

Checking error-free code

We start by checking our existing test code did not raise any errors:

pseudo-9608/tests/test_procedure.py

Lines 30 to 31 in cc22009

	# Procedure should complete successfully
	self.assertIsNone(self.result['error'])

Test design

To keep our tests simple to maintain, let's bundle the test code together with its script instead of putting it in a separate file:

pseudo-9608/tests/test_procedure.py

Lines 6 to 17 in cc22009

	TESTCODE = """
	PROCEDURE TestBool(Succeeded : BOOLEAN)
	IF Succeeded = TRUE AND NOT (-1.2 = -1.5)
	THEN
	OUTPUT "Yay!"
	ELSE
	OUTPUT "Awww!"
	ENDIF
	ENDPROCEDURE
	CALL TestBool(TRUE)
	"""

pseudo-9608/tests/test_procedure.py

Line 26 in cc22009

self.result = pseudo.run(TESTCODE)

Now we have one less file, and skip file-reading issues. This way we can also design our tests so that each test script tests one code block.

[ngjunsiang]

Checking error-raising code

Let's add a test_error.py test:

pseudo-9608/tests/test_error.py

Lines 5 to 12 in 1dd3fb6

	TESTCODE = """
	IF TRUE
	"""
	class ProcedureTestCase(unittest.TestCase):
	def setUp(self):
	pseudo = pseudocode.Pseudo()
	self.result = pseudo.run(TESTCODE)

This should definitely raise a ParseError, if our code is working fine!

So let's get the error from result and check that it's an error of the correct type:

pseudo-9608/tests/test_error.py

Lines 14 to 26 in 1dd3fb6

	def test_error(self):
	# Procedure should complete successfully
	error = self.result['error']
	self.assertTrue(
	issubclass(
	type(error),
	pseudocode.builtin.PseudoError,
	)
	)
	self.assertIs(
	type(error),
	pseudocode.builtin.ParseError,
	)

And that's it! We'll be adding more test cases like these when we encounter code that should return a particular kind of error, but a bug in our code causes a different kind of error to be raised.