22a Scoping: Recursion

[ngjunsiang]

The next thing I want to do actually is to implement some built-in functions, like RND() and RANDOMBETWEEN(). But something more pressing awaits: while pseudo alloss us to declare FUNCTIONs and PROCEDUREs in global space, we can't actually call them from within a PROCEDURE or FUNCTION yet.

And that is because so far, the local scopes only declare whatever parameters they are defined with. We have yet to introduce the wider notion of global scope, where any names declared are available to all function/procedure calls. And we'll need this because ... where else are we going to put built-in functions?

And, in an oblique roundabout way, we are going to approach this by tackling recursion first. After all, recursion involves a procedure/function calling itself, which has to first be declared in the global scope right? Solve that and almost everything else will work.

[ngjunsiang]

Testing recusion

Since we have a testing framework using unittest now, let's use that to add a test for recursion:

Test code:

pseudo-9608/tests/test_recursion.py

Lines 7 to 15 in c9076f8

	PROCEDURE CountDown(Num : INTEGER)
	OUTPUT Num
	IF Num > 0
	THEN
	OUTPUT CountDown(Num - 1)
	ENDIF
	ENDPROCEDURE
	CALL CountDown(10)

The tests:

pseudo-9608/tests/test_recursion.py

Lines 28 to 38 in c9076f8

	def test_recursion(self):
	# Procedure should complete successfully
	self.assertIsNone(self.result['error'])
	def test_output(self):
	# Check output
	output = self.result['output']
	self.assertEqual(
	output,
	"10\n9\n8\n7\n6\n5\n4\n3\n2\n1\n0",
	)

It obviously fails:

AssertionError: LogicError('Not FUNCTION') is not None

And that's because CountDown() cannot access CountDown() from the global frame; it's not declared in local.

[ngjunsiang]

Accessing globals

Now, one way to resolve this without adding new features is to iterate through global names, declaring and assigning them in local if they don't exist there. This is fragile and very prone to breaking (what happens if the globals are updated?)

Instead, let's extend Frame so that it is aware of an outer frame:

pseudo-9608/pseudocode/lang.py

Lines 129 to 131 in 54c4399

	def __init__(self, outer=None):
	self.outer = outer
	self.data = {}

What can we do with an outer though? For starters, our resolver needs to do frame insertion. Each verifier function is given a frame to work with, but the Get exprs it has to resolve might not find the name they were looking for in that given frame. When that happens, we can check in outer to see if the name is there. And in the case of recursion, or nested calls, we can even follow the entire chain of outers until we find the frame with that name declared.

Let's add a lookup() method to do that:

pseudo-9608/pseudocode/lang.py

Lines 163 to 167 in db9d819

	def lookup(self, name):
	if self.has(name):
	return self
	if self.outer:
	return self.outer.lookup(name)

[ngjunsiang]

Resolving in outer

Now let's make this happen in the resolver: we want to insert the frame containing the name when we resolve a Get expr. That happens in the resolveGet() resolver:

pseudo-9608/pseudocode/resolver.py

Lines 111 to 119 in 959864a

	def resolveGet(frame, expr):
	"""Insert frame into Get expr"""
	assert isinstance(expr, Get), "Not a Get Expr"
	if not frame.has(expr.name):
	frame = frame.lookup(expr.name)
	if not frame:
	raise LogicError("Undeclared", expr.name.token())
	expr.frame = frame
	return frame.getType(expr.name)

And we use our shiny new lookup() method to "walk back" through the outers, if the initial frame does not have expr.name declared. Yes, this is like walking a linked list, if you've learned about that before! (In fact, we're implementing a call stack, where the stack here is implemented using our linked list of frames)

Resolving callables for recursion

Next, resolveProcCall() and resolveFuncCall():

pseudo-9608/pseudocode/resolver.py

Lines 121 to 137 in cdc6a97

	def resolveProcCall(frame, expr):
	expr.callable.accept(frame, resolveGet)
	# Resolve global frame where procedure is declared
	callFrame = expr.callable.frame
	callable = callFrame.getValue(expr.callable.name)
	if not isProcedure(callable):
	raise LogicError("Not PROCEDURE", token=expr.callable.token())
	resolveCall(frame, expr)
	def resolveFuncCall(frame, expr):
	expr.callable.accept(frame, resolveGet)
	# Resolve global frame where function is declared
	callFrame = expr.callable.frame
	callable = callFrame.getValue(expr.callable.name)
	if not isFunction(callable):
	raise LogicError("Not FUNCTION", token=expr.callable.token())
	resolveCall(frame, expr)

After we correctly insert the frame (with expr.callable.accept(frame, resolveGet)), we have to retrieve this frame before we getValue() our procedure/function (because the frame argument there is the local frame).

This still doesn't work yet, because in verifyFunction() and verifyProcedure() we verify the nested statements before we've properly declared and assigned the function/procedure:

pseudo-9608/pseudocode/resolver.py

Lines 231 to 252 in cdc6a97

	def verifyFunction(frame, stmt):
	# Set up local frame
	local = Frame(outer=frame)
	for expr in stmt.params:
	# Declare vars in local
	expr.accept(local, resolveDeclare)
	# Resolve procedure statements using local
	hasReturn = False
	for procstmt in stmt.stmts:
	stmtType = procstmt.accept(local, verify)
	if stmtType:
	hasReturn = True
	expectTypeElseError(
	stmtType, stmt.returnType, token=stmt.name.token()
	)
	if not hasReturn:
	raise LogicError("No RETURN in function", stmt.name.token())
	# Declare function in frame
	frame.declare(stmt.name, stmt.returnType)
	frame.setValue(stmt.name, Function(
	local, stmt.params, stmt.stmts
	))

We'll have to rearrange things a little to make it work:

pseudo-9608/pseudocode/resolver.py

Lines 208 to 229 in 05946ee

	def verifyProcedure(frame, stmt):
	# Set up local frame
	local = Frame(outer=frame)
	# Assign procedure in frame first, to make recursive calls work
	frame.declare(stmt.name, 'NULL')
	frame.setValue(stmt.name, Procedure(
	local, stmt.params, stmt.stmts
	))
	for i, expr in enumerate(stmt.params):
	if stmt.passby == 'BYREF':
	exprtype = expr.accept(local, resolveDeclare)
	expectTypeElseError(
	exprtype, frame.getType(expr.name), token=expr.token()
	)
	# Reference frame vars in local
	local.setValue(expr.name, frame.getValue(expr.name))
	else:
	expr.accept(local, resolveDeclare)
	# params: replace Declare Expr with slot
	stmt.params[i] = local.get(expr.name)
	# Resolve procedure statements using local
	verifyStmts(local, stmt.stmts)

pseudo-9608/pseudocode/resolver.py

Lines 231 to 252 in 05946ee

	def verifyFunction(frame, stmt):
	# Set up local frame
	local = Frame(outer=frame)
	# Assign function in frame first, to make recursive calls work
	frame.declare(stmt.name, stmt.returnType)
	frame.setValue(stmt.name, Function(
	local, stmt.params, stmt.stmts
	))
	for expr in stmt.params:
	# Declare vars in local
	expr.accept(local, resolveDeclare)
	# Resolve procedure statements using local
	hasReturn = False
	for procstmt in stmt.stmts:
	stmtType = procstmt.accept(local, verify)
	if stmtType:
	hasReturn = True
	expectTypeElseError(
	stmtType, stmt.returnType, token=stmt.name.token()
	)
	if not hasReturn:
	raise LogicError("No RETURN in function", stmt.name.token())

The workflow here is:

1. Setup the local frame first, with the original frame as its outer.
2. Declare the callable name in the outer frame
3. Initialise the Procedure/Function with local, and assign it to the name in frame.

[ngjunsiang]

Resolving recursive calls

A few more minor tweaks and fixes before this will work:

pseudo-9608/pseudocode/resolver.py

Lines 139 to 147 in bfb5957

	def resolveCall(frame, expr):
	"""
	resolveCall() does not carry out any frame insertion or
	type-checking. These should be carried out first (e.g. in a wrapper
	function) before resolveCall() is invoked.
	"""
	callable = expr.callable.frame.getValue(expr.callable.name)
	numArgs, numParams = len(expr.args), len(callable.params)
	if numArgs != numParams:

Because the resolver isn't concerned with retrieving values (that is the interpreter's work), and we don't have any helper functions/functions/evaluators to do that, we'll have to do it manually:

1. Get the frame from Call.callable
2. Retrieve the callable object from the frame (using Call.callable.name)

Test fixes

And then a couple of fixes for our test code: a wrong procedure call, then some jankiness around terminal line breaks (which we'll strip() away since they are not the key focus for the test): [bfb5957]

And now our recursion tests pass!