24g Improvements: Pseudo

[ngjunsiang]

Let's start by ignoring type errors that we can't/won't fix: [7c634ee], [7819c64]

This isn't closing an eye; it's making sure the "noise" doesn't distract us from more important type errors in future. Anyway, these issues can only be resolved with a python/mypy version upgrade, so we'll come back to them when that happens.

[ngjunsiang]

The Result TypedDict

Most obvious thing that needs fixing here is that the result dict is not typed, so let's make a TypedDict:

pseudo-9608/pseudocode/__init__.py

Lines 15 to 19 in b015209

	class Result(TypedDict):
	"""The metadata dict passed to an Array declaration"""
	lines: List[str]
	frame: Frame
	error: Optional[builtin.PseudoError]

And add the type annotation at instantiation:

pseudo-9608/pseudocode/__init__.py

Lines 62 to 68 in 26bb723

	def run(self, src: str) -> Result:
	globalFrame = Frame(typesys=system.types, outer=system)
	result: Result = {
	'lines': [],
	'frame': globalFrame,
	'error': None,
	}

Registering handlers

Another type issue: we can't add new handlers to a Mapping; `mypy expects only MutableMappings to be changed. So let's update the type annotation:

pseudo-9608/pseudocode/__init__.py

Lines 45 to 51 in b015209

	class Pseudo:
	"""A 9608 pseudocode interpreter"""
	def __init__(self) -> None:
	self.handlers: MutableMapping[str, function] = {
	'output': print,
	'input': input,
	}

[ngjunsiang]

And we are done! No type errors!

Bugfixing

Doesn't mean our code works yet. Let's do a simple test:

DECLARE AnArray : ARRAY[1:10] OF INTEGER
TYPE Student
    DECLARE Surname : STRING
    DECLARE FirstName : STRING
    DECLARE YearGroup : INTEGER
ENDTYPE
DECLARE Pupil1 : Student
DECLARE Var : INTEGER

FOR Var <- 1 TO 10
    AnArray[Var] <- Var
ENDFOR
Pupil1.Surname <- "Johnson"
Pupil1.FirstName <- "Leroy"
Pupil1.YearGroup <- 6
Var <- 9
OUTPUT Pupil1.Surname
OUTPUT Pupil1.FirstName
OUTPUT Pupil1.YearGroup
FOR Var <- 1 TO 10
    OUTPUT AnArray[Var]
ENDFOR

Result:

[Line 11]     AnArray[Var] <- Var
                      ^
ParseError: [Line 11 column 12] <None> 'Var': Expected ('BOOLEAN', 'INTEGER', 'REAL', 'STRING') index

Allow NameExpr in array index

Oops, our parser only handles literal indices. Simple fix really: [836e982]

Exception in:
  File "pseudocode/resolver.py", line 548, in verify
    raise ValueError(f"Invalid Stmt {stmt}")

ValueError: Invalid Stmt DeclareStmt(Declare('UnresolvedName(<pseudocode.lang.Name object at 0x7f681c2c7910>)', 'ARRAY', {'size': ((1, 10),), 'type': 'INTEGER'}, [Line 1 column 8] <None> 'AnArray'))

Oops, we can never exit from verify() because of that ValueError, let's get rid of it: [d41faad]

[ngjunsiang]

[Line 10] FOR Var <- 1 TO 10
              ^
LogicError: [Line 10 column 4] <None> 'Var': Undeclared

Hmm ... seems like declaring variables is broken. This starts from the parser, which we did not completely fix to handle Names correctly: [62ee887]

Along the way we add a __repr__() to Name to help with debugging: [cfb2a56]

Since Names have attached tokens, Declare does not need to be declared with a token anymore, and so we can pass a Name to Declare at instantiation: [e6a0f8b]

Then we fix the handling of Declare.name: [9ff44e4]

[ngjunsiang]

Exception in:
  File "pseudocode/resolver.py", line 397, in resolve
    raise TypeError(f"Encountered {expr} in resolve()")

TypeError: Encountered UnresolvedName(Name(Var)) in resolve()

With some tracing, we find out it is happening in resolveIndex():

  pseudo(24)<module>()
-> result = pseudo.runFile(srcfile)
  pseudocode/__init__.py(62)runFile()
-> return self.run(src)
  pseudocode/__init__.py(87)run()
-> resolver.inspect()
  pseudocode/resolver.py(128)inspect()
-> verifyStmts(self.frame, self.statements)
  pseudocode/resolver.py(411)verifyStmts()
-> verify(frame, stmt)
  pseudocode/resolver.py(517)verify()
-> verifyLoop(frame, stmt)
  pseudocode/resolver.py(448)verifyLoop()
-> verifyStmts(frame, stmt.stmts)
  pseudocode/resolver.py(411)verifyStmts()
-> verify(frame, stmt)
  pseudocode/resolver.py(545)verify()
-> resolveAssign(frame, stmt.expr)
  pseudocode/resolver.py(264)resolveAssign()
-> assnType = resolve(frame, expr.assignee)
  /pseudocode/resolver.py(391)resolve()
-> return resolveIndex(frame, expr)
  pseudocode/resolver.py(303)resolveIndex()
-> intsElseError(frame, *expr.index)
  pseudocode/resolver.py(298)intsElseError()
-> nameType = resolve(frame, indexExpr)
> pseudocode/resolver.py(398)resolve()
-> raise TypeError(f"Encountered {expr} in resolve()")

Ah ... we didn't handle UnresolvedNames in index Exprs: [0ef9a21]

[ngjunsiang]

Exception in:
  pseudocode/interpreter.py", line 456, in execute
    raise ValueError(f"Invalid Stmt {stmt}")

ValueError: Invalid Stmt DeclareStmt(Declare(Name(AnArray), 'ARRAY', {'size': ((1, 10),), 'type': 'INTEGER'}))

Silly, really ... we didn't handle the if-else properly: [2ffdc62]

Now we get stuck in a seemingly infinite loop; turns out we parsed FOR loops as Loops instead of Whiles, so they don't get picked up by execute(): [07ee720]

And then fix evalAssign() because I forgot to change the code when I copied it from execInput(): [beb87ba]

[ngjunsiang]

One last fix; we need to ignore TypeStmts in the interpreter as well: [a3cf903]

Johnson
Leroy
6
1
2
3
4
5
6
7
8
9
10

There we go!

[ngjunsiang]

Type fixing

$ python -m mypy pseudocode/
pseudocode/resolver.py:302: error: Incompatible types in assignment (expression has type "Tuple[Expr, ...]", variable has type "Tuple[Literal, ...]")
pseudocode/resolver.py:458: error: Argument 1 to "get" of "Object" has incompatible type "Name"; expected "str"
pseudocode/interpreter.py:149: error: Missing return statement
Found 3 errors in 2 files (checked 8 source files)

And all fixed: [1a04378], [bd3764f], [1660874]

[ngjunsiang]

We have working code (so far), and no type errors!