IndexError: list index out of range when parsing required empty .lua file

[hsandt]

If the main or a required lua script is empty (ignoring comments), the parsing step will fail with IndexError:

Traceback (most recent call last):
  File "/usr/local/bin/p8tool", line 8, in <module>
    sys.exit(tool.main(sys.argv[1:]))
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/tool.py", line 592, in main
    return args.func(args)
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/build/build.py", line 258, in do_build
    lua_path=getattr(args, 'lua_path', None))
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/build/build.py", line 155, in _evaluate_require
    reqd_lua.reparse(writer_cls=lua.LuaASTEchoWriter)
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 182, in reparse
    version=self.version)
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 133, in from_lines
    result.update_from_lines(lines)
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 145, in update_from_lines
    self._lexer.process_lines(lines)
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lexer.py", line 470, in process_lines
    for line in lines:
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 163, in to_lines
    for line in writer.to_lines():
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 817, in to_lines
    for chunk in self.walk():
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 251, in walk
    for t in self._walk(self._root):
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 804, in _walk
    for t in super()._walk(node):
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 232, in _walk
    for t in result:
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 409, in _walk_Chunk
    yield self._get_semis(node)
  File "/home/hsandt/Developer/PICO-8/picotool-repo/pico8/lua/lua.py", line 396, in _get_semis
    if self._tokens[self._pos].matches(lexer.TokSymbol(b';')):
IndexError: list index out of range

This is due to a do-while pattern (while True-break in Python) in LuaASTEchoWriter._get_semis:

    if self._args.get('ignore_tokens') or not self._tokens:
        return b' '

    spaces_and_semis = []
    while True:
        spaces = self._get_code_for_spaces(node)
        if self._tokens[self._pos].matches(lexer.TokSymbol(b';')):
            self._pos += 1
            spaces_and_semis.append(spaces + b';')
        else:
            spaces_and_semis.append(spaces)
            break

A possible fix is:

    while True:
        # early exit
        if not self._tokens:
            break  # empty string will be returned
        spaces = self._get_code_for_spaces(node)
        # ...

or at the beginning:

    if self._args.get('ignore_tokens') or not self._tokens:
        return b' '  # slightly different, will return a space instead of empty string

etc.

The function is low-level and may be called on chunks that are not files, so I guess using the same behavior as ignore_tokens, i.e. returning a whitespace, is safer.