file containing null byte causes crash

[jkittner]

reproduction

echo -e '\x00' > bytes.txt && babi bytes.txt

Traceback:

Traceback (most recent call last):
  File "/tmp/venv/bin/babi", line 8, in <module>
    sys.exit(main())
  File "/tmp/venv/lib/python3.10/site-packages/babi/main.py", line 176, in main
    return c_main(stdscr, file_infos, stdin, perf)
  File "/tmp/venv/lib/python3.10/site-packages/babi/main.py", line 63, in c_main
    res = _edit(screen, stdin)
  File "/tmp/venv/lib/python3.10/site-packages/babi/main.py", line 25, in _edit
    screen.file.ensure_loaded(screen.status, screen.layout.file, stdin)
  File "/tmp/venv/lib/python3.10/site-packages/babi/file.py", line 276, in ensure_loaded
    self.go_to_line(self.initial_line, dim)
  File "/tmp/venv/lib/python3.10/site-packages/babi/file.py", line 116, in action_inner
    return func(self, *args, **kwargs)
  File "/tmp/venv/lib/python3.10/site-packages/babi/file.py", line 414, in go_to_line
    self.buf.x = 0
  File "/tmp/venv/lib/python3.10/site-packages/babi/buf.py", line 245, in x
    self._x_hint = self._cursor_x
  File "/tmp/venv/lib/python3.10/site-packages/babi/buf.py", line 275, in _cursor_x
    return self.line_positions(self.y)[self.x]
  File "/tmp/venv/lib/python3.10/site-packages/babi/buf.py", line 266, in line_positions
    value = _offsets(self._lines[idx], self.tab_size)
  File "/tmp/venv/lib/python3.10/site-packages/babi/buf.py", line 44, in _offsets
    ret.append(ret[-1] + wcwidth(c))
  File "/tmp/venv/lib/python3.10/site-packages/babi/horizontal_scrolling.py", line 43, in wcwidth
    self._window.addstr(0, 0, c)
ValueError: embedded null character

Versions:

babi==1.5.3
babi-grammars==0.0.43
cffi==1.15.0
identify==2.5.1
onigurumacffi==1.2.0
pycparser==2.21

characters like \x01 or \x02 work fine and are displayed as ^A or ^B

context

I encountered this error while trying to open and fix a somewhat corrupted file. While python was writing to this file, the machine crashed and a bunch of null bytes ended up at the end of the file, removing them makes the file usable again.

nano, less and vim display \x00 as ^@.

[asottile]

it should probably just reject these files -- similar to what it does for non-UTF-8 files

[jkittner]

it only comes up when trying to render the char, so opening the file with \x00 in it works (second case passes, since the display height is 80 lines) fine, as long as it is not rendered (first case).

@pytest.mark.parametrize('s', (b'\x00', b'\n' * 81 + b'\x00'))
def test_file_contains_zero_bytes(s, run, tmpdir):
    f = tmpdir.join('f')
    f.write_binary(s)
    with run(str(f)) as h, and_exit(h):
        h.press('a')
        h.await_text('*')
        h.press('^S')
        h.await_text('saved!')
        # h.await_text('error! zero byte found:')

It should probably display an error message like for non utf-8 files. So how can we catch this earlier, like when opening instead of when it's rendered? somewhere in babi.file.get_lines, but sio is IO[str] so we can't check b'\x00' in ... and a non byte string of '\x00' should still be allowed...

[asottile]

here's where the other errors are checked:

babi/babi/file.py

Lines 67 to 75 in 511de75

	def _load_file(filename: str) -> tuple[list[str], str, bool, str]:
	try:
	with open(filename, encoding='UTF-8', newline='') as f:
	return get_lines(f)
	except UnicodeDecodeError:
	raise OpenError(f'error! not utf-8: {filename!r}')
	except OSError:
	# XXX: not quite correct, but maybe fix another day
	raise OpenError(f'error! not a file: {filename!r}')