Unhandled exceptions when calling load with malformed data

[gabe-sherman]

Two different value errors occur in the below code as the result of unhandled exceptions in the _walk_ast function of lib.py. These unhanded exceptions cause an abrupt exit of the program rather than being handled or providing a more descriptive message. If this is not something of interest I can understand that, but I wanted to bring this to attention.

import json5
import io

data = io.BytesIO(b"--6.0")
loadval1 = json5.load(data)

The above program causes this trace:

Traceback (most recent call last):
  File "rep.py", line 5, in <module>
    loadval1 = json5.load(fuzzData)
  File "venv/lib/python3.10/site-packages/json5/lib.py", line 99, in load
    return loads(
  File "venv/lib/python3.10/site-packages/json5/lib.py", line 174, in loads
    return _walk_ast(ast, dictify, parse_float, parse_int, parse_constant)
  File "venv/lib/python3.10/site-packages/json5/lib.py", line 204, in _walk_ast
    return parse_float(v)
ValueError: could not convert string to float: '--6.0'

import json5
import io

data = io.BytesIO(b"--6")
loadval1 = json5.load(data)

While the above program causes this other trace:

Traceback (most recent call last):
  File "rep.py", line 5, in <module>
    loadval1 = json5.load(fuzzData)
  File "venv/lib/python3.10/site-packages/json5/lib.py", line 99, in load
    return loads(
  File "venv/lib/python3.10/site-packages/json5/lib.py", line 174, in loads
    return _walk_ast(ast, dictify, parse_float, parse_int, parse_constant)
  File "venv/lib/python3.10/site-packages/json5/lib.py", line 207, in _walk_ast
    return parse_int(v)
ValueError: invalid literal for int() with base 10: '--6'

Commit: d103b62

[dpranke]

Hi!. Thanks for the reports. To be clear: what would you expect to happen? The json module would return json.decoder.JSONDecodeError, but json5 doesn't use those exceptions, and returns ValueError when fed invalid data. Is this different than any other kind of invalid string? Or are you looking for a different kind of error message?

[gabe-sherman]

Yea I see what you mean and that makes sense. I am not very familiar with the library so I have a difficult time distinguishing between exceptions that are expected to be raised versus exceptions that potentially occur because of buggy code. Thank you for your time and response!

[dpranke]

No problem. This still gives me a chance to see if I can produce a better error message (I probably can) and improve the documentation (it looks like I don't actually document that we'll raise a ValueError in these situations), so I do appreciate it! :).

[dpranke]

So, interestingly, there is actually a bug here. --6.0 and --6 should not be legal JSON5 documents, but the parser is allowing them, and it's only failing later when trying to turn the parsed document into a value. This should be erroring out
immediately with Unexpected "-" at column 2.

So, I'm reopening this to fix the bug and also document the exceptions.

[dpranke]

Fixed in 6994d00, but leaving this open as I haven't pushed this to main and published it yet.

[dpranke]

v0.11.0 released, so closing this now.