json library needs a non-strict option to decode single-quoted strings

[mumind]

BPO	20298
Nosy	@rhettinger, @pitrou, @ezio-melotti, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-01-21.05:14:38.013>
created_at = <Date 2014-01-19.02:19:14.356>
labels = ['type-feature', 'library']
title = 'json library needs a non-strict option to decode single-quoted strings'
updated_at = <Date 2014-01-21.05:14:38.011>
user = 'https://bugs.python.org/mumind'

bugs.python.org fields:

activity = <Date 2014-01-21.05:14:38.011>
actor = 'rhettinger'
assignee = 'none'
closed = True
closed_date = <Date 2014-01-21.05:14:38.013>
closer = 'rhettinger'
components = ['Library (Lib)']
creation = <Date 2014-01-19.02:19:14.356>
creator = 'mu_mind'
dependencies = []
files = []
hgrepos = []
issue_num = 20298
keywords = []
message_count = 3.0
messages = ['208430', '208479', '208614']
nosy_count = 5.0
nosy_names = ['rhettinger', 'pitrou', 'ezio.melotti', 'mu_mind', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20298'
versions = ['Python 3.5']

[mumind]

Many sloppy JSON APIs return data with poorly-encoded strings, either with single-quotes as delimiters:
{'foo': 'bar'}
or with no delimiters at all on keys:
{foo: 'bar'}

The json library is useless for making sense of this data, because all it will tell you is "No JSON object could be decoded".

It would be incredibly helpful if the json library had some special non-strict decoding mode that could interpret these common misspellings of JSON strings. Or, more generally, it could accept another decoder hook to reformat the remaining string, something like:
def malformed_hook(remaining, parent):
if remaining.startswith("'"):
# We know this is a string, regex it to find the end.
m = re.match(pyparsing.quotedString.reString, remaining)
if m is not None:
# Use json.dumps to add quotes around string literal.
return json.dumps(eval(m.group(0))) + remaining[m.end():]
# If we're inside an object, this could be a naked object key.
if isinstance(parent, dict):
m = re.match(r'([a-zA-Z_]\w*):', remaining)
if m is not None:
return json.dumps(m.group(1)) + remaining[len(m.group(1)):]
print(json.loads("['foo', null, {a: 'b'}]", malformed_hook=malformed_hook))
['foo', None, {'a': 'b'}]

This would at least save you having to write a parser/tokenizer from scratch.

[serhiy-storchaka]

I think this is just not JSON. Try to use YAML parsers.

[rhettinger]

Serhiy is correct. We're trying to stick close to the JSON spec.