Would it be a problem if the errors thrown were specialized exceptions, e.g. JSONParsingError, which has ValueError as a base class with these details (line number, char number, etc.) exposed as public members? The changes required seem to be limited to changing http://hg.python.org/cpython/file/4c4f31a1b706/Lib/json/decoder.py and the related documentation.
msg: The unformatted error message
doc: The JSON document being parsed
pos: The start index of doc where parsing failed
end: The end index of doc where parsing failed (may be None)
lineno: The line corresponding to pos
colno: The column corresponding to pos
endlineno: The line corresponding to end (may be None)
endcolno: The column corresponding to end (may be None)
simplejson v3.5.1 is probably a good target, let me know if there's anything that you'd like me to merge back in from stdlib to make maintenance easier. I haven't been tracking stdlib except when I've been added to issues.
The only error which contains end position is "Extra data". And it doesn't look clean to me why end position should be mentioned at all (it is always len(data) here). I suggest to drop it, this will make error message cleaner and JSONDecodeError simpler.
May be mention line and column in error message only if JSON data is multiline? Most machine generated data has one line only. Instead of "Expecting ',' delimiter: line 1 column 123 (char 124)" I suggest "Expecting ',' delimiter at position 124". Same as in re.error.