Possible error message improvement (2) / additional syntax check

[HolQue]

Expecting property name enclosed in double quotes: line 3 column 4 (char 6)
caused by
param" : "value" ,

Expecting ':' delimiter: line 3 column 14 (char 16)
caused by
"param : "value" ,

Expecting value: line 3 column 14 (char 16)
caused by
"param" : value" ,

Invalid control character at: line 3 column 22 (char 24)
caused by
"param" : "value ,

Four different error messages - but the root cause is the same for all errors: a missing quote.

The JSON parser is not able to detect this properly, and for users (especially if they are not so much experienced) this is a mess.

In my opinion we cannot let the JSON parser do this job - reacting in a useful way on missing quotes.

So why not using other parser to support here? The Python internal lexical parser shlex is suitable to do this.

I suggest to implement an additional pre-check, e.g. in this way:

JSON = """
{
   "param" : "value ,
   "A" : 123 ,
   "C" : "1 and 2 and ${param}",
   "D" : "${param}"
}
"""

listLines = JSON.splitlines()
for sLine in listLines:
   try:
      listDummy = shlex.split(sLine)
   except Exception as reason:
      print()
      print("[shlex exception]\n" + str(reason) + f"\nin line: '{sLine}'")
      print()
      sys.exit()

Every missing (single) quote will cause error messages like this:

[shlex exception]
No closing quotation
in line: '   "param" : "value ,'

This would be much more helpful for users.

[namsonx]

Hello Holger,

These error message came from Python json package, and the root cause is not actually the same for all errors.
For example:

param" : "value" ,  

and

"param : "value" ,

are different root cause, in the first case the property name was missing double quotes. But in the second case it was missing a colon because json format allows colon in the property name, so it will consider "param : " is a property name and in this case a colon was missing.

For your suggestion to implement an additional pre-check, can you test with the jsonp file below:

{"str1" : "ABC", "str2" : "DEF", "var" : { "dict": { "list" : { "var_3" : "initialize"} }}, "st:r3" : "${str1}_test_${str2}test", "${str1}" : ["A", "B", "C"], "verify": ${var.dict}['list']['var_3']}

In this case, all properties name and values are placed on 1 line of jsonp file.

Thank you,
Son

[HolQue]

Hi Son,

you have the JSON interpreter in your mind. But I speak about the users perspective! A user forget - by mistake - to set a quote. At any position in the expression. That is the common root cause. That the JSON interpreter, while computing this expression, divides this into several different use cases with different error messages, is OK for the JSON interpreter, but for the user it's confusing. Therefore I still propose to add this pre check. This check does not harm anything, but provides error messages that are easier to understand.

And by the way: No user will put the entire definition of all parameters in one line of code.

[namsonx]

Hello Holger,

Additional syntax precheck method was added by the pull-request #116

The latest commit was also merged into testing branch namsonx/testing/testing_branch

Thank you,
Son

[HolQue]

Retest successful

[test-fullautomation]

solved and already merged #147