ksh93: When "read -m json" is used to read a single-line JSON object, text fields following the first numeric field are re-interpreted as numeric variable names

[zakukai]

I realize the JSON functionality in Korn Shell isn't fully mature yet, but I built myself a shell from the beta branch on git (Version ABIJM 93v- 2014-12-24) to try it out... I found that when I feed "read -m json" a JSON object with no newlines in it, a numeric field would cause all following fields to appear wrong

$ foo=9
$ read -m json json_test <<<$'{ "squanchy" : "cromulent", "num" : 1, "text": "foo", "notbool": "true", "bool": true }'
$ print -j json_test
{
	"bool": 0,
	"notbool": 0,
	"num": 1,
	"squanchy": "cromulent",
	"text": 9
}
# "squanchy" comes through just fine, because it precedes "num".
# All variables following "num" : 1 were replaced with numeric variable lookup, as if they'd appeared in $(())
$ echo "$json_test"
(
	typeset -l -E bool=0
	typeset -l -E notbool=0
	typeset -l -E num=1
	squanchy=cromulent
	typeset -l -E text=9
)
# typeset -E is a floating-point numeric type. "bool", "notbool", and "text" have all inherited this type from "num"

However, if I insert newlines into the JSON string, this doesn't happen:

$ read -m json json_test <<<$'{ "squanchy" : "cromulent", "num" : 1,\n "text": "bar", "notbool": "true", "bool": true }'
$ print -j json_test
{
	"bool": true,
	"notbool": "true",
	"num": 1,
	"squanchy": "cromulent",
	"text": "bar"
}

I haven't looked at the source code for the JSON support yet. It does appear to be in pretty rough shape overall... When I get some time I'll see what I can do with it.

[kshji]

Version AJ 93v-1203-g8d977fd3

There are more strange in read -m json. Make same read twice. Result is different. 1st reading works almost everytime ... but second reading some reading error. zakukai foo problem is also interesting.

Unset json variable not helps.

nvtree.c has some parsing problem ?

print "========================================================="
read -m json some <<EOF
{
        "squanchy" : "cromulent",
        "num" : 1,
        "text": "foo",
        "notbool": "true",
        "bool": true ,
        "endval" : "yes"
}
EOF

print "-------------------------------"
print -j some
print "-------------------------------"
print $some
print "-------------------------------"
printf "%(json)B\n" some
print "-------------------------------"
printf "%(json)q\n" $some
print "-------------------------------"
printf "%#(csv)q\n" $some
print "-------------------------------"
printf "%(csv)B\n" some

print "========================================================="
read -m json some <<EOF
{
        "squanchy" : "cromulent",
        "num" : 1,
        "text": "foo",
        "notbool": "true",
        "bool": true ,
        "endval" : "yes"
}
EOF

print "-------------------------------"
print -j some
print "-------------------------------"
print $some
print "-------------------------------"
printf "%(json)B\n" some
print "-------------------------------"
printf "%(json)q\n" $some
print "-------------------------------"
printf "%#(csv)q\n" $some
print "-------------------------------"
printf "%(csv)B\n" some

And output is

=========================================================
-------------------------------
{
        "bool": true,
        "endval": "yes",
        "notbool": "true",
        "num": 1,
        "squanchy": "cromulent",
        "text": "foo"
}
-------------------------------
( _Bool bool=true endval=yes notbool=true typeset -l -E num=1 squanchy=cromulent text=foo )
-------------------------------
{
        "bool": true,
        "endval": "yes",
        "notbool": "true",
        "num": 1,
        "squanchy": "cromulent",
        "text": "foo"
}
-------------------------------
'('
_Bool
bool=true
endval=yes
notbool=true
typeset
-l
-E
num=1
squanchy=cromulent
text=foo
')'
-------------------------------
"("
_Bool
"bool=true"
"endval=yes"
"notbool=true"
typeset
"-l"
"-E"
"num=1"
"squanchy=cromulent"
"text=foo"
")"
-------------------------------
(
        _Bool bool=true
        endval=yes
        notbool=true
        typeset -l -E num=1
        squanchy=cromulent
        text=foo
)
=========================================================
json5.sh[31]: read: line 48: ": not found
-------------------------------
{
}
-------------------------------
( )
-------------------------------
{
}
-------------------------------
'('
')'
-------------------------------
"("
")"
-------------------------------
(
)

Previosly next example didn't work, but in this version it works fine

read -m json person <<EOF
{
    "first" : "My",
    "last" : "Name",
    "email" : "My.Name@gmail.com",
    "lucky" : 13,
    "quarter" : 0.25,
    "empty" : null,
    "nerd" : true,
    "lotto" : [ 9, 12, 17, 38, 45, 46 ],
    "children" : [ "boy", "girl" ]
}
EOF



print "-------------------------------"
print "$person"
print "-------------------------------"
print -r "$person"
print "-------------------------------"
print ${person.email}
print "-------------------------------"
print ${person.lotto[*]}
print "-------------------------------"
print -j person
print "-------------------------------"

unset person

compound person
compound person=(firstname="John" lastname="Some" age=32)
print -j person
print "-------------------------------"
printf "%(json)B\n" person
print "-------------------------------"

But if you read twice same lines, second reading give error ...

I have tested built ksh

• Win10 WSL Ubuntu 18.04
• Debian 9

Same problem.

[krader1961]

I realize the JSON functionality in Korn Shell isn't fully mature yet, ....

Not only is native JSON support by ksh not mature it also has significant bugs. See issue #820. My recommendation is to remove the JSON related code. At the moment, by necessity, the focus is on a ksh release that could replace ksh93u+ from 2012 but that is much easier to maintain.

[siteshwar]

Since current implementation for json support is broken, we are going to disable it in next release. We may reintroduce it in future, but it won't be done without solid testing. Closing this issue as it does not serve any purpose now.