Assigning values to a json structur

[gonwalf]

Hi!

I'm testing rl_json right now. Parsing JSON strings and getting values works fine. But when I try to set values, I'm having some difficulties. Consider the following code:

package require rl_json
set jsonstr {
    {
        "foo": "Foo",
        "baz": "Baz"
    }
}

json set jsonstr bar Bar FooBar test {"x"}
puts [json normalize $jsonstr]

json set jsonstr bar Bar FooBar test2 {"y"}
puts [json normalize $jsonstr]

json set jsonstr bar Bar FooBar test3 {"z"}
puts [json normalize $jsonstr]

I'd expect the following end result:

{"foo":"Foo","baz":"Baz","bar":{"Bar":{"FooBar":{"test":"x","test2":"y","test3":"z"}}}}

Instead I get this result:

{"foo":"Foo","baz":"Baz","bar":{"Bar":{"FooBar":{"test":"x","test2":null,"bar":{"Bar":{"FooBar":{"test3":"y"}}},"test3":null}}}}

Another problem I have is changing a value, e.g:

package require rl_json

set jsonstr {
    {
        "foo": "Foo",
        "baz": "Baz"
    }
}

json set jsonstr foo {"Bar"}
puts [json normalize $jsonstr]

exits with the error message

missing value to go with key

The speed of the implementation is very impressive. If setting values would work, JSON would really feel like a native datatype of TCL.

[cyanogilvie]

Hi Gonwalf Sorry for the delay in replying. [json set] and [json merge] are still WIP. I left them undocumented but people seem to have been pretty good at finding them :) I'm having a bit of trouble getting them to work correctly for certain cases, but the ones you give shouldn't be a problem, and look like bugs in the path traversal code used by [json set]. I can probably fix those quickly but some of the other issues are trickier and might require replacing libyajl with a different or custom parser, and possibly changing the syntax of the set method. I'll have a go at fixing these issues but beware that there are some nasties lurking there still. Cyan

[gonwalf]

Hi Cyan!

Thank you for the info. I've been searching for quite some time for a tcl json lib which is fast and allows full round trips of json data. I've written a json to huddle parser for yajl-tcl which is quite fast. Unfortunately the json generation from huddle is extremely slow.

I'll be happy to test your fixes as soon as they are checked in.

gonwalf

[cyanogilvie]

I've pushed some changes that should fix those two cases and a number of
other [json set] related problems, but there is still a problem using it to
set a bare numeric value, which appears to be from the yajl library
refusing to parse that as a json val (it should, if it already parses the
other bare atomic types - strings, bools and nulls). Test "set-1.2"
exercises this case.

Indexing into arrays lacks test coverage and may contain problems (the code
to handle the path traversal for [json set] is really nasty, hard to spot
problems).

If you play with it and find any issues besides the numeric one, let me
know and I'll take a look. The numeric issue might require a replacement
for yajl or custom code to handle that case, or changing yajl's behaviour
(not likely - it's very widely used).

Cyan

On Wed, Apr 22, 2015 at 3:31 PM, gonwalf notifications@github.com wrote:

Hi Cyan!

Thank you for the info. I've been searching for quite some time for a tcl
json lib which is fast and allows full round trips of json data. I've
written a json to huddle parser for yajl-tcl which is quite fast.
Unfortunately the json generation from huddle is extremely slow.

I'll be happy to test your fixes as soon as they are checked in.

gonwalf

—
Reply to this email directly or view it on GitHub
#2 (comment).

[cyanogilvie]

The numeric issue comes from a yajl bug (lloyd/yajl#95). I've got a small patch that fixes the issue so I'll create a pull request, but there are a lot of open pull requests so I don't know if it will be merged or how long it will take. A dirty work-around for now is to add a space or newline to the end of the number:

json set jsonstr foo "42 "

My yajl fork with the change: https://github.com/cyanogilvie/yajl

[cyanogilvie]

With the new parser implemented the libyajl dependency and bugs are removed. [json set] and [json unset] are implemented so it is now possible to fully manage JSON values in-place.

[json merge] is not implemented (removed that method) since the behaviour is difficult to define for all possible input conditions, and would lead to surprising results at best.

[gonwalf]

We've tested the new version quite extensively and everything works as expected and fast. JSON now feels like a native datatype. This should be an official tcl module!

[cyanogilvie]

Thanks :)

On Wed, Feb 24, 2016 at 1:25 PM, gonwalf notifications@github.com wrote:

We've tested the new version quite extensively and everything works as
expected and fast. JSON now feels like a native datatype. This should be an
official tcl module!

—
Reply to this email directly or view it on GitHub
#2 (comment).