Yajl fails to decode null characters in strings #45

Open
vapir opened this Issue Feb 24, 2011 · 4 comments

3 participants

@vapir

Yajl fails to decode null characters in json-encoded strings, silently discarding them.

Yajl::Parser.parse("\"\\u0000\"")

results in "" when it should be "\000".

JSON gem gets this right, for example (array added because JSON gem doesn't like parsing strings on their own):

>> JSON.parse("[\"\\u0000\"]")
=> ["\000"]

Whatever else is in the string, the null bytes are just discarded:

>> Yajl::Parser.parse("\"foo\\u0000bar \\u0000\"")
=> "foobar "
>> JSON.parse("[\"foo\\u0000bar \\u0000\"]")
=> ["foo\000bar \000"]
@brianmario
Owner

Dammit my comment was lost in the transfer...

Anyway I had a few examples in here of how Chrome, Safari and Firefox handle this. Chrome and Safari act like Yajl whereas Firefox acts like the JSON gem.

Also, the output from nodejs:

> JSON.parse("[\"foo\\u0000bar \\u0000\"]")
[ 'foo\u0000bar \u0000' ]

Will research more and get back to you.

@ConradIrwin

Chrome is bad at displaying strings containing the null byte (it displays them as if it wasn't there). It certainly doesn't strip them when it parses JSON:

JSON.parse("\"\u0000\"") //-> "" [the null byte is invisible in chrome]
JSON.parse("\"\u0000\"") == "\u0000" //-> true
JSON.parse("\"\u0000\"") == "" //-> false [actually an empty string]

@ConradIrwin

I've sent a patch to YAJL to fix this lloyd/yajl#19 — would you like me to create a pull request for here too? (I'm not sure how you go about keeping the copy of yajl you have in sync with upstream)

@vapir

saw that pull request was merged. thanks very much for the fix, Conrad. I hope it makes it into yajl-ruby soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment