json cannot decode non-BMP character escapes

[kmcallister]

Per RFC 4627:

To escape an extended character that is not in the Basic Multilingual Plane, the character is represented as a twelve-character sequence, encoding the UTF-16 surrogate pair. So, for example, a string containing only the G clef character (U+1D11E) may be represented as "\uD834\uDD1E".

(This is a legacy of JavaScript's UCS-2 string semantics.)

Rust's serialize::json does not decode these escapes properly:

extern crate serialize;

use serialize::json;

fn main() {
    println!("{:?}", json::from_str("\"\\ud83d\\udca9\""));
}

produces

task '<main>' failed at 'called `Option::unwrap()` on a `None` value'

[kmcallister]

I'm working on this.

[pnkfelix]

Nical has a long open PR with a rewrite of JSO to make it streaming. Might be good to coordinate changes here