Support for UTF-16 surrogate pair encoded emojis

[tech4him1]

Currently, if I try to parse YAML data containing Unicode emojis split into UTF-16 surrogate pairs (i.e. 1F468 as \uD83D\uDC68 in YAML), go-yaml returns the error "found invalid Unicode character escape code".

According to the YAML spec parsers are supposed to support UTF-8 and UTF-16, including surrogate pairs:
http://www.yaml.org/spec/1.2/spec.html#id2770814
http://www.yaml.org/spec/1.2/spec.html#id2771184

This looks intentional, are you planning on supporting these, or not?

yaml/scannerc.go

Lines 2443 to 2447 in 25c4ec8

	if (value >= 0xD800 && value <= 0xDFFF) || value > 0x10FFFF {
	yaml_parser_set_scanner_error(parser, "while parsing a quoted scalar",
	start_mark, "found invalid Unicode character escape code")
	return false
	}

[Destroy666x]

Still not supported 😞

[andreif]

I had similar problem somewhere in my pipeline and thought it's because of this issue, but it looks like it works? Or am I reading it wrong:

[andreif]

[andreif]

It looks like it almost works -- it does if the data is just a string, but does not if the string is in arrays or objects.

[steveh]

I've created a PR to add support for surrogate pairs: #1029