fix: Reject surrogate pairs with invalid low surrogate

[TysonAndre]

Closes #1894

Reject low surrogates outside of the range U+DC00—U+DFFF

Related to https://unicodebook.readthedocs.io/unicode_encodings.html#utf-16-surrogate-pairs

A surrogate pair should consist of a high surrogate and low surrogate. They're used to represent 0x010000-0x10FFFF in the JSON spec because the JavaScript specification originally only supported \uXXXX.

Previously, simdjson would accept some combinations of valid high surrogates and invalid low surrogates due to a bug in the check. (e.g. \uD888\u1234 was accepted)

U+D800—U+DBFF (1,024 code points): high surrogates
U+DC00—U+DFFF (1,024 code points): low surrogates

[TysonAndre]

The performance check job fails for twitter.json, which isn't using \u escapes https://github.com/simdjson/simdjson/actions/runs/3158742870/jobs/5141198629 - maybe the code size increased, or is this spurious?

1. Would using the simdjson_likely/simdjson_unlikely macros help in benchmarks for the code change?
2. Unrelatedly, this is a padded string so I'm assuming (*src_ptr)[1] points to valid memory all the time. Can the check above be combined into something along the lines of (((*src_ptr)[0] << 8) | (*src_ptr)[1]) == ('\\' << 8) | 'u') (which would get combined to loading an int16 and comparing an int64 on supported platforms?) or is there a reason to compare byte by byte.

    if (((*src_ptr)[0] != '\\') || (*src_ptr)[1] != 'u') {
      return false;
    }

[lemire]

is this spurious?

Likely spurious.

or is there a reason to compare byte by byte.

Do you think you could design a benchmark where this optimization would shine?

[TysonAndre]

Do you think you could design a benchmark where this optimization would shine?

For the combining comparisons, a string which had lots of surrogate pairs might have a tiny but consistent performance difference.

I'll look into creating a separate PR for that.

What are your thoughts on this PR?

[lemire]

What are your thoughts on this PR?

Merged.