Add u16::is_utf16_surrogate

[clarfonthey]

Right now, there are methods in the standard library for encoding and decoding UTF-16, but at least for the moment, there aren't any methods specifically for u16 to help work with UTF-16 data. Since the full logic already exists, this wouldn't really add any code, just expose what's already there.

This method in particular is useful for working with the data returned by Windows OsStrExt::encode_wide. Initially, I was planning to also offer a TryFrom<u16> for char, but decided against it for now. There is plenty of code in rustc that could be rewritten to use this method, but I only checked within the standard library to replace them.

I think that offering more UTF-16-related methods to u16 would be useful, but I think this one is a good start. For example, one useful method might be u16::is_pattern_whitespace, which would check if something is the Unicode Pattern_Whitespace category. We can get away with this because all of the Pattern_Whitespace characters are in the basic multilingual plane, and hence we don't need to check for surrogates.

[rust-highfive]

r? @scottmcm

(rust-highfive has picked a reviewer for you, use r? to override)

[ChrisDenton]

We can get away with this because all of the whitespace characters are in the basic multilingual plane, and hence we don't need to check for surrogates

Is that guaranteed or could that change in the future?

[scottmcm]

We can get away with this because all of the whitespace characters are in the basic multilingual plane, and hence we don't need to check for surrogates

I agree with @ChrisDenton here -- this doesn't seem like a safe assumption to bake into an API. People can convert it to a char themselves if they want to encode that assumption.

But it doesn't need to block this PR.

[scottmcm]

Given all the _ascii_ methods on u8, this seems like a reasonable thing to add.

However, I think it needs a different name. A char can never be a surrogate, so mentioning it seems wrong.

I think the _ascii_ names and from_utf8 and such mean that it should mention the encoding it's using, not a datatype.

So how about this?

Suggested change

	pub const fn is_char_surrogate(self) -> bool {
	pub const fn is_utf16_surrogate(self) -> bool {

[clarfonthey]

You're right; I was originally thinking of going with is_unicode_surrogate but decided it was too long.

My main apprehension with calling it utf16 surrogate is it implies that it's specific to UTF-16, which it actually isn't; it's specific to Unicode, even though its inclusion in Unicode is specific for UTF-16. I will agree that the name is better than the original, though.

[scottmcm]

Hmm, this makes me think that the API might want to be -> Option<bool> or something to distinguish high/low surrogates.

This code looks like it would be better written with exhaustive range patterns today (they almost certainly didn't exist when it was written, though). So the nice code change, to me, would be one that could match c.tell_me_about_surrogate_ness() instead.

[clarfonthey]

So, I was debating on how exactly to introduce these APIs, but what I was thinking is a good logical step is some form of check whether a surrogate is a low or high surrogate, or a way to combine surrogates into a code point. But at least that last point is mostly covered by decode_utf16, so, I wasn't sure what extent was useful.

Either way, there is definitely room for expansion.

[clarfonthey]

We can get away with this because all of the whitespace characters are in the basic multilingual plane, and hence we don't need to check for surrogates

Is that guaranteed or could that change in the future?

So, I was partially right; although the White_Space property is open to expansion despite how unlikely this is, Pattern_White_Space (which is larger than just ASCII whitespace, but smaller than general White_Space) is guaranteed by the standard to never include future characters, and is strictly within the BMP. So, this could be a reasonable addition in the future.

[clarfonthey]

Renamed, rebased, fixed build errors (hopefully), and opened a tracking issue since this seems desired.

[clarfonthey]

@rustbot ready

[scottmcm]

This looks good for nightly to me!

@bors r+

[bors]

📌 Commit d580367 has been approved by scottmcm