Tracking issue for `str_utf16` stabilization

[aturon]

The str_utf16 feature tracks APIs like str::utf16_units that support utf16 encoding. These APIs are working well today, but it would be good to have a clear picture of the final state of APIs we'd like to have for various string encodings in std, and how that relates to other unicode support crates.

cc @SimonSapin

[SimonSapin]

Regarding naming: utf16_units is not a good name. As far as I can tell, the choice of "code unit" in the Unicode Standard is mostly “what’s another word for "thing" that we haven’t used yet?” I suggest renaming str::utf16_units to str::encode_utf16 (and its return type accordingly, if it stays around at all).

Regarding the API: &str -> Iterator<Item=u16> in the middle of the balance between terseness and generality:

• If what you want is a Vec<u16>, having the method return an iterator makes you write "redundant" .collect() (or even .collect::<Vec<_>>()).
• If what you have is an iterator of char, collecting it into a String is wasteful as the only thing the method does with &str is get a str::Chars iterator out of it.

I suggest changing str::encode_utf16 to go all the way to the "convenience" side and make it return a Vec<u16>. (This is symmetrical with String::from_utf16* where neither input or output is an iterator.) char::encode_utf16 (#27784) can be used directly to avoid extra allocations in cases where a different type of input or output is desired.

[aturon]

While this still isn't ready for stabilization, I'm nominating for libs team discussion about @SimonSapin's proposals.

[retep998]

I've seen several people try to use this rather than OsStrExt::encode_wide when trying to work with Windows API. Perhaps some sort of warning about this would help.

[SimonSapin]

@alexcrichton why remove the needs-decision tag? What do you think of the changes proposed in my previous comment?

[alexcrichton]

Ah I misinterpreted @aturon's original intention, but the libs team decided yesterday to talk about this during the next triage meeting, and I-nominated is now our best way to do that, so I'm gonna leave as is :)

[aturon]

Libs team consensus: we agree with the proposed renaming, but feel that the current signature (returning an iterator) is the most consistent with the design elsewhere in the standard library, and that on balance the flexibility of an iterator outweighs the minor pain of writing .collect() in most uses. We return Vec very very rarely in the standard library today.

[alexcrichton]

🔔 This issue is now entering its cycle final comment period to be stabilized in 1.8 🔔

The libs team would like to rename the method to encode_utf16, but beyond that we have no changes planned

[alexcrichton]

The libs team discussed this during triage yesterday and the conclusion was to stabilize with the name encode_utf16.