bytes: docs do not always state that UTF-8 encoding is used

[ghost]

Not all functions in the bytes package that operate on strings and runes specify that UTF-8 encoding is used. From an initial glance, it appears the following functions are missing that information from their documentation:

• ContainsRune
• Count
• Fields
• Runes
• Title
• ToLower
• ToLowerSpecial
• ToTitle
• ToTitleSpecial
• ToUpper
• ToUpperSpecial
• TrimSpace

Does it make sense to include that detail in each of the above functions?

[ianlancetaylor]

I do see that many of the function doc comments do explicitly say "UTF-8-encoded". It doesn't really seem necessary to me, but I guess it might be reasonable to say that a few more places.

[robpike]

The package documentation for the strings packages says, "Package strings implements simple functions to manipulate UTF-8 encoded strings." Adding that information to every function in the package is unnecessary.

On the other hand, the bytes package makes no such sweeping claim, so for the functions where that is an issue, such as ContainsRune, it would indeed be reasonable to add the information although I believe context does make it unnecessary. Go is a UTF-8 language.

[as]

@robpike
I admittedly don't know how many users get that UTF-8 replaces invalid runes in the stream, but I know I didn't when I first started learning the language. The result is more unexpected than usual in the bytes package.

The example below bytes.ToLower returns a byte slice with replacement characters when the input contains an invalid rune. The doc doesn't mention that functions like bytes.ToLower may do this, but it would be nice if it did.

https://play.golang.org/p/Qkv_hPXUwL

[gopherbot]

Change https://golang.org/cl/66750 mentions this issue: bytes: explicitly state if a function expects UTF-8-encoded data