mime: use "charset=UTF-8" instead of "charset=utf-8"

[jamshid]

What version of Go are you using (go version)?

go version go1.7 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN="/go/bin"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/root/go"
GORACE=""
GOROOT="/go"
GOTOOLDIR="/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build903236581=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

What did you do?

Not a big deal, the current code is not wrong, but please uppercase "UTF-8" in golang source.

Reference https://golang.org/src/mime/type.go, the map builtinTypesLower has some default Content-type's for file extensions like:

var builtinTypesLower = map[string]string{
...
		".xml":  "text/xml; charset=utf-8",
}

What did you expect to see?

                 ...
		".xml":  "text/xml; charset=UTF-8",

What did you see instead?

The charsets with "utf-8" should be spelled "UTF-8". The current lowercase version isn't necessarily wrong, but uppercase is more correct according to:
https://blog.codingoutloud.com/2009/04/08/is-utf-8-case-sensitive-in-xml-declaration/

This came up because the java server jetty uppercases the "utf-8" in the incoming request's Content-type charset before the servlet starts. Normally this isn't a problem, but it breaks S3-like signature verification, where client and server sign properties of a request (e.g. the Content-type header).

[bradfitz]

Normally this isn't a problem, but it breaks S3-like signature verification, where client and server sign properties of a request (e.g. the Content-type header).

Elaborate, please.

Changing this arbitrarily seems just as likely to break other people (and lots of tests!). We would need a very good reason to change this.

[jamshid]

The scenario is a golang client (e.g. https://github.com/ncw/rclone/) makes a request to a Jetty server. They both use AWS S3-like signatures to authenticate the request. I.e. both the client and server compute a hash using various request headers and sign the hash with a shared "secret" to produce a signature.

The client signature includes Content-type "text/xml; charset=utf-8" but Jetty (wrongly, IMO) normalizes the charset so the servlet sees "text/xml; charset=UTF-8". This causes the signatures to not match.

This can be worked around with some Jetty hacking, e.g. to preserve the original Content-type, but as the link above shows the official spellings of the charset is "UTF-8", not "utf-8", so seems preferable to use that.

But you could argue that http://www.iana.org/assignments/character-sets/character-sets.xhtml does say "However, no distinction is made between use of upper and lower case letters", so if you really don't want to change golang types.go I guess that's not unreasonable.

[bradfitz]

@ncw, can you elaborate? What's at fault, here? Is the "S3-like" signature algorithm just poorly designed? Is this a specified protocol, or something that rclone invented?

[jamshid]

Just to be clear I'm not saying anything is at fault. Well, except maybe Jetty, it should not normalize the "charset" in the request header "Content-type".

The S3 authentication protocol
http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html requires the client and server to use the exact same Content-type value when computing the signature.

That's not possible when a golang client sends ...;charset=utf-8 to an S3-compatible server based on Jetty, which quietly changes the header value to ...;charset=UTF-8.

Of course this can (and should) be fixed or worked around in the Jetty server. I'm not saying golang is wrong to use "utf-8", just that it might as well use the official charset spelling "UTF-8" instead. Of course, changing to that spelling now might not be worth the trouble.

[bradfitz]

Okay, then I'll close this. Changing it would break people's tests unnecessarily if the bug is in Jetty.