net/http: discrepancies in the MIME Sniffing Spec and the implementation

[kshitij10496]

What version of Go are you using (go version)?

$ go version
go version devel +d71d81c5a2 Mon Mar 4 16:29:25 2019 +0530 darwin/amd64

While reading the MIME Sniffing Spec and the corresponding implementation in the net/http package in sniff.go, I found few discrepancies between the two. As, I do not know whether these diffs are intentional or not (though I am curious to know), I thought it best to share my findings so that a conscious decision can be taken.

Here are few suggestions for updating the current implementation in accordance with the spec:

1. Using pattern rather than mask for assertion and iteration in the Pattern Matching Algorithm.
   Currently, we are using m.mask instead of m.pat at both the places.
   The reason this substitution works is because their lengths are the equal (implicit) in all the currently support MIME signatures, and the logical AND operation: a & b = c => a & c = b.
   Refer:

   go/src/net/http/sniff.go

   Lines 185 to 194 in d188767

   	if len(data) < len(m.mask) {
   	return ""
   	}
   	for i, mask := range m.mask {
   	db := data[i] & mask
   	if db != m.pat[i] {
   	return ""
   	}
   	}
   	return m.ct

2. Rename image/vnd.microsoft.icon to image/x-icon and add another signature for the same.
   We are using the same signature for image/vnd.microsoft.icon instead of image/x-icon. This could be a possible bug or a decision that should be documented as a divergence from the spec.
   Refer:

   go/src/net/http/sniff.go

   Line 93 in d188767

   &exactSig{[]byte("\x00\x00\x01\x00"), "image/vnd.microsoft.icon"},

3. Using named strings instead of hexadecimal representation in application/zip and application/x-rar-compressed.
   Reading the source code, I experienced that using human-readable string definition of the signatures is an implicit convention over the mechanistic byte pattern sequences (as it should be!). These are a couple of places which can be changed with the sole purpose of maintaining consistency and improving visual aesthetics.
   Refer:

   go/src/net/http/sniff.go

   Lines 146 to 147 in d188767

   	&exactSig{[]byte("\x52\x61\x72\x20\x1A\x07\x00"), "application/x-rar-compressed"},
   	&exactSig{[]byte("\x50\x4B\x03\x04"), "application/zip"},

4. Reordering the sniffSignatures according to the section Identifying a resource with an unknown MIME type.
   In the current implementation, the signatures are almost sorted according to the spec, with a few exceptions. For example, matching video/webm before video/mp4.

[agnivade]

/cc @bradfitz @odeke-em

[bradfitz]

From memory, I don't think any of this is intentional. @odeke-em, you want to handle this? You tend to work on this code most frequently lately.

[kshitij10496]

I would be happy to fix this and make my first contribution to the project! 😃

[odeke-em]

Thank you for filing this issue @kshitij10496 and welcome to the Go project! Awesome, please go for it and I am looking forward to reviewing your CL.

[gopherbot]

Change https://golang.org/cl/165277 mentions this issue: net/http: remove discrepancies between the MIME Sniffing Spec and its implementation