net/http: sniff.go DetectContentType failing to detect text/html correctly #16275

turbodonkey · Jul 6, 2016

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?
go version go1.6.2 darwin/amd64
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH=""
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GO15VENDOREXPERIMENT="1"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"
What did you do?

package main

import (
    "fmt"
    "net/http"
    "io/ioutil"
)

func main() {
    fp, err := ioutil.ReadFile("test.html")
    if err != nil {
        fmt.Printf("error opening file %s\n", err)
        return
    }
    fmt.Printf("Content-type %s\n", http.DetectContentType(fp))
}

With test.html being:

<!--
this is a dummy comment
-->

<html>
        this is a simple html page
</html>

output: Content-type text/plain; charset=utf-8

What did you expect to see?

output: Content-type text/html; charset=utf-8

Basically DetectContentType should detect the '<!--' at the start of the .html file, and it does not. As soon as I remove the '<!--' tag, then the file is correctly detected as text/html

What did you see instead?

Content-type text/plain; charset=utf-8

fraenkel · Jul 6, 2016

How do you know it isn't an XML document?

davecheney · Jul 6, 2016

A valid HTML, XHTML, XML, or HTML4 document must not start with anything
except an opening element, ie , , etc. <!-- is a comment, it
is not permitted at the top level.

On Wed, Jul 6, 2016 at 11:25 PM, Michael Fraenkel notifications@github.com
wrote:

How do you know it isn't an XML document?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#16275 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAAcA8ns3OtZngMLqVzvgi4SUEWEGY88ks5qS6y2gaJpZM4JF8xK
.

turbodonkey · Jul 6, 2016

ok you can take the spec / standard high road on this, or we could improve the code. right now, '<!--' is considered text/html (from sniff.go):

// Data matching the table in section 6.
var sniffSignatures = []sniffSig{
    htmlSig("<!DOCTYPE HTML"),
    htmlSig("<HTML"),
    htmlSig("<HEAD"),
    htmlSig("<SCRIPT"),
    htmlSig("<IFRAME"),
    htmlSig("<H1"),
    htmlSig("<DIV"),
    htmlSig("<FONT"),
    htmlSig("<TABLE"),
    htmlSig("<A"),
    htmlSig("<STYLE"),
    htmlSig("<TITLE"),
    htmlSig("<B"),
    htmlSig("<BODY"),
    htmlSig("<BR"),
    htmlSig("<P"),
    htmlSig("<!--"),

The issue is in this line of htmlSig:

    // Next byte must be space or right angle bracket.
    if db := data[len(h)]; db != ' ' && db != '>' {
        return ""
    }

My html files do not have a trailing ' ' against '<!--', they have a newline instead.

bradfitz · Jul 6, 2016

ok you can take the spec / standard high road on this, or we could improve the code

We're going to follow the spec. If we don't draw the line somewhere then DetectContentType will be under constant churn and feature requests.

Please file a bug at https://github.com/whatwg/mimesniff/issues if you disagree with the spec. Once fixed upstream, we'll fix Go.

Until then, unless there's some place where we're in violation of the mimesniff spec, I'm going to close this bug.

turbodonkey · Jul 7, 2016

Thats fair, I'll chase with mimesniff! Thanks for the immediate response all.

ianlancetaylor added this to the Go1.8 milestone Jul 6, 2016

bradfitz closed this Jul 6, 2016

golang locked and limited conversation to collaborators Jul 7, 2017

gopherbot added the FrozenDueToAge label Jul 7, 2017

golang/go

net/http: sniff.go DetectContentType failing to detect text/html correctly #16275

net/http: sniff.go DetectContentType failing to detect text/html correctly #16275

turbodonkey commented Jul 6, 2016 •

edited

ianlancetaylor added this to the Go1.8 milestone Jul 6, 2016

This comment has been minimized.

fraenkel commented Jul 6, 2016

This comment has been minimized.

davecheney commented Jul 6, 2016

This comment has been minimized.

turbodonkey commented Jul 6, 2016

This comment has been minimized.

bradfitz commented Jul 6, 2016

bradfitz closed this Jul 6, 2016

This comment has been minimized.

turbodonkey commented Jul 7, 2016

golang locked and limited conversation to collaborators Jul 7, 2017

gopherbot added the FrozenDueToAge label Jul 7, 2017

golang/go

Join GitHub today

net/http: sniff.go DetectContentType failing to detect text/html correctly #16275

Comments

turbodonkey commented Jul 6, 2016 • edited

ianlancetaylor added this to the Go1.8 milestone Jul 6, 2016

This comment has been minimized.

fraenkel commented Jul 6, 2016

This comment has been minimized.

davecheney commented Jul 6, 2016

This comment has been minimized.

turbodonkey commented Jul 6, 2016

This comment has been minimized.

bradfitz commented Jul 6, 2016

bradfitz closed this Jul 6, 2016

This comment has been minimized.

turbodonkey commented Jul 7, 2016

golang locked and limited conversation to collaborators Jul 7, 2017

gopherbot added the FrozenDueToAge label Jul 7, 2017

turbodonkey commented Jul 6, 2016 •

edited