Standard library packages not being returned for simple queries

[myitcv]

Picking up on #505 (comment) I'm still seeing issues where a search query with the term that is the path.Base of a standard library package import path fails to return the standard library package in the list of results.

For example the query for math:

https://godoc.org/?q=math

Doesn't appear on the entire results page, let alone near the top

This was also the case for strings and os (as reported in #505 (comment)) but they now appear to be fixed (thanks @shantuo).

Doing a quick analysis of the Go 1.9 standard library, the following packages appear to suffer from this problem:

• bytes (https://godoc.org/?q=bytes)
• compress/flate (https://godoc.org/?q=flate)
• crypto/rand (https://godoc.org/?q=rand)
• crypto/sha1 (https://godoc.org/?q=sha1)
• crypto/sha256 (https://godoc.org/?q=sha256)
• crypto/tls (https://godoc.org/?q=tls)
• encoding/gob (https://godoc.org/?q=gob)
• go/importer (https://godoc.org/?q=importer)
• math/bits (https://godoc.org/?q=bits)
• net (https://godoc.org/?q=net)
• net/http (https://godoc.org/?q=http)
• reflect (https://godoc.org/?q=reflect)
• runtime/cgo (https://godoc.org/?q=cgo)
• runtime/race (https://godoc.org/?q=race)

List compiled using script:

for i in $(go list std | grep -v internal | grep -v ^vendor\/)
do
        b=$(basename $i)
        r=$(curl -s --fail https://godoc.org/?q=$b | pup "table[class=\"table table-condensed\"] tbody tr td a[href=\"/$i\"] text{}")
        d=0
        if [ "$r" == "" ]
        then
                echo "* $i (https://godoc.org/?q=$b)"
        fi
done

I'm fairly certain that something has changed here?

Thanks

[myitcv]

Updated script and list above... and in that time math appears to have fixed itself... which is strange.

[shantuo]

These packages are suppressed from the search result page because of the previous bug #504, I guess they are still in queue to be updated. I manually refreshed package math and it shows up now. I can try to manually bump the crawl of these packages.

[myitcv]

Thanks @shantuo

[shantuo]

Some works now, there are a few (net/http, crypto/rand ...) still not working because gddo cannot generate the doc due to some "unrecognized import path" error. Perhaps something is wrong with the build environment/deployment.

[dmitshur]

From https://godoc.org/net/http:

The go get command cannot install this package because of the following issues:

• Unrecognized import path "golang_org/x/net/proxy" (transport.go:32:2)
• Unrecognized import path "golang_org/x/net/idna" (h2_bundle.go:47:2)
• Unrecognized import path "golang_org/x/net/http2/hpack" (h2_bundle.go:46:2)
• Unrecognized import path "golang_org/x/net/lex/httplex" (server.go:31:2)

Those are vendored packages:

https://github.com/golang/go/tree/master/src/vendor/golang_org/x/net

Perhaps the cause is that gddo doesn't support finding vendored packages (this is a guess, it might be something else).

[shantuo]

The vendored packages should be there after the std lib list was updated. There must be some other bug.