x/website: redirect golang.org/pkg and golang.org/cmd to pkg.go.dev #44356
Thanks to significant work by @julieqiu and team on pkg.go.dev, godoc.org now redirects to pkg.go.dev, and we've now reduced the number of documentation servers run by the Go team from three to two. If pkg.go.dev is ready now to serve godoc.org's traffic, then it's also ready to serve golang.org's documentation traffic. We should do that, reducing the number of documentation servers from two to one (the ideal number!).
One concern we had was whether there are request parameters that pkg.go.dev does not support. I analyzed the logs, and the answer is no, there aren't. The most important ones are GOOS= and GOARCH=, and those work on pkg.go.dev for the standard library (other modules in progress).
The only other parameter with any frequency is ?m=all, which displays unexported API. We do not plan to add that feature to pkg.go.dev - it would expand the index significantly without creating much value, if any, for users. But most of the ?m=all requests are coming from search engine results that have inadvertently indexed that form of the page. Those would be served just as well (or better) by only showing exported API.
There is about one request per day using m=all to link to unexported functions, or about 0.0007% of traffic. That one request per day will break with the redirect, and that seems an acceptable cost for cutting our doc server count by 50%.
Query parameters overall.
Downloaded 10,000 request URLs from App Engine logs matching "golang.org/pkg", spanning 96 minutes (1.7 QPS).
Only 39 of the 9,192 non-bot requests (0.42%) had query parameters.
21 used ?m=all to see unexported data; all were coming from www.google.com results
9 fetched /pkg/syscall/?GOOS=windows.
3 used ?source=post_page---------------------------, which does nothing.
Downloaded 500 request URLs matching "/pkg.*m=", spanning 41 hours (0.0034 QPS).
Of the 11 non-bot, non-m=all requests:
Downloaded 500 request URLs matching /GOOS/, spanning 53 hours (0.0026 QPS).
The text was updated successfully, but these errors were encountered:
How would pkg.go.dev work for access tip.golang.org docs? As far as I can tell there’s currently no way to do that.
Also I’m clearly very much in the minority here, but I happen to use ?m=all quite often and would be really sad to see it go. I find it very useful when working on my own code, and also when poking around the standard library to understand why something works or doesn’t, or how things are implemented.
This hasn't happened yet because I realized that it would break visitors using golang.google.cn, because pkg.go.dev is not visible in China.
The new plan is to redirect by default but allow ?m=old to access the old docs. And requests from China will get ?m=old automatically.
The README is no longer displayed at pkg.go.dev/std. It is also no longer stored in the database when a std module is fetched. For golang/go#44356 Change-Id: I6b1605389bc3c97694b6ec2b06e28006a38be999 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296950 Trust: Julie Qiu <firstname.lastname@example.org> Reviewed-by: Jonathan Amsterdam <email@example.com>
Packages in cmd/ are now omitted from the directory listing on pkg.go.dev/std, as in the case of golang.org/pkg. For golang/go#44356 Change-Id: I160965d1b914a11863a815db3184d8b9823e8ae1 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296951 Trust: Julie Qiu <firstname.lastname@example.org> Reviewed-by: Jonathan Amsterdam <email@example.com>
The following is refactored: * The UnitPage fields Subdirectories and NestedModules are removed, since they are not needed. UnitPage.Directories is used to determine if the directories section should be shown. * The directories section now displays the internal directory at the bottom of the page. For golang/go#44356 Change-Id: I3e6f7e3cacf27e129d6d7ea78d7a89e6e58e4be7 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296953 Trust: Julie Qiu <firstname.lastname@example.org> Reviewed-by: Jonathan Amsterdam <email@example.com>
Internal sudirectories that are not at the top level of the unit are no longer displayed in directories section to reduce noise in this section. For golang/go#44356 Change-Id: Id4c141f215e4fdab61e57361e5d265594eb21bd6 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/297109 Trust: Julie Qiu <firstname.lastname@example.org> Run-TryBot: Julie Qiu <email@example.com> Reviewed-by: Jonathan Amsterdam <firstname.lastname@example.org>