New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc, x/tools/cmd/godoc: move golang.org content into its own repository #29206

Open
andybons opened this Issue Dec 13, 2018 · 17 comments

Comments

Projects
None yet
7 participants
@andybons
Copy link
Member

andybons commented Dec 13, 2018

With the website slated for a refresh, we should move the static content out of the main go source tree (https://go.googlesource.com/go/+/master/doc/) and into its own repository (web? website?).

This will reduce binary and source code download sizes (#27151) and allow us to apply changes unrelated to our release schedule without having to cherry-pick them to a release branch. Hopefully we can also implement continuous delivery as part of this as well.

We also won’t have to worry about bloating the main go source tree as we add more assets and content for the refresh.

We still only want to display package documentation and source files from a GOROOT pinned to the current stable release (and not show content only relevant for future releases like release notes). So there are elements that still require logic that ties content to a specific Go version. What we consider fluid vs tied to a release is to be determined and can be fleshed out here.

/cc @bradfitz @ianlancetaylor @julieqiu @katiehockman @dmitshur @FiloSottile @cnoellekb

@bradfitz

This comment has been minimized.

Copy link
Member

bradfitz commented Dec 13, 2018

"website" SGTM. I like the repo name golang/website.

Inside that I'd like something like:

  • README.md
  • codereview.cfg
  • content/ directory
  • cmd/ directory
  • cmd/goweb or cmd/golangweb or cmd/golang-org-server (name TBD)

We could start with mirroring stuff over from the existing golang/go repo (and any necessary code from x/tools/cmd/godoc) but without modifying either for now. And then serve the new one on something like https://new.golang.org for a bit and when we're happy then switch over and start deleting code & assets from the old locations.

We also need to figure out whether this new cmd/golang-org-server is the new name of the recently-web-only x/tools/cmd/godoc (the tool for users to browse their local $GOPATH/etc documentation in the web on localhost on airplanes/etc).

If they're the same, then do we develop it in x/tools or in x/website? I kinda like having the code next to the assets but I could go either way and haven't thought about it much.

@andybons

This comment has been minimized.

Copy link
Member

andybons commented Dec 13, 2018

We also need to figure out whether this new cmd/golang-org-server is the new name of the recently-web-only x/tools/cmd/godoc (the tool for users to browse their local $GOPATH/etc documentation in the web on localhost on airplanes/etc).

My preference is for x/tools/cmd/godoc to become a binary for viewing local $GOPATH/etc documentation but not anything else (which would be more in line with its name anyhow). A common go-source-to-html rendering library could be pulled out (if it’s not already general enough in x/tools/godoc) and used in places like godoc.org, x/tools/cmd/godoc, and the new website. This way most of the logic specific to new.golang.org is right next to the content.

@dmitshur

This comment has been minimized.

Copy link
Member

dmitshur commented Dec 13, 2018

We also need to figure out whether this new cmd/golang-org-server is the new name of the recently-web-only x/tools/cmd/godoc (the tool for users to browse their local $GOPATH/etc documentation in the web on localhost on airplanes/etc).

Yes, we need to figure that out. At this time, I strongly suspect that we’ll want to make the new cmd/golang-org-server be strictly the website and not a tool users run locally, other than to contribute to the golang.org website development.

@bradfitz

This comment has been minimized.

Copy link
Member

bradfitz commented Dec 13, 2018

Pulling the render-godoc-as-HTML handler out and using it on both places SGTM. That can happen first probably.

And making x/tools/cmd/godoc be super minimal also SGTM.

@cnoellekb

This comment has been minimized.

Copy link

cnoellekb commented Dec 20, 2018

We discussed 4 main milestones for moving the website:

  1. Separate out rendering logic (converting code -> HTML) from x/tools/cmd/godoc. There should be no behavior change at this point.
  2. Creating website binary and serving templates (the static content) from go/doc to x/website
  3. Using the common rendering library in x/website.
  4. Reduce x/tools/cmd/godoc to be the minimal version (since the website can now be run out of x/website).

Right now I'm focusing on the second step because I was able to get a prototype up and running. I copied tools/cmd/godoc, tools/godoc, and go/doc into a local directory that will eventually become the root for the main website. The plan is to just copy everything for now to maintain x/tools functionality. Can someone help me set up an x/website repo?

Also, I was looking into how the rest of the tools directory would be affected by removing any godoc files and I found that x/tools/cmd/present/play.go uses the tools/godoc/static package. Does anyone think this may be an issue?

Lastly, is the final version of godoc going to need an app engine configuration? If not, then tools/internal/memcache can be moved to the x/website repository.

@bradfitz

This comment has been minimized.

Copy link
Member

bradfitz commented Dec 21, 2018

@dmitshur, @andybons, where are the notes from last time we created a new repo? I'd rather not do it myself. It'd be nice if we can have somebody else run through the steps from the docs. (gerrit, github, maintner, gitmirror updating, etc)

@dmitshur

This comment has been minimized.

Copy link
Member

dmitshur commented Dec 21, 2018

where are the notes from last time we created a new repo?

@bradfitz The steps are documented at golang.org/wiki/CreatingSubRepository.

It'd be nice if we can have somebody else run through the steps from the docs.

Yep, that's a good idea. If anyone on the team wants to go through the process to learn it, I'm happy to assist and fill in any gaps in docs. Note that most of the bottlenecks will be having the right permissions to all components involved. I can also do it myself.

@bradfitz

This comment has been minimized.

Copy link
Member

bradfitz commented Jan 2, 2019

I created https://github.com/golang/website and https://go.googlesource.com/website (Gerrit).

@cnoellekb, can you send CLs for the maintner and dashboard and gitmirror bits? And gopherbot if needed.

@cnoellekb

This comment has been minimized.

Copy link

cnoellekb commented Jan 3, 2019

Sure, I'll take care of it!

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 3, 2019

Change https://golang.org/cl/156217 mentions this issue: maintner/maintnerd, cmd/gerritbot, cmd/gitmirror: add golang/website to

gopherbot pushed a commit to golang/build that referenced this issue Jan 3, 2019

maintner/maintnerd, cmd/gerritbot, cmd/gitmirror: add golang/website …
…to whitelist

The website now has its own subrepo with gerrit mirroring set up.

Updates golang/go#29206

Change-Id: I6fb6fb62dfd50b48d2f78db2503641c521600ae7
Reviewed-on: https://go-review.googlesource.com/c/156217
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 4, 2019

Change https://golang.org/cl/156337 mentions this issue: cmd/godoc: add x/website redirect

gopherbot pushed a commit to golang/tools that referenced this issue Jan 4, 2019

cmd/godoc: add x/website redirect
Add a redirect for the recently created x/website subrepository.

It's not yet included at https://golang.org/pkg/#subrepo because it's
in development. Once development reaches the point that x/website
is the canonical location of the golang.org server, we can consider
including x/website at https://golang.org/pkg/#subrepo (just like
x/blog, x/tour, x/build are already included there).

Updates golang/go#29206

Change-Id: I6889c1f5e40f11abca944b217a7354f76c08c8eb
Reviewed-on: https://go-review.googlesource.com/c/156337
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 4, 2019

Change https://golang.org/cl/156321 mentions this issue: x/website: add code and static files for website

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 4, 2019

Change https://golang.org/cl/156326 mentions this issue: x/website: add code and static files for website

gopherbot pushed a commit to golang/website that referenced this issue Jan 8, 2019

x/website: add code and static files for website
All of the code and static assets that the website uses to run have been
copied to this repo. There was also a few lines of code added telling
the website where the doc directory, favicon.ico and robots.txt are.

go repo change-id: Ife6443c32673b38000b90dd2efb2985db37ab773
x/tools repo change-id: Ia979a8b06d1b4db47d25ffdfdf925ba8a0ac67de

Real new code additions:
- main.go
    * lines 89-95 added getFullPath method
    * lines 217-222 mapped paths to doc/, favicon.ico, robots.txt in vfs
- appinit.go
    * lines 147-153 added getFullPath method
    * lines 80-84 mapped paths to doc/, favicon.ico in vfs

Several files were copied from x/tools and go so paths (and corresponding
import paths) were changed as follows:

"x/tools/cmd/godoc/" --> "x/website/cmd/golangorg/"
"x/tools/godoc/static/" --> "x/website/content/static/"
"x/tools/godoc/" (without godoc/static/) --> "x/website/cmd/golangorg/godoc/"
"x/tools/internal/memcache" --> "x/website/internal/memcache"
"go/doc/" --> "x/website/content/doc/"
"go/favicon.ico" --> "x/website/favicon.ico"
"go/robots.txt" --> "x/website/robots.txt"

Updates golang/go#29206

Change-Id: I53985fc027f73e60c6946038f85133acf1ecb08c
Reviewed-on: https://go-review.googlesource.com/c/156321
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@cnoellekb

This comment has been minimized.

Copy link

cnoellekb commented Jan 9, 2019

Some decisions that have been made over the last few days:

  1. I am currently prioritizing turning shared documentation-rendering logic between x/website and the godoc binary into a package, maybe it will live in a directory called "unstable" in x/tools?
  2. Deployment of the website will come later when godoc has been properly stripped down with all non-documentation stuff ready to be deleted from x/tools
  3. In the long term @dmitshur plans to expand the package to support rendering documentation for modules
@katiehockman

This comment has been minimized.

Copy link
Contributor

katiehockman commented Jan 9, 2019

  1. I am currently prioritizing turning shared documentation-rendering logic between x/website and the godoc binary into a package, maybe it will live in a directory called "unstable" in x/tools?

This sounds good. Perhaps under x/tools/godoc/unstable since it will be for go documentation.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 9, 2019

I wouldn't bother with the /unstable suffix: just put a prominent note in the package comment explaining that it's unstable.

Actually, better still, could we start a new repo with the path golang.org/x/internal? Then the Go tool's visibility rules would enforce that it is only used within other golang.org/x modules.

@dmitshur

This comment has been minimized.

Copy link
Member

dmitshur commented Jan 9, 2019

We already have the golang.org/x/tools/godoc package, which is documented as:

Package godoc is a work-in-progress (2013-07-17) package to begin splitting up the godoc binary into multiple pieces.

This package comment will evolve over time as this package splits into smaller pieces.

I think this existing package is a good candidate to move code that we want to share between x/tools/cmd/godoc (for viewing documentation) and x/website/cmd/golangorg (for serving golang.org) in the short term.

I like the idea of x/internal in general, because if it existed, we could use it in other instances to do similar refactors between subrepos. However, I think that should be a separate proposal. Also, golang.org/x/internal would preclude golang.org/dl from being able to use it, so maybe it should be golang.org/internal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment