Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wiki: move to x/wiki #61940

Closed
rsc opened this issue Aug 10, 2023 · 18 comments
Closed

wiki: move to x/wiki #61940

rsc opened this issue Aug 10, 2023 · 18 comments

Comments

@rsc
Copy link
Contributor

rsc commented Aug 10, 2023

Update, Dec 14 2023: This work is done. github.com/golang/go/wiki/Foo is now go.dev/wiki/Foo. The go.dev URLs used to redirect to the github.com ones; now the github.com ones serve a link to the go.dev ones.

The go.googlesource.com/wiki repo is updatable with the usual mechanisms, except that the rules for submitting are looser. In particular submit only requires a +2; it does not require two Google employees to be involved. Also, we can grant wiki +2 access separate from the other repos. Anyone who did significant work in the wiki in 2023 should already be on the wiki +2 list, but if not, please feel free to reach out to me (rsc@golang.org). Thank you!


The golang/go wiki is world-writable, which is a low-level source of spam and just generally not fantastic. When Go was a little-known project just starting out, this was probably fine, but I think we've outgrown world-writability.

Another problem with world-writability is that GitHub marks the pages as no-crawl, so that they don't appear in search results. When you search for things like [go release cycle] the top result is a mirror, zchee.github.io, not the actual page.

Another problem with the wiki is that there is no review process. We set up the golang-wikichanges@googlegroups.com mailing list long ago so that at least there is notification.

Given these problems, it seems the time has come to arrange some kind of review or permissions or both for editing the wiki.

GitHub provides no way to lock down the wiki other than to all users with push access to GitHub's copy of the main go repo. We can't reasonably grant that power just so that people can edit the wiki.

Instead, it seems like the most reasonable path forward is to copy the md files out of the go.wiki repo and into x/website/_content/wiki, and then lock down the GitHub wiki and replace each page with a forwarding link.

This would mean that people who want to edit things like the Conference or GoUsers list will have to send a Gerrit CL or a GitHub PR. I expect most current casual wiki editing will arrive as a GitHub PR, and @thepudds is doing some great work to make that experience smoother.

@rsc rsc added this to Proposals Aug 10, 2023
@rsc rsc moved this to Incoming in Proposals Aug 10, 2023
@gopherbot gopherbot added this to the Proposal milestone Aug 10, 2023
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/518297 mentions this issue: _content: copy GitHub wiki pages to this repo

@jakebailey
Copy link

FWIW GitHub repo wikis are themselves git repos; on TypeScript, we have two way syncing enabled such that edits of https://github.com/microsoft/TypeScript/wiki are automatically pushed to https://github.com/microsoft/TypeScript-wiki via the GitHub gollum event type (GHA can trigger on this, but so can other API uses), and then the opposite direction on pushes to the wiki repo. I'm betting that you could just do the latter to keep the content there but only writable by pushers, e.g. gopherbot.

@seankhliao
Copy link
Member

I guess this supersedes #34038 and #38393 ?

@rsc
Copy link
Contributor Author

rsc commented Aug 10, 2023

@jakebailey Indeed, and we could do that, but we already use the URLs go.dev/wiki/Foo to redirect to github, and we also already have a repo with our web content. We can just put the content there instead and have one fewer repo. There's not an obvious benefit to having a second repo with 160 assorted md files in it.

@rsc
Copy link
Contributor Author

rsc commented Aug 10, 2023

@seankhliao Thanks, closed those with a link here.

@thepudds
Copy link
Contributor

Instead, it seems like the most reasonable path forward is to copy the md files out of the go.wiki repo and into x/website/_content/wiki, and then lock down the GitHub wiki and replace each page with a forwarding link.

FWIW, I think this would be an overall improvement, especially for the review benefits and given the change GitHub made a few years ago to stop crawlers from indexing wiki content. (That situation then started to improve, including there's a slow-moving GitHub community discussion, but it seems unlikely to return to the prior state of affairs AFAICT).

Compared to other official documentation and website content, a wiki can foster a greater feel of broader community ownership of the content, and people likely feel there is a lower bar to editing a wiki, which then often results in a greater diversity of wiki content. And when reading a wiki, people often have a greater awareness that something might be stale or lower quality compared to more "official" content, and that an outbound link to some other resource mostly means that someone from the broader community thought it worthwhile without it necessarily being an official endorsement from the core Go team.

It might be worth considering what can or should be done to help this content feel more (or less) like a wiki.

Some topics:

  1. Perhaps part of the import process could add a standard header or similar that helps make it clear that this is a wiki or wiki-ish content, with a link to a page that covers how to edit, who should edit, and so on. (It looks like the draft CL puts "/wiki/" in the URL, which helps, but people are generally used to how a GitHub wiki looks and behaves, and this won't be a standard GitHub wiki).

  2. Would an edit on topic X need to be reviewed by the 1-2 official owners of area X on the core Go team, vs. a broader group?

  3. What would be the documented threshold for giving a +2 on a wiki edit (e.g., "certain it is correct" vs. "sounds right" vs. "not obviously wrong" vs. ___)?

  4. As I understand it, the proposal repo for example allows a single person to review and approve a change because that repo does not contain any code that gets shipped to users and hence does not run into the same security and compliance issues as most of the other repos. On the other hand, I think at one point the website repo had a higher requirement for review than other repos, though that might not be the case any longer. I don't know if the wiki is worth creating a special case, but I wonder if the wiki should have lower enforced review requirements than the website repo, or perhaps be a separate repo.

In any event, I do think this would be a good step forward. 🚀

@rsc
Copy link
Contributor Author

rsc commented Aug 10, 2023

Good questions.

The draft CL keeps serving the content at the same URLs that already redirect to GitHub today (go.dev/wiki/PageName).

We can definitely put headers and footers on the pages to indicate that this is the wiki, along with a link to more info.

The draft CL put the files in x/website because that was easy, but you are right that being there comes with the usual 2-Googler rule. If we put the files in a new x/wiki repo then the web server can watch and serve from that repo and help wiki editors get their work done faster. I think that's probably worth doing, and I'll look into the feasibility.

Thanks!

@rsc
Copy link
Contributor Author

rsc commented Aug 30, 2023

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals Aug 30, 2023
@mdempsky
Copy link
Contributor

Another problem with world-writability is that GitHub marks the pages as no-crawl, so that they don't appear in search results.

I rarely look at the wiki for exactly this reason. I find GitHub search generally very unhelpful, so if I can't find something with Google search, then it might as well just not exist.

The only exception is the MinorReleases page, which I frequently consult because I can never remember the exact syntax for requesting and/or creating backport CLs, and team members have directed me there enough times that I remember its existence.

@rsc
Copy link
Contributor Author

rsc commented Sep 7, 2023

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

@rsc rsc moved this from Active to Likely Accept in Proposals Sep 7, 2023
@rsc rsc moved this from Likely Accept to Accepted in Proposals Oct 3, 2023
@rsc
Copy link
Contributor Author

rsc commented Oct 3, 2023

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

@rsc rsc changed the title proposal: wiki: move to x/website wiki: move to x/website Oct 3, 2023
@rsc rsc modified the milestones: Proposal, Backlog Oct 3, 2023
@magical
Copy link
Contributor

magical commented Oct 3, 2023

@rsc thepudds made some good points in favor of moving the wiki pages to x/wiki instead of x/website. You seemed to agree, and said that you would "look into the feasibility" of doing so. Is that still the plan? And if so, should this issue be retitled?

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/532577 mentions this issue: repos: add wiki

gopherbot pushed a commit to golang/build that referenced this issue Oct 4, 2023
For golang/go#61940.

Change-Id: If4f9a1bd8170459a2f4ad5a26ac2449c6dde79db
Reviewed-on: https://go-review.googlesource.com/c/build/+/532577
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/533295 mentions this issue: go.mod: update x/build to CL 532577

willpoint pushed a commit to orijtech/website that referenced this issue Oct 17, 2023
For golang/go#61940.

Change-Id: I5e7bce3ee2e8fb0824772496b86f4bcc16cdca72
Reviewed-on: https://go-review.googlesource.com/c/website/+/533295
Reviewed-by: Heschi Kreinick <heschi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/550135 mentions this issue: cmd/golangorg: fix cloudbuild.yaml

@rsc rsc changed the title wiki: move to x/website wiki: move to x/wiki Dec 15, 2023
@rsc
Copy link
Contributor Author

rsc commented Dec 15, 2023

This is done. Echoing what I just added to the top comment:

Update, Dec 14 2023: This work is done. github.com/golang/go/wiki/Foo is now go.dev/wiki/Foo. The go.dev URLs used to redirect to the github.com ones; now the github.com ones serve a link to the go.dev ones.

The go.googlesource.com/wiki repo is updatable with the usual mechanisms, except that the rules for submitting are looser. In particular submit only requires a +2; it does not require two Google employees to be involved. Also, we can grant wiki +2 access separate from the other repos. Anyone who did significant work in the wiki in 2023 should already be on the wiki +2 list, but if not, please feel free to reach out to me (rsc@golang.org). Thank you!

gopherbot pushed a commit to golang/website that referenced this issue Dec 15, 2023
For not having been able to run this before submitting,
I was *so* close.

For golang/go#61940.

Change-Id: Ifb14b1ec35f731e7e886956ad422f92122625a38
Reviewed-on: https://go-review.googlesource.com/c/website/+/550135
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
@dmitshur dmitshur modified the milestones: Backlog, Unreleased Dec 15, 2023
k1LoW added a commit to k1LoW/gostyle that referenced this issue Dec 17, 2023
zchee pushed a commit to zchee/golang-wiki that referenced this issue Jan 19, 2024
Normalized heading levels, updated titles, fixed a few link syntax problems.

We might delete some of these as outdated at some point.

For golang/go#61940.
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/592856 mentions this issue: WebAssembly: convert to Markdown

gopherbot pushed a commit to golang/wiki that referenced this issue Jun 17, 2024
This was the only non-.md wiki page, causing it not to render at
go.dev/wiki. It was previously converted from Markdown to AsciiDoc
to get a generated table of content, but that happens for Markdown
pages too now.

Add one level of indentation to headings so they fit better under
the generated h1 page title heading.

For golang/go#61940.

Change-Id: I7407879273588cf73eb9f4a276b5c533aa560c1b
Reviewed-on: https://go-review.googlesource.com/c/wiki/+/592856
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
ardrabczyk added a commit to ardrabczyk/project-layout that referenced this issue Jun 23, 2024
Go wiki has moved golang/go#61940. Wiki at
github.com/golang/go/wiki does not automatically move to go.dev/wiki
but says that wiki has moved.

I used the following shell one-liner to convert all links:

    $ find . -type f -print0 | xargs -0 sed -i 's,github.com/golang/go/wiki,go.dev/wiki,g'

To verify if they return 200:

    $ -ag -a --nofilename -o 'https?.+wiki/.+?\b' | sort | uniq | xargs curl  -sI
ardrabczyk added a commit to ardrabczyk/project-layout that referenced this issue Jun 23, 2024
Go wiki has moved golang/go#61940. Wiki at
github.com/golang/go/wiki does not automatically move to go.dev/wiki
but says that wiki has moved.

I used the following shell one-liner to convert all links:

    $ find . -type f -print0 | xargs -0 sed -i 's,github.com/golang/go/wiki,go.dev/wiki,g'

To verify if they return 200:

    $ -ag -a --nofilename -o 'https?.+wiki/.+?\b' | sort | uniq | xargs curl  -sI
    HTTP/2 200
    (...)
    HTTP/2 200
ardrabczyk added a commit to ardrabczyk/project-layout that referenced this issue Jun 23, 2024
Go wiki has moved golang/go#61940. Wiki at
github.com/golang/go/wiki does not automatically move to go.dev/wiki
but says that wiki has moved.

I used the following shell one-liner to convert all links:

    $ find . -type f -print0 | xargs -0 sed -i 's,github.com/golang/go/wiki,go.dev/wiki,g'

To verify if they return 200:

    $ ag -a --nofilename -o 'https?.+wiki/.+?\b' | sort | uniq | xargs curl -sI
    HTTP/2 200
    (...)
    HTTP/2 200
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/600997 mentions this issue: cloudfns/{sendwikidiff,wikiwebhook}: delete

gopherbot pushed a commit to golang/build that referenced this issue Jul 25, 2024
The need for this setup has been made obsolete via wiki move to Gerrit,
proposal golang/go#61940. Thanks for serving us well, as well as being
quite a nice, simple setup that involved little follow-up maintenance.

Fixes golang/go#64730.
Fixes golang/go#55263.
Fixes golang/go#33187 (obsolete).

Change-Id: I2f6161ec8bf7f9604826e7f803d1084fe9be32aa
Reviewed-on: https://go-review.googlesource.com/c/build/+/600997
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Accepted
Development

No branches or pull requests

8 participants