Make wiki independent of GitHub

[capital-G]

Motivation

A wiki is hopefully a permanent reference.
But in the history of SC, there has already been (at least?) one abandoned wiki that was migrated to the GitHub wiki, and while GitHub is a convenient place, it is not a place to support.
I therefore advocate, in support of #6114, to move the current wiki https://github.com/supercollider/supercollider/wiki to https://wiki.supercollider.online and making it a repository that creates a static website, which serves as the new wiki.
This would allow to take ownership of the wiki URL and be independent of the code hosting platform, making it a permanent location for the precious information.

This would also prepare the community/wiki in case GitHub needs to be evacuated.

Description of the proposed feature

I first thought about making a real wiki, but I think a static website is easier to maintain and has a longer lifespan.
A small wiki like SC should be suitable for a static site generator.

I decided to use Sphinx as a static website generator because it is an established tool and has a pretty stable API (compared to NodeJS based tools).

I also don't have a clue how to contribute to the existing wiki (fork the wiki-repo and do a PR?) as I don't have editor privileges, but making it a repo would allow anyone with access to git to edit it via a PR workflow.

Sphinx would also allow to create indices, references, glossary, build local tocs, provide a search and also verify that all internal links are working during build time.

Implementation plan

I cloned the existing wiki and converted it to a Sphinx website - this means that the history of most pages is untouched, preserving valuable git history (see https://github.com/capital-G/supercollider-wiki/blame/main/docs/contributing/workflow/Unit-Testing-Guide.md)

Repo: https://github.com/capital-G/supercollider-wiki
Website: https://capital-g.github.io/supercollider-wiki/

Currently hosted on GitHub pages - although this PR is motivated to evacuate GitHub, I think it is still okay to use their CI for now. The build step and serving of static content can easily be moved to e.g. GitLab.

I would be happy to take up some styling suggestions or commits, as this wiki would allow us to style and modify in such a way we want, and not how GitHub thinks it should be done.

[paum3]

It's not possible to move entire SuperCollider to Codeberg?

[capital-G]

It would be possible, but then we would be dependend again upon codeberg (which would be already way better, but also wouldn't solve some other issues like local TOC, dead internal references, clear PR procedure and custom styling which this PR implements). Also afaik Supercollider does not have a codeberg representation.

I think the best is to aim to be provider agnostic if possible, and wiki would be such a use case that, as a static website, can be hosted anywhere and is not dependent on a workflow of a provider.

[cdbzb]

I'm enthusiastic about moving in principal but not totally clear whether a static site would be better than an actual wiki.

To be able to evaluate it would be great to see a static site that was an "apples to apples" comparison with the existing wiki. The current wiki has a menu on the right titled "Wiki Directories" the first of these is "Contributing-directory.md" https://github.com/supercollider/supercollider/wiki/Contributing-directory
which shows up in the static site repo as having been deleted.
instead I see a "Navigation" meno on the left and first is Contributing https://capital-g.github.io/supercollider-wiki/contributing/index.html which is quite different!

[capital-G]

The repo/PR/Issue is more a first sketch, but I think it is already surpassed the POC state so its worth to be discussed if we want to adapt this concept. If so, we can figure out the points which need improvement, work on them, and then succeed with the plan.

I'm enthusiastic about moving in principal but not totally clear whether a static site would be better than an actual wiki.

The main advantages I see

• independence of any hosting platform - hosting a static website can be easily done these days
• searching the wiki
• editing can be performed offline
• internal references can be checked for dead links at build time
• custom styling
• Clear PR workflow (how can the current wiki be edited? It has a git repo but how do I make a PR on it?)
• website structure is independent of folder structure, which allows to improve the organization of files

To be able to evaluate it would be great to see a static site that was an "apples to apples" comparison with the existing wiki. The current wiki has a menu on the right titled "Wiki Directories" the first of these is "Contributing-directory.md" https://github.com/supercollider/supercollider/wiki/Contributing-directory
which shows up in the static site repo as having been deleted.
instead I see a "Navigation" meno on the left and first is Contributing https://capital-g.github.io/supercollider-wiki/contributing/index.html which is quite different!

There are also some different themes which can be changed really easily and also change the clickflow, but I stayed with the default one for now - but take a look at https://sphinx-themes.org/ (but I'd like avoid discussions about styling/aesthetics for now).

For example, I personally don't like a GitHub wiki at all - I don't understand how documents are structured there is no way to search for elements. But most likely it is about what one is used to.

Sphinx would allow to mimic the current behavior but I don't want to spend time on developing a Github Wiki clone in Sphinx.

On how this TOC gets build: Internally, Sphinx can handle two ways, either referencing other files in a TOC like this https://github.com/capital-G/supercollider-wiki/blob/344aa489f381f3d8d849076eea08e0c5939bdec9/docs/index.rst?plain=1#L43-L51 or use glob to "just include everything in this folder in the order of the filenames" https://github.com/capital-G/supercollider-wiki/blob/344aa489f381f3d8d849076eea08e0c5939bdec9/docs/community/index.rst?plain=1#L13-L18

[mtmccrea]

Thanks for making a proof of concept! It's very helpful for discussing the concrete implementation.

Was the migration process particularly cumbersome? I ask because I'd be curious to see it using the up-to-date wiki structure, since I think that update controlled a bit of the chaos and may give a closer impression of what this new wiki might feel like. Not worth spending a large effort on it if that's required, though.

I agree with all of the advantages you listed!

It's very helpful that dead links can be spotted automatically, that's already a bit help. I wonder if we can do anything about dead links in the wild.... it would be great if addresses persisted through page name changes (as they do on scsynth.org, for example).

I wonder if people have thoughts on edits done through a PR process rather than editing directly. It may discourage quick additions and fixes, but on the other hand offline editing perhaps means more well-structured contributions (esp. if contributors are using a wysiwyg markdown editor).

[capital-G]

Was the migration process particularly cumbersome? I ask because I'd be curious to see it using the up-to-date wiki structure, since I think that update controlled a bit of the chaos and may give a closer impression of what this new wiki might feel like. Not worth spending a large effort on it if that's required, though.

This is one of the things I want to discuss here. I think with Sphinx allows to manage more complex TOCs, but one needs to be explicit about it.
Actually I tried to replicate the current layout of the wiki :/ The only thing I changed is to put the FAQ under its own label.

vs

If we would decide to go for the material theme then the docs could also look like this (needs 10 additional loc)

I deployed now the sphinx-material theme online, so you can try it out: https://capital-g.github.io/supercollider-wiki/

It's very helpful that dead links can be spotted automatically, that's already a bit help. I wonder if we can do anything about dead links in the wild.... it would be great if addresses persisted through page name changes (as they do on scsynth.org, for example).

This would be possible, see https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.linkcheck.CheckExternalLinksBuilder

I wonder if people have thoughts on edits done through a PR process rather than editing directly. It may discourage quick additions and fixes, but on the other hand offline editing perhaps means more well-structured contributions (esp. if contributors are using a wysiwyg markdown editor).

Thats interesting - I don't see an edit button, otherwise I would have fixed/updated some little things sometimes - maybe you need some kind of permission to edit the wiki?

[cdbzb]

...yes there is a permission required for the GitHub wiki ATM (apparently it used to be unlocked but there were some issues...)

I think the alternative to a fixed site would be something live like dokuwiki -- on the plus side would be quick fixes and also perhaps contributions from users who are not familiar with the PR workflow. On the minus side it would need to be pretty actively maintained to roll back any mischief... I can definitely see it both ways!

[capital-G]

I think the alternative to a fixed site would be something live like mkdocs

I don't understand the live part of mkdocs? Do you mean the live server? Sphinx has such features as well https://github.com/executablebooks/sphinx-autobuild - it is also possible to directly link to the md/rst file in question

Mkdocs is actually a Sphinx clone (b/c Sphinx used to not support md), but I personally prefer Sphinx because it is more matured, has more features and supports rst as a native format (my 2ct: although I am a big md fan, I think it is not suitable for writing docs - CommonMark etc. is an example why md is not sufficient)

In the status quo format only authorized people can contribute and although there is a guide to contribution, I don't know who to contact in order to obtain edit rights. Via PR anybody (with sufficient skills) can propose edits which then can be reviewed/discussed.

[cdbzb]

sorry wrong paste - meant dokuwiki!

[capital-G]

sorry wrong paste - meant dokuwiki

Oh okay. Something like this would require someone to host it and create backups of it etc. - this is therefore more centralized and prone to "not-survive" than static websites, although the live-edits are indeed a very nice thing.

[mtmccrea]

Actually I tried to replicate the current layout of the wiki :/ The only thing I changed is to put the FAQ under its own label.

Hmmm I think some things got mixed up in the migration, in the screenshot you posted, the contributing directory page is missing some wikis, and for some reason the table of contents of The wiki wiki's (which is its own page) ended up on the Contributing directory page, and the UGen quality standards wiki shows up as a top-level category rather than a wiki like any other.

Anyway, not a huge deal for the sake of demonstration, it just may come across as more confusing which could distract from the impact of otherwise many improvements.

[mtmccrea]

...yes there is a permission required for the GitHub wiki ATM (apparently it used to be unlocked but there were some issues...)

Realistically there won't be that many people contributing, so having access control seems ok to me. Anyone looking to make changes to the wiki likely have some presence already in the development cohort and can request access.

I don't see an edit button, otherwise I would have fixed/updated some little things sometimes

For this reason, I think having a low barrier to contributions (aside from initial access) will be important to keep the wikis up to date and with good coverage — plenty of engaged people will spot and want to fix/update little things but may not bother if there's too much process.

[prko]

It looks great!

In HelpBrowser this site has no compatibility issues (scsynth.org is not supported in HelpBrowser) and is beautiful!
Could we include this wiki in the home for HelpBrowser or integrate it into the help documentation?

HelpBrowser.goTo("https://capital-g.github.io/supercollider-wiki/")

HelpBrowser.goTo("https://scsynth.org")

[capital-G]

Hmmm I think some things got mixed up in the migration, in the screenshot you posted, the contributing directory page is missing some wikis, and for some reason the table of contents of The wiki wiki's (which is its own page) ended up on the Contributing directory page, and the UGen quality standards wiki shows up as a top-level category rather than a wiki like any other.

I updated it to mirror the current wiki TOC https://capital-g.github.io/supercollider-wiki/ - although this TOC has the "quirk" that the same document shows up under multiple TOCs - to me this is an antipattern b/c it is unclear which namespaced URL to use/build.
At the same time I don't see the clear separation between Development and Contributing - development is one sort of contributing to me and could therefore be moved into Contributing, but I don't want to argue about the TOC in this PR.

For this reason, I think having a low barrier to contributions (aside from initial access) will be important to keep the wikis up to date and with good coverage — plenty of engaged people will spot and want to fix/update little things but may not bother if there's too much process.

In the current status quo I don't know how to obtain access as there it is nowhere stated on how to obtain edit rights, and therefore I think it is not very welcoming at all and one becomes a bit paranoid - why do some people who contribute have access, others not? It feels like ones contributions are not regarded trustworthy, therefore one has no rights to edit the wiki?
The PR workflow wouldn't change this per se, but at least I can request something and make it public for other people to see as well, and this would be to me much more welcoming as everyone is welcomed to contribute.
I found this lack of transparency in certain regards of SC a big problem in engaging with dev issues, as there are people who have the power of saying yay/nay on things but they are never stated and only the "inside-dev-circle" knows who these people are - one feels easily excluded and one is unsure if this is b/c of the quality of the contribution or one does not know who to contact in order to get a "ruling" on the issue.
But maybe this is not the place to talk about these issues, they have been multiple threads in the forum about it already.

[mtmccrea]

Thanks for updating the wiki mockup. Hopefully it helps people get a better sense of what it can be.

Regarding the structure, I agree it needs improvement, and it's meant to be changed. If you haven't yet, please see the scsynth forum topic on the wiki. It lays out the intention and explains that the current state is a stepping stone to better structure and to encourage new additions to find the right home more quickly. Please share your thoughts there.

this TOC has the "quirk" that the same document shows up under multiple TOCs

The rationale for the anti-pattern is here.
https://capital-g.github.io/supercollider-wiki/contributing/The-wiki-wiki.html#directory-pages
That's also just an experiment.

In the current status quo I don't know how to obtain access as there it is nowhere stated on how to obtain edit rights, and therefore I think it is not very welcoming at all and one becomes a bit paranoid

I sympathize and relate to this sentiment! It should stated somewhere clearly who has access to what...probably in wiki the the Community directory ;) To clarify my comment about access control, I'm loosely in favor of access control to the current wiki, just for the reason that it became a dumping ground, and not as a long-term solution. I agree the PR process is one solution to this, with the added benefit of more transparency overall. I think access control should also be discussed on the forum, so this conversation can focus on your PR.

You're one of the contributors most engaged with the wiki, and I think you should definitely request access from one of the owners:
https://github.com/orgs/supercollider/people

I found this lack of transparency in certain regards of SC a big problem in engaging with dev issues, as there are people who have the power of saying yay/nay on things but they are never stated and only the "inside-dev-circle" knows who these people are

I'd love to see this conversation continue on the forum (do you know of a Topic already?). FWIW I agree with your stated points. I've tried to scrub references to "Developer" from the wiki and forum, in favor of simply "Development". Perhaps "contributor" is better? So there's work to be done there still...

[cdbzb]

maybe a shortcut is to request to join this team: https://github.com/orgs/supercollider/teams/trusted-reviewers

[mtmccrea]

Looking over the list of members of the SC Organization, I don't see you there, @capital-G. I think simply being added as a member should give you write access to the wiki. I think Owners grant this membership.

Also added a section to the wiki about this.

[capital-G]

I added an edit button on each site which will redirect to the source code within the repo from where one can spawn a PR via the visual editor of GitHub.

For example, clicking on the pink pencil on

redirects to https://github.com/capital-g/supercollider-wiki/blob/main/docs/contributing/style/Style-Guidelines:-Cpp.md

maybe a shortcut is to request to join this team: https://github.com/orgs/supercollider/teams/trusted-reviewers

e

^^ :D

But actually I don't want to become a trusted reviewer, I wouldn't feel comfortable in such a position. And it wouldn't change that the next person who just passes along can suggest an edit.
The suggested PR workflow could also be avoided for minor changes when people have the ability to directly perform commits on the main branch (aka are trusted editors). Than they could edit it directly in the source code on GitHub and perform a commit.

Also added a section to the wiki about this.

Thanks for adding the comment! But I still wouldn't know how one would contact someone on github - you could try to extract their mail address from their commits, but I don't think thats a nice thing to do. Hitting one up via a mention is also possible, but is also considered a rude action by some people.

[mtmccrea]

I added an edit button on each site which will redirect to the source code within the repo from where one can spawn a PR via the visual editor of GitHub.

Nice! That's a useful improvement :)

[mtmccrea]

Thanks for adding the comment! But I still wouldn't know how one would contact someone on github - you could try to extract their mail address from their commits, but I don't think thats a nice thing to do. Hitting one up via a mention is also possible, but is also considered a rude action by some people.

Not ideal, I know. For now, it could be that it comes with the territory of being an org owner. If they're annoyed, that will perhaps motivate change toward a better process X-P. Continue that discussion here? https://scsynth.org/t/the-sc-wiki/8123/13?u=mike

[mtmccrea]

@mossheim brought up the good point that this should be discussed as an RFC. I also think that it will help to reach a formal consensus, since in this Issue space responsibility is more diffuse and it might never see a clear "go ahead".

@capital-G, you've put in a lot of work up to this point, but would you mind filling out an RFC and pointing to this Issue? I think the description you've provided is already 90% of the work in filling out the template.

[capital-G]

@capital-G, you've put in a lot of work up to this point, but would you mind filling out an RFC and pointing to this Issue? I think the description you've provided is already 90% of the work in filling out the template.

Thanks for keeping interest in this @mtmccrea, but I don't really see what a RFC would change as everything has already been proposed, stated and implemented for this (minor) change. If there are any questions left I am happy to answer them, but if a RFC is the only way to make progress I'd rather close this issue.

The only thing that's missing is a decision if we should go for this or not, and as I mentioned before, I don't know the devs in charge who can provide such a definitive yay/nay on this issue.

[mossheim]

The only thing that's missing is a decision if we should go for this or not, and as I mentioned before, I don't know the devs in charge who can provide such a definitive yay/nay on this issue.

That's exactly what an RFC is supposed to accomplish. BTW, moving project wiki to a new repo with totally new tooling and maintenance process is not "minor change."

[cdbzb]

@capital-G if the issue is that you don't need the extra work/annoyance of filing the rfc I'd be happy to do it (copying your text) if that's acceptable.

To find how to do it I actually consulted the wiki and was immediately annoyed by lack of search! (which helps makes the case for this proposal)

[mtmccrea]

To elaborate a bit on why an RFC makes sense: the first section of the RFC repo describes that the whole processes is oriented toward making a decision, so participants there understand where the conversation is heading. A useful feature is also the stage of a Final Comment Period, which signals that, ok we've discussed enough, it's decision time. When that stage is reached is still a matter of discretion, but still useful.

Another advantage of an RFC over filing an Issue: There are 823 open Issues, and 10 open RFCs. Anyone interested in RFCs is unlikely to pass over it. You'll see from the description also that RFCs are understood to be "advertised", which gives them somewhat elevated status on the forums but also imply some advocacy work. (I just added a category tag for this.)

Your frustration is well understood—you're bumping into issues of process that the SC community struggles to iron out, and in sticking with it you'd be setting a good example, building institutional knowledge of process (new RFCs have been dormant for months/years), and hopefully making documentation for this kind of thing more robust in the process (I will try to help with this). The more successful initiatives we can complete, the more confidence builds for maintainers and future contributors.

[mtmccrea]

[OT]

To find how to do it I actually consulted the wiki and was immediately annoyed by lack of search!

link added, and it's also on the How to contribute page ;) But yea search would help!

[cdbzb]

I wrote a quick rfc here: https://github.com/cdbzb/supercollider-rfcs/blob/move-wiki/rfcs/0000-move-wiki-from-GitHub-and-host-as-static-site.md @capital-G do you want to copy and do a pull request? or I'm happy to do the PR. Just hoping to move this forward!