A design wiki?

[roberth]

Issue description

Describe the issue you'd like the nixpkgs architecture team to consider solving

Currently the meetings are in a large part taken up by knowledge sharing and a lot of that knowledge only ends up in the notes at best. This is unproductive, because the knowledge isn't organized in any way beyond the minds participating in the meeting (and even then very lossy)

Goal

Describe when this issues should be considered done

• Extend the scope of the team to maintain a knowledge base about the potential design of Nixpkgs in a wiki.
• Delineate what should go into the wiki as opposed to what should be in separate discussions and in official docs

[infinisil]

I like the idea, for a start I just enabled the Wiki for this repository, I think anybody should be able to edit it

[roberth]

https://github.com/nixpkgs-architecture/issues/wiki

[fricklerhandwerk]

@roberth @infinisil I would like to raise my concern about, and active opposition to - excuse the hyperbolic wording - yet another knowledge silo maintained by a small clique, that will almost surely be hard to discover for regular users and contributors.

I just recently saw the same thing done by @jonringer and the release managers and was shocked. What is the reason not to dump this into a directory in Nixpkgs documentation, why not make it part of user-facing documentation?

Anything that is not in lockstep with the actual code will almost inevitably get out of sync. In addition, knowledge transfer across generations is one of the hardest and most important things there is - all that stuff should happen in plain sight instead of some random corner of the internet with different interfaces and mechanisms for subscriptions and version control. There are enough tools for people to learn and deal with as it is, and adding more, if ever so convenient, will introduce friction of which we already have enough for various other reasons.

[roberth]

@fricklerhandwerk I understand your concern, and I have considered these downsides. Nonetheless, I don't see a good alternative. The goal of the wiki is to document things that aren't, rather than things that are. It's one step better than only explaining things during the meetings.

Here's what I wrote, because I share the same concern:

The audience for this documentation is Nixpkgs developers who are interested in making changes to the "core" of Nixpkgs or improve existing patterns.
You are encouraged to contribute "user" documentation to the relevant manuals.

The Haskell community puts this kind of stuff in its general wiki and I hate it. I'm not a Haskell compiler developer and most of the time the haskell wiki serves outdated design considerations instead of even just describing the status quo, let alone how to get something done. This is awful, and not something I want to see repeat on the nixos wiki.

If you have any other ideas where to put low quality information that's irrelevant to users, I'm all ears.

[fricklerhandwerk]

If the idea is to eventually produce design documents or implementation, why not maintain PRs against Nixpkgs? This can be off a fork that all team members have write access to. Once any of this matures it can just be merged.

I had good experience with this, because the comment workflow makes it convenient to work on individual pieces asynchronously. The extreme problem with Wikis is that they only have version history, but not branch and merge.

A PR would solve the problem of being invisible to users: it's the archetype of things that aren't (yet).

[jonringer]

I just recently saw the same thing done by @jonringer and the release managers and was shocked. What is the reason not to dump this into a directory in Nixpkgs documentation, why not make it part of user-facing documentation?

The original motivation to move the release wiki outside of nixpkgs was:

• We could choose something which wasn't docbook for content

• The release process was evolving,

  • it didn't really make sense to have it as part of the nixpkgs manual, where it was cemented in time
  • Also improvements to the wiki usually accumulated throughout the release process, so updating the release wiki should be easier than doing a nixpkgs PR.

  Anything that is not in lockstep with the actual code will almost inevitably get out of sync

  There's very little code for the process.

  Personally I'm of the orientation that unless something should be bundled with a release, then it should probably live outside of NixOS/nixpkgs. Making nixpkgs bigger has repercussions in many regards. Being able to prune what we can into more focused repositories is a big win in being able to orient those repositories in terms of goals, CI/CD, and other processes.

[infinisil]

@fricklerhandwerk I think the purpose of the wiki was described a bit ambiguously. I hopefully fixed this here. The wiki should be for the potential future design of nixpkgs, not the current design. And if one of the potential future design proposals get implemented, that design should be documented in the nixpkgs docs, not in this wiki. During meetings when we notice the same topic being brought up, we can use the wiki article to not go through the same points again. And if new points are brought up we can easily add those to the wiki.

Why not use nixpkgs PRs/issues? Because then you get lost in the sea of its thousands of PRs. This is also the reason we're using a separate issue tracking here instead of the nixpkgs one.

Why a wiki and not another issue/PR tracker? So that it's easy for anybody to edit.

[fricklerhandwerk]

It's not a hill I want to die on, but since we're at it I only would like to add that the sea of PRs appears to me very much like a problem of the wrong data types. Everything is a list, but really PRs go into a DAG of requirements to reach certain goals. It's just that GitHub only informally tracks dependencies and doesn't allow displaying them appropriately.

And ease of editing is a UI problem of Git and plain text editors. That's a pain no one should endure just to make use of version control, and yet here we are...

Okay, please just somehow make sure these repositories and wikis and all these dispersed collections are clearly visible in the contexts they they are relevant (team info pages, etc.). In the end of course it doesn't matter if it's a mess in one file system tree or several database systems, so the challenge is to avoid the mess.

[ParetoOptimalDev]

Everything is a list, but really PRs go into a DAG of requirements to reach certain goals. It's just that GitHub only informally tracks dependencies and doesn't allow displaying them appropriately.

This could be solved pretty well with "overview issues" and tags.

You create an issue like "get nodejs cross compiling working on aarch64-Darwin" and tag it overview.

In it you link to "fix libuv cross compilation on aarch64-Darwin".

Then if you filter issues by tag "overview" (or perhaps "goal") you get a list of goals and their linked tasks/issues.