RFC: restructure website content, roughly following Diátaxis

[h-vetinari]

There are several ongoing efforts to improve the conda-forge website, which has yielded the new website, status page, as well as some work around the infrastructure pages.

Additionally @jaimergp wrote in the context of the more tightly-scoped #2154:

Last year, I experimented with how Diátaxis would apply to our docs in https://cf-infra-docs.netlify.app/docs. Some of the docs written for that experiment are being migrated over to our new website. @zklaus is almost finished, I think, with that effort.

I took that idea and came up with a rough draft of how the website could be structured (originally looking only at the infrastructure page, but I felt I needed a big picture view even for that). Some new content would have to be written, but mainly it is intended to reshuffle existing content into easily discoverable and more granular pieces.

As Diátaxis intentionally aims at gradual transformation rather than big bang refactoring, this outline would likely change as we iterate, but I wanted to get people's thoughts on the rough strokes (and then discuss it in the core call), so we can agree on the overall direction and let people start chipping away at some first pieces.

This is based on looking at our current docs as well as @jaimergp's experiment with Diátaxis; a more detailed comparison is available below the fold.

Welcome
For users
└ ...                               # not in scope here
For maintainers
├ Tutorials
|  ├ Generate a conda recipe        # Using greyskull or whatever
|  ├ Write your first recipe        # Segue from first tutorial, make some changes
|  ├ Submit your first recipe       # Segue again, process for submission
|  └ ...
├ How-to guides
|  ├ Easy                           # the below is just a selection, obviously
|  |  ├ Read a recipe               # Component overview, refer to in-depth explanation
|  |  ├ Rerender feedstock
|  |  ├ Test packages
|  |  ├ Deal with numpy
|  |  ├ Enable osx-arm64            # Same for arches
|  |  ├ Talk to the bots            # refer to bot command reference page
|  |  └ ...
|  ├ Medium
|  |  ├ Cross-compile
|  |  ├ Enable CUDA
|  |  ├ Multi-output recipes
|  |  ├ LTS branches
|  |  └ ...
|  └ Emergencies
|     ├ Patch repodata
|     └ Mark a package broken
├ Understanding conda-forge         # Explanation part of Diátaxis
|  ├ Life-cycle of a package        # essentially https://cf-infra-docs.netlify.app/docs/fundamentals/life-cycle/
|  |  └ ...
|  ├ How a package gets built
|  |  ├ Compilation concepts        # shared/static/etc.
|  |  ├ Environment roles           # e.g. build: vs. host:
|  |  ├ Compilers                   # e.g. role of activation
|  |  ├ Global pinning
|  |  ├ Run exports
|  |  ├ Jinja functions
|  |  └ ...
|  ├ How our bots work
|  |  ├ Migrations
|  |  ├ regro infrastructure
|  |  └ ...
|  ├ Security
|  |  ├ Output validation
|  |  └ ...
|  ├ Maintainer roles
|  └ ...
├ Reference
|  ├ Infrastructure                 # more in-depth discussion in #2154
|  |  ├ Staged recipes              # including teams
|  |  ├ Compilers
|  |  ├ Other core packages         # sysroots, CDTs, msys2, ...
|  |  ├ CI Setup                    # docker images, conda-forge-ci-setup
|  |  ├ CI providers
|  |  ├ Global pinning
|  |  ├ Feedstock evolution         # conda-smithy
|  |  ├ Feedstock metadata          # graphs, feedstocks, feedstock-outputs
|  |  ├ Metadata corrections        # repodata-patches, admin-requests
|  |  ├ Bot repos                   # all things regro
|  |  ├ Website
|  |  └ ...
|  ├ Bot commands
|  ├ Feedstock settings             # conda-forge.yml, CBC, migrators
|  ├ Glossary
|  ├ Current pins                   # Auto-generated / subset?
|  ├ Ongoing migrations             # refer to status page
|  └ ...
├ Miniforge
└ FAQ                               # separate section because questions cover both how-to & reference...?

Comparison with current docs and Diátaxis experiment

	vs. docs	vs. Infra Exp.	Comment
Welcome	== here	== Welcome
For users	== here	n/a	could surely be improved...
└ ...			... but not in scope here
For maintainers	== here	n/a
├ Tutorials	partially here	== Tutorials
| ├ Generate a conda recipe			Using greyskull or whatever
| ├ Write your first recipe			Segue from first tutorial, make some changes
| ├ Submit your first recipe		~= placeholder	Segue again, process for submission
| └ ...
├ How-to guides	distributed all over	== How-to guides
| ├ Easy
| | ├ Read a recipe			Component overview, refer to in-depth explanation
| | ├ Rerender feedstock
| | ├ Test packages
| | ├ Deal with numpy	roughly here
| | ├ Enable osx-arm64	roughly here		Same for arches
| | ├ Talk to the bots			refer to bot command reference page
| | └ ...
| ├ Medium
| | ├ Cross-compile	~= here
| | ├ Enable CUDA	~= here
| | ├ Multi-output recipes	~= here
| | ├ LTS branches	~= here
| | └ ...
| └ Emergencies
| ├ Patch repodata
| └ Mark a package broken	~= here
├ Understanding c-f	not much	~= Fundamentals	"Explanation" part of diataxis
| ├ Life-cycle of a pkg		== here
| | └ ...
| ├ How a pkg gets built
| | ├ Compilation concepts			shared/static/etc.
| | ├ Env. roles	short comment		e.g. build: vs. host:
| | ├ Compilers	partially here		e.g. role of activation
| | ├ Global pinning	~= here
| | ├ Run exports	~= here
| | ├ Jinja functions
| | └ ...
| ├ How our bots work		see here
| | ├ Migrations	~= here
| | ├ regro infra
| | └ ...
| ├ Security	~= here	see placeholder
| | ├ Output validation
| | └ ...
| ├ Maintainer roles	roughly here & here
| └ ...
├ Reference	all over	== Reference
| ├ Infrastructure		~same content split somewhat differently
| | ├ Staged recipes		roughly here	incl. teams
| | ├ Compilers	roughly here
| | ├ Other core packages			sysroots, CDTs, msys2, ...
| | ├ CI Setup			docker images, conda-forge-ci-setup
| | ├ CI providers		roughly here
| | ├ Global pinning	~= here		+explain migrators
| | ├ Feedstock evolution			conda-smithy
| | ├ Feedstock metadata			graphs, feedstocks, feedstock-outputs
| | ├ Metadata corrections			repodata-patches, admin-requests
| | ├ Bot repos			all things regro
| | ├ Website		== here
| | └ ...
| ├ Bot commands	~= here	== here
| ├ Feedstock settings	mostly here	== here	conda-forge.yml, CBC, migrators
| ├ Glossary	== here	== here
| ├ Current pins		== here	Auto-generated / subset?
| ├ Ongoing migrations			refer to status page
| └ ...
├ Miniforge		== here
└ FAQ	~= here		separate section because questions cover both how-to & reference...?

[jaimergp]

Love this. The biggest concern is not breaking any existing URLs and anchors that might refer to existing content.

How we have addressed this so far:

• If there's a one-to-one correspondence between URLs (the same page with the same anchors just moved from one place to other), we can use the redirects defined in docusaurus.config.js.
• If different sections have moved to different pages, the original page can remain in place as an unlisted page with the old anchors pointing to the new URL and section. Example at https://github.com/conda-forge/conda-forge.github.io/blob/main/src/pages/announcements.md.

Another point is how sometimes the line between "For users" and "For maintainers" is not entirely clear. At least, I struggled while categorizing it during that experiment in the draft site. Maybe because I was trying to shuffle existing docs around, and didn't leave room for new content.

In that vein, if categorizing things in the filesystem is too restrictive, we can also leverage the custom sidebar setup (the paths and the categories match in this case, but they don't have to), and the document tags to label pages with certain information (for-beginners, easy, advanced, whatever).

cc @zklaus for awareness

[h-vetinari]

The biggest concern is not breaking any existing URLs and anchors that might refer to existing content.

Yeah that would likely be a large part of the challenge - for now I was happy to draw outlines unencumbered by that though 😅

Another point is how sometimes the line between "For users" and "For maintainers" is not entirely clear. At least, I struggled while categorizing it during that experiment in the draft site.

When I was looking at this, I kind of saw the utility of the existing separation, i.e. someone wanting to just consume packages has a very different set of concerns than someone trying to maintain a feedstock. Consequently, I had descoped the "for users" part in my draft, but I agree with you that the separation is not obvious, and likely harmful in terms of having an easy "on ramp" for turning package consumers into conda-forge contributors. It would be nice if we can smoothen that transition somehow.

[zklaus]

[...] having an easy "on ramp" for turning package consumers into conda-forge contributors. It would be nice if we can smoothen that transition [...].

My thoughts exactly!