Move process/development/etc docs to static site

[bitprophet]

Most of the development and index pages of the current docs would work better as a dedicated static site, where changes can be made without having to copy them to each release branch, etc.

This will also be a good excuse to rearrange a little bit.

Specifically, move these to the new site:

• Everything in index.rst which is not the Documentation section -- so the intro/about, installation, development, and getting help sections
• The Installation and Development pages themselves
  • Installation is slightly more tied to the code itself than the rest, so it's a maybe -- e.g. we occasionally get patches that add to the install docs (such as the PyPM chunk).
• Split out the Releases & Support of older releases sections from Development into their own page -- that info is both user and dev related.
• Probably split existing intro material into front-page and "about the project" content...
• ...so we can add more to the front-page, such as:
  • Latest stable versions & link to their changelogs
  • Up-front-and-personal links to various things ("Need help? ", "Want to download? ", "Want to contribute? " etc)

This would then leave the in-code docs covering:

• Some sort of basic intro, probably pointing to the main site
• The notes about the docs themselves
• Tutorial
• Usage docs
• FAQ
• API docs
• Changelog

Finally, add this entirely new content to the new site:

• Blog for announcements/releases - and then ML posts become links here
• Roadmap page that gets updated periodically with what's ahead

[bitprophet]

Poking at this now, also with an eye for a similar setup for Paramiko and others (Invoke etc).

Feel like there are two minimal-effort ways to go for a static site that integrates with Sphinx API docs:

• Clone my personal Octopress site
• Create a non-RTD-hosted Sphinx site based off of https://github.com/kennethreitz/requests/blob/master/docs/_themes/README.rst

Plus of Octopress site is that it has a blog engine built in, which gives me a useful non-ML/non-Twitter announcement channel, and lets me move that crap off my personal blog where few people will think to look. Minus is that it would be difficult to meaningfully visually distinguish such sites from my personal blog - there's almost no color on it to change, for example.

Plus of Sphinx site is that it'd visually/navigationally integrate better with the API docs on RTD (especially if I were to use the same theme on both parts), and the Kr theme gives slightly more visual room to elements I could change up (name/logo, fonts are heavier so link color changes will stand out more etc). Minus is inverse of Octopress - no useful blog/stream.

---

Alternately, I could try using Git submodules or some other Sphinx import hook (e.g. an 'extension' that simply pulls in content?) to continue hosting everything at RTD, but split out the static content into a separate repo.

Benefit of that is I gain the "keep things that are not version related out of the versioned code repo" angle of a static site, but there's no need to try and bridge two different installations/themes/tools. Downside is, like option 2 above, there's still no blog.

[bitprophet]

Looks like Tinkerer is effectively a Sphinx extension that does blogging - could work well for what I want, or at least serve as a base to hack on.

Given all I really want is "bloglike" behavior for a subsection of the site, that feels like the best compromise. Will poke.

[bitprophet]

Wrote this: http://bitprophet.org/blog/2013/09/29/project-website-musings/

tl;dr probably gonna try the '2x sphinx, w/ the static repo using Tinkerer for blog section' approach.

[bitprophet]

Poking at Tinkerer now, it's pretty opinionated and wants to "own" the entire Sphinx site - need to dig further but not sure it'd be easy to try and shoehorn it into a subsection of a "regular" Sphinx setup.

Then again since it's designed to have non-post 'pages' and that fits the overall UX/IA of these small static sites - it's probably easiest to just let Tinkerer drive things after all & make a custom theme on top of it to match the API site.

[bitprophet]

N.B. poking at paramiko.org for this - once happy with the approach re: how that site works, will then apply same to Fabric.

A fully Tinkerer driven site differs from a traditional Sphinx one (at least re: how I make them, with the repo root being the Sphinx source dir and _build/html being the output dir) like so:

• Blog posts go into top level year-based folders (and then a month/day hierarchy), pages go into a top level pages/ folder. No other top level .rst files exist besides the Tinkerer-managed master.rst (see below.)
• These files need to be created via tinker --(page|post), though they're simple enough - live in the above locations, contain some metadata directives at page bottom, and get added to master.rst below.
• Tinkerer manages a master.rst which holds the top level toctree directive, as per its internals docs.
• It builds into blog/html and puts .doctree files into blog/doctrees (I've never run into these before, not sure what they are offhand; they're binary.)
  • EDIT: this is generated by sphinx -d <path> but apparently it's effectively default behavior (e.g. in Fab's docs there's a _build/doctrees folder alongside _build/html - I just never look in there.)

Things I would want to change if I was to go this route:

• Figure out if I can change what the index page "is" - right now it is another custom-generated HTML page containing the blog homepage, and is called "Home" in the nav.
  • I'd want it to be called Blog and live at /blog/ or /blog.html, and use one of my custom pages as the root.
• Theme it to look like e.g. the Kr theme - unclear how its page structure maps to "normal" Sphinx sites, would have to compare DOMs. All its builtin themes are fugly.
• Update its sidebar to include the desired links to the sister API site (as per the overall approach I am taking here) - seems like I can write a simple extension for that or just update the templates.
• Blog archive is only linked from that home/index page, might be nice to have it linked on any blog page in the sidebar or footer. Also possibly doable just by tweaking the templates.

[bitprophet]

Seems Tinkerer hardcodes the first page of blog pagination as being pagename = 'index' (which is then yielded up a chain until it ends up hitting the builtin Sphinx html-collect-pages event). This is done in ext/aggregator.py in make_aggregated_pages, which is called in ext/blog.py in collect_additional_pages, which is in turn called in html_collect_pages, which is linked to that html-collect-pages event (Sphinx docs).

So I can't easily change this without hacking up the blog/aggregator 'extension' files above :( Would need to copy/paste/modify at least one or two large functions (they're not set up for extensibility.)

[bitprophet]

Turning this around, what's the minimum we want in a blog?

• RSS or Atom feed, so folks can follow/get news
  • A sorted-by-date page (given RSS feeds require ability to do so) could logically follow
  • Which wants pagination, at least eventually.
• Ability to link to individual posts, so we can reference news updates on twitter/irc/etc
• Don't care as much about tags/categories since it's a very focused "blog".

So, possibly reusing chunks of Tinkerer if they make sense, could do:

• Store blog posts in blog/<post name>.rst
• Metadata directive(s) in those files w/ date, maybe tag/category later if desired
  • See if can reuse Tinkerer for this bit
• Extension that hooks into build step, generates the RSS and display page also in the blog/ hierarchy
  • Ditto here, at least for RSS - as above the aggregator page stuff may be too opinionated

So tl;dr a much more lightweight, add-to-your-Sphinx-site version of Tinkerer. Yay, another project :(

[bitprophet]

Have a custom blog extension working for the basics outline above, sans RSS (but since the collection, display & sorting of the blog posts is all implemented, RSS should come easily.) Also sans opening-paragraph display.

Only downside is that the listing (and I reckon RSS will go the same way) is outputted via a regular directive/node within a document, meaning pagination would need to be done by hand or (as it's set up now) simply doesn't happen. Can solve that later - as long as the "first" landing page + the per-post pages do not change URL structure, it should be backwards compatible enough to prevent link rot.

[bitprophet]

Opening paragraph done, RSS done. Time to tie everything together and get theming.

[bitprophet]

Working on a custom variant of Kr/Sphinx theme, see e.g. sphinx-theme #1

[bitprophet]

Have that in reasonably good shape now, back to Paramiko/Fabric static-site specific "how do I want these things organized" stuff. Which may drive more changes to sphinx-theme as needed.