New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move process/development/etc docs to static site #419
Comments
Poking at this now, also with an eye for a similar setup for Paramiko and others (Invoke etc). Feel like there are two minimal-effort ways to go for a static site that integrates with Sphinx API docs:
Plus of Octopress site is that it has a blog engine built in, which gives me a useful non-ML/non-Twitter announcement channel, and lets me move that crap off my personal blog where few people will think to look. Minus is that it would be difficult to meaningfully visually distinguish such sites from my personal blog - there's almost no color on it to change, for example. Plus of Sphinx site is that it'd visually/navigationally integrate better with the API docs on RTD (especially if I were to use the same theme on both parts), and the Kr theme gives slightly more visual room to elements I could change up (name/logo, fonts are heavier so link color changes will stand out more etc). Minus is inverse of Octopress - no useful blog/stream. Alternately, I could try using Git submodules or some other Sphinx import hook (e.g. an 'extension' that simply pulls in content?) to continue hosting everything at RTD, but split out the static content into a separate repo. Benefit of that is I gain the "keep things that are not version related out of the versioned code repo" angle of a static site, but there's no need to try and bridge two different installations/themes/tools. Downside is, like option 2 above, there's still no blog. |
Looks like Tinkerer is effectively a Sphinx extension that does blogging - could work well for what I want, or at least serve as a base to hack on. Given all I really want is "bloglike" behavior for a subsection of the site, that feels like the best compromise. Will poke. |
Wrote this: http://bitprophet.org/blog/2013/09/29/project-website-musings/ tl;dr probably gonna try the '2x sphinx, w/ the static repo using Tinkerer for blog section' approach. |
Poking at Tinkerer now, it's pretty opinionated and wants to "own" the entire Sphinx site - need to dig further but not sure it'd be easy to try and shoehorn it into a subsection of a "regular" Sphinx setup. Then again since it's designed to have non-post 'pages' and that fits the overall UX/IA of these small static sites - it's probably easiest to just let Tinkerer drive things after all & make a custom theme on top of it to match the API site. |
N.B. poking at paramiko.org for this - once happy with the approach re: how that site works, will then apply same to Fabric. A fully Tinkerer driven site differs from a traditional Sphinx one (at least re: how I make them, with the repo root being the Sphinx source dir and
Things I would want to change if I was to go this route:
|
Seems Tinkerer hardcodes the first page of blog pagination as being So I can't easily change this without hacking up the blog/aggregator 'extension' files above :( Would need to copy/paste/modify at least one or two large functions (they're not set up for extensibility.) |
Turning this around, what's the minimum we want in a blog?
So, possibly reusing chunks of Tinkerer if they make sense, could do:
So tl;dr a much more lightweight, add-to-your-Sphinx-site version of Tinkerer. Yay, another project :( |
Have a custom blog extension working for the basics outline above, sans RSS (but since the collection, display & sorting of the blog posts is all implemented, RSS should come easily.) Also sans opening-paragraph display. Only downside is that the listing (and I reckon RSS will go the same way) is outputted via a regular directive/node within a document, meaning pagination would need to be done by hand or (as it's set up now) simply doesn't happen. Can solve that later - as long as the "first" landing page + the per-post pages do not change URL structure, it should be backwards compatible enough to prevent link rot. |
Opening paragraph done, RSS done. Time to tie everything together and get theming. |
Working on a custom variant of Kr/Sphinx theme, see e.g. sphinx-theme #1 |
Have that in reasonably good shape now, back to Paramiko/Fabric static-site specific "how do I want these things organized" stuff. Which may drive more changes to sphinx-theme as needed. |
Next up for this site junk is tying them together:
|
Re: where changelogs live:
I started writing this thinking "yea, changelogs should be in the static site" but then I thought of the 2nd counterpoint above. I'd rather continue having multiple changelogs (especially ones that tend to only differ via additives, given how they merge currently) than require users to submit PRs to two repos. So...never mind. On the other hand - now that Releases is in use, older changelogs can have misleading info still. To wit:
So that behavior kind of wants a single changelog, but still not bad enough to be worth 2x repos. (Maybe move the static site repo into the main repo but build them separately? Worth thought.) Other possible fixes to the above:
|
ALso, already seeing real world examples of exactly what I fear - folks looking at the 1.8 changelog and not seeing the 'unreleased feature' stuff from master (because 1.8 branch doesn't know about them!) Definitely worth seeing if 'one repo, two doc trees built separately' is possible. Hopefully just tell RTD to look in |
In fact I just checked and RTD does let you build a project with a single version, aka no So the single-repo approach actually looks feasible now. Will try it out in a branch w/ paramiko + .org. |
Next up for Paramiko's in progress version of this setup:
|
Intersphinx, from perspective of a changelog or other www-site file referencing stuff in the docs/api site:
|
Yup RTD has a slot for conf.py location, so I should be able to get away with more import-based shenanigans. Set up a "Paramiko (WWW)" project on RTD to get this started testing as well EDIT: Spoke too soon, RTD seems to have a bug preventing it from switching Git branches. Trying to work around for now so I don't have to troubleshoot this via master. EDIT 2: Can work around git issue by just making a different Version entry, but then run into this issue. Can prob work around that by just being stupid and using EDIT 3: And now RTD is assuming my doctree root / master doc (e.g. EDIT 4: Yup, same thing actually, |
Well that was a fun half day. Turns out I was really stupid for forgetting the Now have a working setup where local builds of one doc site can reference the other, and same for public. Would be easy to extend it for reciprocal intersphinxing, but for now that's not even necessary (the www sites for projects are unlikely to contain material linked to from the API/usage docs.) |
On to navigation stuff (both general and attempting to unify the 2 sites):
Concerns specific to this new split-doctree world:
|
Sadly Intersphinx inventory files only seem to track specifically targeted references, i.e. So my hope to "pull in all 'document' like targets from the other site and stick in nav" isn't going to fly. Dunno if I can do something super dumb like straight up generate a new toctree object from the other site, or if I have to go full manual and just assume the top level pages for both sites won't change often. Or just link to the other site w/ one link and not worry about trying to show a unified nav. |
|
Poking at nav stuff, again trying to keep things quick n dirty to get things moving wrt Paramiko:
|
Went with an in-alabaster nav that can accept settings for external links. Works well enough for now. Also a bunch of style updates so the nav/toc is a bit bigger/nicer/etc. Shutting up in here & moving to the Paramiko ticket for specifics. But this approach seems to work pretty well now so I'm totally trying it for Fabric next! |
Applying this to Fabric now:
|
In lieu of rewriting nearly everything, these are changes that should get applied once the above is done (but sooner, not later):
|
Turns out Paramiko's link colors were actually appropriate for the Fabric "livery". Will probably just remove them from Paramiko so it uses the theme default for now. |
Realized that the new "docs" site will need to continue showing the current (and soon, "old") versions of the all-in-one site. No real way to avoid this unfortunately - cannot in good conscience remove that info from the Web so we'll just have to deal. Does mean that I will need to be careful about RTD sites - if I make two new ones as I was planning, I'll have to go through and recheck all the build boxes for all currently published versions. Alternately, just use one of the two new ones as a testbed, then apply its changes to the old one when I go-live. However that leaves us with confusingly named projects on the RTD listing/admin side, so possibly best to suck it up & copy over the checkbox state. |
Migration checklists! RTD:
DNS:
Nginx:
Finally:
|
Just realized that the non-websites-merged branches will not build naturally as I have to use a different settings file path for RTD for this type of setup ( I think I can get away with changing this setting back temporarily (pre launch), building all the old docs (whee), then setting it back. As long as nobody tries to go back and trigger builds of the old versions, should be fine? |
Realized I am a total moron and I can simply take the old/current 'Fabric' project and...rename it. Yes, it means once I change its settings to use the new shit, it will appear semi broken until DNS finishes percolating, but I think that's preferable over forcing RTD to rebuild a bazillion old versions. I tested that inter-linking works fine with the two new sites now, so tomorrow (tonight??) I can just nuke the fabric-docs one and mutate the existing one instead. |
Minor problem with the 'reuse the old project' idea, RTD doesn't seem to tweak slug/subdomain when a project's name changes. Not too surprising though. Pinged the fine folks there via IRC; will keep the "new old" fabric-docs project around as filler until it is resolved. The only downside is that older versions of the docs (1.6 and below) will appear to disappear until then. This is OK for a short period of time; if they don't respond by tomorrow midday I'll suck it up and go back to Plan A, "build all the versions! on fabric-docs". |
Also ran into some odd Travis failures that don't seem like they'd be related to anything o_O The docs part did get overlooked, and I just fixed that, but actual test suite errors were firing off. May need to compare to e.g. a recent release tag to see WTF, but as of eg f49d6cd the suite was passing... |
More docs failures (working on those, derp. circular dependencies FTL!) but also still the regular test failures - ones I cannot recreate locally. Really strange; they don't suggest any obvious 'broken on Travis only' pattern offhand. EDIT: was testing on 2.5 as that's still my default venv; trying 2.6 now. |
Also, the docs subdomain continues to look like the old site even when my local DNS seems updated. Suspect it's because I never did force builds of 1.7/1.8 on the new/temp project; thus they would 404 and maybe that is triggering use of the old content. Building those to see. |
And nope, can't recreate the test suite failures on 2.6. Super bizarre. Will need to dig further and possibly stick debug printing in, tomorrow. |
Also also, the other issue (stale subdomain) isn't resolved after those builds. Clearly trying to tackle all this late in the evening was a stupid idea. Will see what the morning brings. |
Hm. So at docs.fabfile.org, /en/latest does have the new site, but /en/1.8 does not. So clearly it is hitting the new fabric-docs project (as I updated DNS and saw nothing from RTD yet), but unclear why the builds didn't update w/ new data. Manually forcing new builds of 1.7/1.8 to see if it was just bad timing. |
That seems to work, though the redirect at apex of docs.fabfile/fabric-docs.rtd was going to 'latest' not '1.8' despite 1.8 being the default version on the versions page. However it was not the default on the advanced settings page (was blank) so I set it there and seems to work now. Worried that is somehow distinct from the Versions page dropdown, and may not update when I push a new branch. Will have to see. |
Going to just start building the old docs on fabric-docs so I don't have to worry about the RTD guys having time to migrate the subdomain (or if it's just not really doable.) A few at a time. |
Actually, to limit clutter in the version listings, I'm for now just going to check off the newest version for each of the older release lines that lack the per-branch builds (so ~1.4 backwards). So e.g. 1.3.8 and so on. It's trivial to rebuild the rest later, and each of those versions includes the previous ones in their changelogs (and versionchanged/added info bits) so this should be fine. |
Back to Travis - the "build www without warnings, build docs with warnings, build www with warnings" attempt still fails because the status badge (yes...the travis one...hah) is nonlocal and generates warnings. This wasn't an issue on Paramiko as its website doesn't include the badge. Not sure if there is a good way to deal with this + the circular dependency junk without just going back to non-warnings-are-errors style builds :( |
Wonder if moving that sort of nonlocal image into the sidebar would make Sphinx shut up, since it would no longer be part of the ReST and instead be part of the template. |
n.b. filed readthedocs/readthedocs.org#770 so I can remove the old project w/o it reflecting poorly on anything. |
Success, moving the Travis button to be a configurable Alabaster option makes Sphinx STFU. |
Fixed what seems like the only other outstanding doc level warning-fail, meaning those bizarro failing tests are all that is left. |
Got it, a dumb tweak added to the test runner to skip the new top level Invoke tasks file, was causing the integration test run to invoke both suites; the default suite gets kind of mad if you run it with |
I...I think this is done! Only took 3 freakin' years. |
\0/ |
Most of the development and index pages of the current docs would work better as a dedicated static site, where changes can be made without having to copy them to each release branch, etc.
This will also be a good excuse to rearrange a little bit.
Specifically, move these to the new site:
index.rst
which is not the Documentation section -- so the intro/about, installation, development, and getting help sectionsThis would then leave the in-code docs covering:
Finally, add this entirely new content to the new site:
The text was updated successfully, but these errors were encountered: