A script to generate options reference pages for the docsite.

[benjyw]

• Uses mustache templates (plus some light munging in python) to
  generate the content.
• The script can sync the pages to the docsite, or just render
  them locally, for ease of iteration and debugging.

Makes some small changes/refactors to the help info extracter,
to support the markdown/HTML generation.

Removes a docstring line in external_tool.py that contained
< > characters that rendered as >/< in the markdown.

[ci skip-rust-tests]

[benjyw]

See the result on the live docsite: https://www.pantsbuild.org/v2.0/docs/

[benjyw]

[Eric-Arellano]

This is excellent!

[stuhood]

So, readme's search is much, much better than our old search. But I still sortof wonder whether we should inline all related options into each goal's page so that you can truly see "all options that matter to test" at a glance. Ironically, a downside might be that it would make search results worse.

[Eric-Arellano]

So, readme's search is much, much better than our old search. But I still sortof wonder whether we should inline all related options into each goal's page so that you can truly see "all options that matter to test" at a glance. Ironically, a downside might be that it would make search results worse.

I don't think so. Having related subsystems is already awesome. There isn't much benefit imo to inlining the related subsystem options, other than saving a click. The downsides are a) worse search, and b) much larger pages, which are less performant and imo a little overwhelming to look at.

[stuhood]

There isn't much benefit imo to inlining the related subsystem options, other than saving a click.

The benefit (afaict) is that when you want to figure out "how do I enable XML for the test goal", find in page on the test goal page will tell you that. Otherwise, you have to do a global site search for XML, and potentially find a bunch of matches that aren't related to test running at all. You would then manually filter those matches to figure out which of them are transitively used by test, maybe.

The same goes for the CLI I think (./pants help-advanced test should give you at least a hint of the names of options that might be relevant), but moreso because we don't have search there.

---

One option would be to inline a super abbreviated version with links to the actual help pages maybe...

Basic options
--[no-]test-debug
default: False
Run a single test target in an interactive process. This is necessary, for example, when you add breakpoints to your code.

<snip>

Related subsystem options

pytest
  --pytest-junit-xml-dir
  --pytest-timeout-default

... where each related subsystem option was a link to the page.

[benjyw]

Going to merge this, as we can and will modify the page content in any number of followups...