Revamp DOCS, fo realz

[goodboy]

Our docs are terrible and outdated.. ya we know 😂

Yes you can find that terrible version (at time of writing) here:

• https://tractor.readthedocs.io/en/latest/

---

sphinx themes I'm thinking we should use:

These are the hot ones I've found really lovely to read in python
(related) land:

• general theme tips:

  • https://www.sphinx-doc.org/en/master/usage/theming.html
  • https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.html.DirectoryHTMLBuilder

• obviously the sphinx_docs_theme which I've already opened
  issues on ages ago..

  • RTD render doesn't show a .svg logo? executablebooks/sphinx-book-theme#365
  • https://github.com/executablebooks/jupyter-cache/blob/master/.readthedocs.yml
  • https://sphinx-book-theme.readthedocs.io/en/latest/configure.html
  • UPDATE: Sphinx v4.0.0 executablebooks/sphinx-book-theme#364
  • executablebooks/jupyter-cache@3519065
  • https://sphinx-book-theme.readthedocs.io/en/latest/layout.html
  • https://github.com/executablebooks/sphinx-book-theme/blob/master/sphinx_book_theme/topbar.html#L27

• msgspec with its use of furo:

  • https://jcristharif.com/msgspec/usage.html
  • https://github.com/pradyunsg/furo
  • FYI, docs deps list: https://github.com/jcrist/msgspec/blob/main/setup.py#L34

• ray with it's similar theme: https://docs.ray.io/en/latest/ray-core/walkthrough.html

  • seems to use the sphinx_book_theme: https://github.com/ray-project/ray/blob/master/doc/source/conf.py#L263
    (like i originally mentioned in this issue..

• polars' user guide: https://pola-rs.github.io/polars-book/user-guide/index.html

  • uses the pydata_ theme I guess: https://github.com/pola-rs/polars/blob/main/py-polars/docs/source/conf.py#L79
  • https://pydata-sphinx-theme.readthedocs.io/en/latest/index.html

• pandas also using the pydata theme:

  • https://pandas.pydata.org/docs/user_guide/basics.html#head-and-tail
  • https://github.com/pandas-dev/pandas/blob/master/doc/source/conf.py

---

Diagramming frameworks

Once we've settled on a (better) theme we need to add a ton of
diagrams with something that can embed in RST ideally, though we
might have to do some automation to just generate using some other
tools?

• mermaid seems the obvious answer but it's markdown only AFAIK:

  • general docs:

    • https://github.com/mermaid-js/mermaid
    • https://stackoverflow.com/questions/50762662/how-to-install-mermaid-to-render-flowcharts-in-markdown
    • https://mermaid.js.org/syntax/examples.html

  • CLI editor B)

    • https://github.com/mermaid-js/mermaid-cli
    • maybe TODO: hook this up with xonsh, maybe make a xontrib?

  • online editor!

    • https://mermaid-js.github.io/mermaid-live-editor/edit/#pako:eNplj00KwjAQha8SZpOF9QJZdKUncBsoYzJqaPNjmoCl9O6mJAjFWQzDe9-DNysorwkEzPTO5BRdDD4jWulYmYAxGWUCusTIznqgD6lh8j78-_foR4q6GnUfI-e-PzVIsJWjSsY7Lhi_54VvNdH8HT2GC4Zq5JWCDixFi0aX4uuuSUgvsiRBlFNjHCVItxUuB42JrtokH0E8cJqpA8zJ3xanfkKl2utN3b7EcGB3
    • example from piker emsd order msg flow:
      https://mermaid-js.github.io/mermaid-live-editor/edit/#pako:eNqVk11PwyAUhv8K4YYZq-k0xoXEJX5de7HbXozBmZJR2h2oUZv9d-lKXddlOrmgcHre53wANZWFAsqpg3UFVsKTFq8o8sySMEqBXktdCuuJNBqsP7RD7tShdYHFCjD-aOcWcDGdnjcSTl5QAY7qErUETq7TMBLi9FfYpJfpOCHwAZITZvQ7sISwQisWtreL8QSu5puzltqwGmYMyMlDu4h0tsWzHz5rArAuwhFmZO1Sjcx7uRrVQbJTJARh3TOwYVpt0Zz8Q3os-swLX7lRbUUeWsSW2hjWa1h6cyx2J0Rw5a_C4Qk9inAhzCDvvxrfifarm59andt-75jcYgwoNmzU_MQ6-4Riv9E0oTlgLrQKN79uaBn1b5BDRnlYKoGrjGZ2E_yqUgkPz0r7AilfCuMgoaLyxezTSso9VtA5xacTvTbfCPwRJg

    sequenceDiagram
      participant client
      participant emsd
      participant brokerd
    
      client->>+emsd: Order({price: 30000, size: 0.01, exec: 'live', 'oid': '7b18e2`})
      emsd->>+brokerd: BrokerdOrder({'price': 30000, 'size': 0.01, 'oid': '7b18e2`})
      brokerd->>+emsd: BrokerdAck({oid: '7b18e2`, reqid: '7b18e2'})
      emsd->>+client: Ack({oid: '7b18e2`, reqid: '7b18e2'})
      brokerd->>+emsd: BrokerdStatus({name: 'fill', size: 0.005})
      emsd->>+client: Status({resp: 'fill', size: 0.005})
      client->>+emsd: Cancel({oid: '7b18e2`})
      emsd->>+brokerd: BrokerdCancel({reqid: '7b18e2`})
      brokerd->>+emsd: BrokerdStatus({status='cancelled', reqid: '7b18e2`})
      emsd->>+client: Status({resp: 'cancelled', oid: '7b18e2'})

    which renders to
    

  • obsidian integrations:

    • https://github.com/dartungar/obsidian-mermaid
    • https://heymichellemac.com/knowledge-management-flow-diagram-in-obsidian

• Pikchr: a
  PIC-like markup language for diagrams in technical
  documentation. -> looks decent but not sure if it's too niche?

  • examples: https://pikchr.org/home/doc/trunk/doc/examples.md?skin=darkmode&once
  • online editor: https://pikchr.org/home/pikchrshow

---

Docs frameworks

Alts and/or extensions to sphinx worth a shot?

• just heard about mkdocs which apparently has sphinx
  inventory compat now?
  • https://www.mkdocs.org/getting-started/
  • auto-gen from docs strings: https://mkdocstrings.github.io/

---

Content/section IDEAS

Now that #129 is about to land we need some sections on:

• using the debugger in a variety of process tree configurations

  • examples of both .breakpoint() and the crash handler system
  • example of how it's like pytest's --pdb even with just plain old process local tasks

• adjustment of all examples to drop use of namespace prefixes when calling Portal.run() and ActorNursery.run_in_actor() once a fix for Allow passing function refs to Portal.run() ? #69 lands

  • focus on simplicity of invoking remote functions with simple types as input
  • demonstrate how one should not rely on pickling objects through an example of state mutating?

• re-org all major sections into new pages, a rough outline might look like:

• spawning basics with process tree snap shots for each example

• cancellation and error propagation

• debugging

  • examples of using logging to figure out how things went wrong if the debugger isn't enough

• streaming and a real world example

• process tree and messaging patterns

  • maybe should include some details about backends and gotchas?

• pretty up the readme including Add logo and github actions badge to readme #153

• a stylistic change would be good where we introduce the
  project as something like *trio-across-processesand/or a next-genmultiprocessing`:

  • start examples with talking about processes
  • compare IPC with channels to equiv mp data structures
  • discuss sync code calling from async function as being better then relying on sync IPC systems

• there was an existing issue RE a streaming example:
  Add a simple real-time streaming example to the README #114 (comment)

[goodboy]

The readme update in #163 also mentions maybe adding some diagrams for both messaging / protocols docs and supervision trees stuff:

add some process tree diagrams using both something like mermaid.io and output from pstree?

[goodboy]

There was a project i saw recently that had a sphinx theme I really loved; too bad can't find it again.

It had the automatic toc arrows pointing to the section on the left as you scrolled down the page.

[ryanhiebert]

I'm not sure, but I've seen a couple projects using https://sphinx-book-theme.readthedocs.io/en/latest/, and I rather like it. Was thinking about using it for some internal documentation at work.

[goodboy]

@ryanhiebert oh nice! Yeah this is definitely getting closer.

I'm looking for a very simple black and white theme and a nice big logo.
This is one of the closest I've seen 🏄🏼

Looks to be based on the pydata themes.

Damn, they made pandas docs look good.

[goodboy]

So sphinx-book-theme turns out to be the same one that ray uses.

I like a lot about it but think it could be simplified a bit to fit the style I've got envisioned for this project.

I've created executablebooks/sphinx-book-theme#296 which seems to so far be well received 🥳

Summary:

• move top right nav-bar into left side-bar thus removing a bit of clutter (especially when using the margins support)
• possibly leveraging the built-in support for local headings into the left sidebar

I've pushed up some draft changes on https://github.com/goodboy/tractor/tree/new_docs_polish

[goodboy]

Made executablebooks/sphinx-book-theme#365 to address missing logo stuff.

@Fuyukai also mentioned in chat that our github badge link is borked 😿

[goodboy]

as per msgspec docs, apparently the hip new minimalist theme is this bby:
https://github.com/pradyunsg/furo

[goodboy]

Also learned about mkdocs the other day which apparently has sphinx compat inventory generation?

• https://mkdocstrings.github.io/
• https://www.mkdocs.org/getting-started/