cookbook for apps

[KatherineCaley]

Apps need documentation!! This should be structured based on the type of apps, i.e., loaders, filtering steps, data sampling apps.

• reorganise the current app documentation

todo: cookbook

• io
• dist
• tree
• align
• evo
• translate
• sample

todo: docstrings

• io
• dist
• tree
• align
• evo
• translate
• sample

[GavinHuttley]

related to #1378

[GavinHuttley]

Following up on how to include code snippets, here is the numpydoc description of how they should be formatted.

[fredjaya]

Running pytest catches the code snippets in a simple example by adding:

• The code snippet in the docstring, with a pass and fail conditions
  https://github.com/fredjaya/DemoProject/blob/ea776bc185b1cd252068619e2f0862a0d34b7c78/src/myproject/cli.py#L94-L101

• doctest options in the .toml
  https://github.com/fredjaya/DemoProject/blob/ea776bc185b1cd252068619e2f0862a0d34b7c78/pyproject.toml?plain=1#L53-L55

Output is:

================ test session starts =================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/frederickjaya/GitHub/DemoProject
configfile: pyproject.toml
testpaths: tests, src/myproject
plugins: anyio-4.1.0, cov-4.1.0, xdist-3.5.0
collected 3 items                                    

tests/test_myproject.py ..                     [ 66%]
src/myproject/cli.py F                         [100%]

====================== FAILURES ======================
_______ [doctest] myproject.cli._parse_csv_arg _______
095 
096     >>> _parse_csv_arg("test")
097     ['test']
098     >>> _parse_csv_arg("test")
Expected:
    ['fail']
Got:
    ['test']

/home/frederickjaya/mambaforge/envs/c311/lib/python3.11/site-packages/myproject/cli.py:98: DocTestFailure
============== short test summary info ===============
FAILED src/myproject/cli.py::myproject.cli._parse_csv_arg

[GavinHuttley]

Thanks @fredjaya ! I'm going to create another issue to trial this for one of the apps.

[fredjaya]

• docstring == True: App contains docstring examples and merged into cogent3/develop.
• assigned == Katherine: Key apps for the wrangling paper.
• * the docstring will contain a URL to the docs for more info

module	name	docstring	cookbook	assigned
cogent3.app.io	load_aligned*		True	Katherine
cogent3.app.io	load_unaligned*		True	Katherine
cogent3.app.io	load_tabular*		True	Katherine
cogent3.app.io	load_json*		True	Katherine
cogent3.app.io	load_db*			Katherine
cogent3.app.io	write_json *		True	Katherine
cogent3.app.io	write_seqs*		True	Katherine
cogent3.app.io	write_tabular*		True	Katherine
cogent3.app.io	write_db *		True	Katherine
cogent3.app.dist	fast_slow_dist
cogent3.app.dist	jaccard_dist	PR open		Fred/Kath
cogent3.app.dist	approx_pdist	PR open		Fred/Kath
cogent3.app.dist	approx_jc69	PR open		Fred/Kath
cogent3.app.dist	gap_dist
cogent3.app.tree	scale_branches
cogent3.app.tree	uniformize_tree			Yapeng
cogent3.app.tree	quick_tree			Yapeng
cogent3.app.align	align_to_ref
cogent3.app.align	progressive_align	PR open	True	Fred/Kath
cogent3.app.align	smith_waterman
cogent3.app.align	ic_score	PR open		Fred/Kath
cogent3.app.align	cogent3_score	PR open		Fred/Kath
cogent3.app.align	sp_score	PR open		Fred/Kath
cogent3.app.evo	model	True		Yapeng
cogent3.app.evo	model_collection			Yapeng
cogent3.app.evo	hypothesis			Yapeng
cogent3.app.evo	bootstrap
cogent3.app.evo	ancestral_states
cogent3.app.evo	tabulate_stats	True		Yapeng
cogent3.app.evo	natsel_neutral
cogent3.app.evo	natsel_zhang
cogent3.app.evo	natsel_sitehet
cogent3.app.evo	natsel_timehet
cogent3.app.translate	select_translatable	PR open		Fred/Kath
cogent3.app.translate	translate_seqs	PR open		Fred/Kath
cogent3.app.sample	concat	True	PR-ready	Fred
cogent3.app.sample	omit_degenerates	True	PR open	Fred
cogent3.app.sample	omit_gap_pos	True	PR open	Fred
cogent3.app.sample	take_codon_positions	True	PR open	Fred
cogent3.app.sample	take_named_seqs	True	PR open	Fred
cogent3.app.sample	take_n_seqs	True	PR open	Fred
cogent3.app.sample	min_length	True	PR open	Fred
cogent3.app.sample	fixed_length	True	PR open	Fred
cogent3.app.sample	omit_bad_seqs	True	PR open	Fred
cogent3.app.sample	omit_duplicated	True	PR open	Fred
cogent3.app.sample	trim_stop_codons	True	PR open	Fred

[KatherineCaley]

I propose a new structure for app documentation that is consistent with the structure of the main documentation, i.e., sections for cookbook and tutorial content. Within the cookbook, we have subsections that loosely reflect the modules within cogent3.app.

The current app documentation on sequence alignment and modelling sequence evolution is cookbook style so can be moved there. The rest of the current documentation within the app is tutorial-style, so it could be moved under that section.

• apps
  • overview
  • cookbook
    • loading data
    • sequence alignment
    • sampling data
    • modelling sequence evolution
    • calculating genetic distances
    • writing data
  • tutorials
    • Testing hypotheses about sequence evolution
    • Testing for natural selection
  • writing and publishing apps

Given the effort that fred is going though to document app usage in docstrings it shouldn't be too much original content, mainly just reorganisation and putting together clear examples.

thoughts/comments @fredjaya @GavinHuttley?

[KatherineCaley]

@GavinHuttley @fredjaya

*for loaders, it's hard to document in a docstring without including data with cogent3; for writers, it's similarly tricky. Should the examples not be strictly runnable but instead be indicative of how to run?

[fredjaya]

Thinking how it fits with the diataxis framework, will "cookbooks" be how-to guides? And is a reference/API section for apps required?

[GavinHuttley]

For the Q1 deadline:

1. focus on docstrings (since that goes out with the code).
2. cookbook (how-to's)
3. docs on "writing and publishing apps"

@KatherineCaley you are right about the docstring difficulty for loaders / writers. So those should probably use .. code-block:: python directives. (May not have the directive name correct...)

[KatherineCaley]

@fredjaya Yes, cookbooks are the how-to equivalent. I, too, am not sure about where API should go. The thesis for separating out the app documentation is that they are a distinct programming style. I've got two ideas (noting these are not necessary Q1 goals):

1. apps get there own API section
2. apps are included in the general API (because they are cogent3 classes after all) and there is a link to this section from the app section

[GavinHuttley]

I have removed the apps that don't need to be plugins from the .toml file and updated the table above accordingly.

[YapengLang]

I'll help it out!

[fredjaya]

Note to check examples with doctest-modules once #1844 is addressed.

[KatherineCaley]

@GavinHuttley @fredjaya todo: Decide on the TOC location, whether it is hidden from the central document or not. Then, apply it systematically to the documents.

Note that the current documents are not consistent!!

see https://cogent3.org/doc/examples/index.html for example of central TOC

see https://cogent3.org/doc/cookbook/index.html for example of LHS TOC only