DevNotes

Overview

For a general schematic of the open tree architecture, see this [architectural diagram] 1. The basic goal of this repo is provide web-service interfaces to the corpus of phylogenetic studies. The corpus is referred to as phylesystem.

The phylesystem-api provides basic read/write access and format conversion. Search functionality is supplied by oti.

Most of the business logic of dealing with the phylesystem corpus is coded in peyotl. This is nice because if facilitates easier code reuse and testing that does not require running the full web stack. This is a pain because devs will need to coordinate merges of peyotl branches and phylesystem-api branches that depend on them.

Workflows

Study editing

A typical series of study edit operations, as choreographed by the open tree curator app (which is running the code in the curator subdir of the opentree repo is shown below. We are in the process of moving from v1 of the API to v2, so some of the URLs used could be stale. The template configuration file holds the patterns used to construct the actual URLs used by the curator app; so you should use that if you need the exact URLs.

1. request brief list of studies and metadata from oti's findAllStudies service
2. user selects a study, and curator app fetches a "NexSON with extra info" using a GET to phylesystem-api's v1/study/{STUDY_ID}.
3. the user corrects various deficiencies of the study, and the curator app saves these changes using a PUT to phylesystem-api's v1/study/{STUDY_ID}

Study creation

1. the curator app prompts the user to enter a new study from scratch or upload a file.

2. No studies "in the wild" will be in NexSON. If the user uploads data to be imported, the curator app uses its own controllers to convert the inputs to NexSON. These calls are documented in the opentree/curator README. Briefly, they are:

   1. to_nexson with the blob of input to use NCL to convert to NeXML and peyotl to convert the NeXML to open tree NexSON.

   2. If there is a previous NexSON blob associated with this study (e.g. if the user is uploading trees as separate newick tree in a series of operations), then merge_otus is called because the conversion of "external" sources to NexSON is not aware of previously created IDs

3. Alternatively, the user can create a new OT study using a tree base ID, in this case the curator app just prompt the user for that ID.

4. A POST to phylesystem-api's v1/study will validate the input, create a new study ID, and return a receipt with the ID and git SHA's for the new study.

Overview of the phylesystem-api's implementation of its part of the workflows

Step 2 of editing - the GET by the curator app:

On the server side this triggers several calls to peyotl's Phylesystem wrapper. The key one's are:

1. phylesystem.return_study
2. phylesystem.add_validation_annotation
3. phylesystem.get_version_history_for_study_id

In terms of the actions performed on the server, these steps entail.

1. the phylesystem-api waits for lock on the phylesystem git repo
2. the master branch is checked out
3. the requested study is read.
4. If a no cached validation annotation for the study is available, then one is generated.
5. The annotation injected into the NexSON
6. the version history of the study is constructed (this is where the call will be after https://github.com/OpenTreeOfLife/phylesystem-api/issues/107 is fixed).
7. the phylesystem git repo is unlocked.
8. the "extra info" is added to the response JSON which will also hold the NexSON

Side note (or slide note, if you prefer)

The slide presentation containing the [architectural diagram] 1 is posted at http://phylo.bio.ku.edu/slides/ot.html and it can be regenerated by running https://github.com/OpenTreeOfLife/phylesystem-api/blob/master/docs/build-presentations.sh)