Config Builder in the future !

Current pipeline config and suggested changes

Each step in the pipeline_config always has an input role, and one or more output roles. At present, there also exists a concept of role priority; as such we cannot display the current pipeline config as a graph (in the mathematical sense). In order to be able to plot the pipeline config as a graph, we need to remove the idea of role_priority. This will have consequences for the rest of the code, which will be addressed below.

At present, only the step_controller and selection_page_presenter are considered as parent presenters in the pipeline_config. This can be made more generic, encapsulating all presenters in a page presenter. The role of the page controller is simple: it encapsulates other presenters/controllers required for a page, and displays them accordingly.

Each page controller will contain:

• ONE Role changing controller (eg selection page, connected, rack scan) - NECESSARY
• Any number of non-role changing controllers (eg kit) - OPTIONAL

A "role changing" controller is a presenter which is in charge of changing the role of the input labware from "input role" to "output role".

A "non-role changing" controller is a presenter/controller which has no impact on the roles. For instance, the kit_presenter is necessary to achieve a task, but by itself does not change the roles.

If there is no role changing controller within our page, then the process ceases to advance; therefore, we need a 'role-changer'. For instance, out of our existing presenters, the page presenter could have one of either the selection_page_presenter, connected_presenter or rack_scan_presenter as its role changing presenter.

At present, we could re-use the step_controller as a page_presenter. The step presenter could be changed, such that it no longer has buttons, or anything to display of its own. Instead, it just takes us from one presenter to another: from one step to the next. This means that each presenter has to tell the page-presenter when it can move to the next presenter (ie when that presenter is 'done').

This would be useful in making the display more compact. One of our main issues is the display taking up too much space on the screen; the user has to scroll to see everything. Instead, we could hide, or fold up (jQueryUI accordion for example), the controllers on the page when they are done, so that the user is guided through the process. Another option would be to have a column on the left of the page with an overview of the processes; the user sees the currently selected process.

Work that would need to be done

The Config Builder -> this would be a tool which allows a developer to create a pipeline-component file graphically, from which we can build our pipeline_config. The pipeline-config with components file (from here referred to as the plc_config file) is created by the config builder, and contains all the information the config builder needs to display a fully formed graph, with components included. This is NOT the same thing as the pipeline_config (from here referred to as the pl_config) file which is used in our code.

As discussed above, the idea of role-priority will need to be removed. This will make the pl_config more complex, but this additional complexity will be handled by the Config Builder, since pl_config is generated by the config builder.

Before making the config builder

The below estimates include time to for manual testing, and automated testing where appropriate. These are changes to the existing code, so it is crucial that when these changes are made, we don't break the existing code!

At this stage we already have a skeleton graph running, using a force layout in d3. It is running as a separate app, is accessible on local_server_url/graph.html. This can be found on this branch.

Task	Description	Estimated Time
Change the structure of our existing code	The existing pipeline_config will need to be restructured to be more homogeneous, as discussed in this section. **Time: 4hr** workflow_engine -> how it finds the next workflow will need to change. **Time: 3hr** step_controller -> this will take on the role of a generic page presenter. Any of its own display items must be removed. The step_presenter must also take on input and outputs: these will get passed to the "role_changing_presenter" contained in the step_presenter's sub-presenters. **Time: 4 hr** Other controllers will need to change to contain their own buttons, namely the connected_presenter and the rack_scan_presenter. **Time: 3hr**	14hr
Create config_templates as json files, one for every controller.	See the config templates section in this document for more information.	3hr
Setup code to support a graph html page	Create a new graph_app file and html page for the config_builder.	Time: 0hr: this has already been done.

The graph will need:

Task	Description	Estimated Time
Fixed "fake root" node	A fake controller (a node in the graph) where all the entry roles come from. This will need to deal with the all the possible input roles (currently 6; Manual and Automated extraction each have 3: DNA and RNA, DNA only, RNA only) when a new graph is created, all that should be displayed is this fixed root node as a starting point. User should not be able to delete the root node.	2hr
Context menu -> each controller should have a context menu which displays when it is clicked on. This context menu could be displayed in a separated div next to the graph. The context menu should contain:	a section that displays the config given to the controller. This includes sub-presenters, inputs, outputs etc. This is a high estimate as presentation is an issue here: for a page presenter with many sub presenters, the configuration of each of these sub-presenters will need to be displayed! **Time: 6hr** the config is editable, whether this is directly on the context menu, or by clicking a button etc. Changes here will need to be reflected in the data. **Time: 6hr** an option to delete a node. This will have the consequence of isolating the rest of the process! A warning to the user may need to be made. Also, this option does not apply to the "fake root" node as described above. **Time: 3hr**	15hr
Add a node, representing a page controller ->The user should be presented with the option to add a node when another node is clicked -> the new node will be linked by default to the new node. The process is as follows:	Choose sub-controllers that will go in the page presenter. This will require the creation of a page which allows for this selection. **Time: 3hr** Set up input role: this should come from a drop-down list of all of the output roles of the *source* node. **Time: 2hr** Set up the output role **Time: 1hr**. Each sub-controller needs to be configured. The configuration options presented to the user will be as given in the appropriate config_template for that presenter; the layout of this page will change depending on which presenter is being configured **Time: 6hr** Add the newly configured page (node) to the graph. Might need a check: do any of the output roles match any input roles on other controllers? If they do, links need to be made! **Time: 2hr** Refresh the graph **Time: 1hr**	15hr
Toolbox -> Note that the below estimates do not factor in the functionality, rather just the creation of the toolbox in the view. contains options to:	Add a component (from the components that have been created previously). View the graph of an existing component. Create a new component (which you can select from a drop-down list).	2hr
A way of creating a component -> detailed information can be found in the section about components in this document.	create a view for this -> this will be the almost identical to the graph view, since a component will just consist of controllers (nodes) and role changes (links). On this view, need to be able to add nodes, and be able to edit the role extensions on the links. Depends on "add a node" task described above. **Time: 6hr** save button for when the component is finished. This will cache the component in memory, not in a file. **Time: 3hr** component is added to a drop down list in the toolbox when it is saved **Time: 1hr** specification note: The input role on the first controller of the component should be fixed to a default, e.g. input_role. This will just be a dummy -> the actual input role will be given by the role which is given as an input to the component. This will need to be included in the "add a node logic". **Time: 3hr**	13hr
A way of placing a created component in to the graph. A component should be reusable -> can appear multiple times on a graph	when inserted, user could be presented with a list of current output roles that aren't in use as inputs: component can be placed here. Alternatively, the user could add the component from the context menu for a node. Either way, component won't make sense without an input role! Possibility of multiple input/ output links. One input for every output. Output role is computed by component, based on input role.	6hr
make it obvious to the user where there are end of process pages/ nodes	These correspond to nodes without an output role, e.g. the wate tube controller.	2hr
view of the sub graph of component	You should be able to view any existing component, and be able to edit it in the same manner as creating a component.	3hr

On top of this, we will want to be able to:

Task	Description	Estimated Time
Save graph in a plc_config file that contains all the information needed to display the above mentioned graph	put a save button in the view **Time: 1hr** decide on a file format -> this needs to contain all configuration, **and** and the information about the saved components. **Time: 4hr**	5hr
Load a previous graph from a *plc_config* file	put a load button in the view **Time: 1hr** same drag and drop functionality that we have in the manifest reader for loading a file. **Time: 2hr** write a parser for the plc_config file and test **Time: 3hr**	6hr
Use a previous plc_config file to generate a *pl_config* file.	Using the saved information about components and controller configuration, we should be able to generate a full pipeline_config. This is the main purpose of this app!	4hr

In total, these estimates sum to 90 hours. This is optimistic, since these issues will only become clearer once work begins. It would be sensible to round it ~120 hours, to include contingency.

Config templates for controllers

When a controller is set-up, it needs a set of configuration parameters. The configuration parameters that are required vary from one presenter to the next. Since we don't have a generic configuration structure that we can use for all presenters, we need something to specify what each presenter needs. We can create a "config template" file for each presenter, so that we have a clear definition of how each presenter gets configured.

Each config template will define the following for a controller:

• what properties it can have
• which are required
• which are optional

These templates can be defined as in the below example:

{
	"title": "required",
	"input": {
		"role": "required"
	}
	"output": [{
		"role": "required",
		"batched": "not-required",
		"tracked": "not-required"
	}]
}

Note: Input and output always have the same list of required and optional properties.

Components

The config_builder module would be a graphical tool for building pipeline_config. Each node will represent a page controller, and each link between nodes represents both the output role of the source node, and the input role of the target node. (one edge === one role)

The way that roles have been defined in the pipeline_config allows us to break down parts of the overall process into components. It should be noted however that this is purely a display feature only within the config_builder! The concept of a component should not need to be represented in the pipeline config in any way, other than in the role names.

Definition of component: Here, the word component used, since this is how it will be displayed graphically. In reality, a component is just several controllers, linked by intermediary roles, no different any of the other presenters. The only difference is in how the rolenames are defined.

For example, in the pipeline config we can see:

->qD.input_tube_dnap.to_be_retubed.batched -> qD.input_tube_dnap -> qD.input_tube_dnap.batched<-

Here, the prefix of qD.input_tube_dnap persists in the role-name. This part of the process could be displayed as a component, in order to simplify the presentation of the pipeline_config.

In some parts of the process, the rolenames are modified with different suffixes. Again it must be noted that this idea only appears in how the roles are defined. If their definition changes, then we have no concept of components in the pipeline_config.

Below is a basic illustration of how we can pull out a component in a general case.

We could factorise the intermediate states into a component, such that we get a role, * , going in, and a different role, *' , coming out.

We can also have a component that is re-used in order to display a section of the process that is performed in the same way. Again, an example is shown below.

According to these rolenames, both a and b undergo the same process. As such, we should be able to have both of the processes using the same component.

A component should be able to be viewed in greater detail - clicking on a component will display a sub-graph of all the page_controllers making up that component. If a component is edited within its subgraph, all places in which the component is used will have the same changes.

New pipeline config template

A step in the pipeline_config could be as simple as outlined below. Each step, is representative of a single instance of the page_controller in the workflow.

Note: Each step requires an input and an output; these should be passed to the last controller in the presenters array. This is because the role-changing presenter must be the last presenter; if it weren't, the process would move on to the next page presenter, prior to showing any of the presenters listed after the role-changer!

{
	input: {…},
	output: [{…}, …],
	controllers: [
		{ 
			controller:presenter1,
			config:config1
		},
		{ 
			controller:presenter2,
			config:config2
		},
		{ 
			controller:presenter3,
			config:config3
		},
	]
}

Each controller needs to be specified by name, so that the presenter_factory can construct it. It also needs configuration parameters specific to the presenter type, as discussed in this section.