Skip to content

Commit

Permalink
Small updates to dataflow, fix of heading inconsistencies
Browse files Browse the repository at this point in the history
  • Loading branch information
awicenec committed Jul 1, 2021
1 parent b72ae89 commit a317c9c
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 12 deletions.
20 changes: 10 additions & 10 deletions docs/dataflow.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Concepts and Background
-----------------------

This section briefly introduces key concepts and motivations underpinning
|daliuge|.
This section introduces key concepts and motivations underpinning
the |daliuge| system.

Dataflow
^^^^^^^^
Expand Down Expand Up @@ -85,29 +85,29 @@ Concretely, we have made the following changes to the existing dataflow model:

.. _dlg_functions:

|daliuge| Functions
^^^^^^^^^^^^^^^^^^^
|daliuge| provides eight Graph-based functions as shown in
:numref:`dataflow.fig.funcs`.
|daliuge| operational concepts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
As mentioned above, |daliuge| has been developed to enable processing of data from the future Square Kilometre Array (SKA) observatory. To support the SKA operational environment |daliuge| provides eight Graph-based functions as shown in
:numref:`dataflow.fig.funcs`. The implementation of these operational concepts in general does not restrict the usage of |daliuge| for other use cases, but it is still taylored to meet the SKA requirements.

.. _dataflow.fig.funcs:

.. figure:: images/dfms_func_as_graphs.jpg

Graph-based Functions of the |daliuge| Prototype

The :doc:`graphs` section will go through implementation details for each function.
Here we briefly discuss how they work together in our data-driven framework.
The :doc:`graphs` section describes the implementation details for each function.
Here we briefly discuss how they work together to fullfill the SKA requirements.

* First of all, the *Logical Graph Template* (topleft in
:numref:`dataflow.fig.funcs`) represents high-level
data processing capabilities. In the case of SDP, they could be, for example,
data processing capabilities. In the case of the SKA Data Processor, they could be, for example,
"Process Visibility Data" or "Stage Data Products".

* Logical Graph Templates are managed by *LogicalGraph Template
Repositories* (bottomleft in :numref:`dataflow.fig.funcs`).
The logical graph template is first selected from this repository for a specific pipeline and
is then filled with scheduling block parameters. This generates a *Logical Graph*, expressing a workflow with resource-oblivious dataflow constructs.
is then populated with parameters derived from the detailed description of the scheduled science observation. This generates a *Logical Graph*, expressing a workflow with resource-oblivious dataflow constructs.

* Using profiling information of pipeline components executed on specific hardware resources, |daliuge|
then "translates" a Logical Graph into a *Physical Graph Template*, which prescribes a manifest of all Drops without specifying their physical locations.
Expand Down
4 changes: 2 additions & 2 deletions docs/installing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ Direct Installation
**NOTE: For most use cases the docker installation described above is recommended.**

Requirements
############
^^^^^^^^^^^^


The |daliuge| framework requires no packages apart from those listed in its
Expand All @@ -66,7 +66,7 @@ installed on the system:
* gcc >= 4.8

Installing
##########
^^^^^^^^^^

|daliuge| is based on setuptools, and thus it follows the standard python installation
procedures.
Expand Down

0 comments on commit a317c9c

Please sign in to comment.