Skip to content

Commit

Permalink
Merge a7be535 into 2c341d2
Browse files Browse the repository at this point in the history
  • Loading branch information
tiborsimko committed Apr 23, 2018
2 parents 2c341d2 + a7be535 commit 080eabb
Show file tree
Hide file tree
Showing 9 changed files with 375 additions and 331 deletions.
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ About
analyses. It permits researchers to structure their analysis data, code,
environment and workflows in reusable manner. REANA command-line client allows
users to instantiate and run computational research data analysis workflows on
the cloud.
remote containerised compute clouds.

REANA was born to target the use case of particle physics analyses, but is
applicable to any scientific discipline. The system paves the way towards
Expand All @@ -39,7 +39,7 @@ Features

- help to structure research data analysis in reusable manner
- instantiate computational workflows on remote clouds
- rerun analyses with modified input data or code
- rerun analyses with modified input data, parameters or code
- support for several compute clouds (Kubernetes/OpenStack)
- support for several workflow specifications (CWL, Yadage)
- support for several shared storage systems (Ceph, EOS)
Expand Down
141 changes: 133 additions & 8 deletions docs/administratorguide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,137 @@ Administrator guide
===================

This administrator guide is meant for people who would like to deploy and manage
REANA clusters.
REANA clusters. (The researchers are probably interested in reading the
:ref:`userguide` instead.)

Architecture
------------

REANA system is composed of multiple separated components that permit to define
and manage computing cloud resources that run computational workflows on the
cloud.

.. image:: /_static/reana-architecture.png

REANA uses the following technologies:

- `Python <https://www.python.org/>`_
- `Flask <http://flask.pocoo.org/>`_
- `Docker <https://www.docker.com/>`_
- `Kubernetes <https://kubernetes.io/>`_
- `RabbitMQ <http://www.rabbitmq.com/>`_
- `Yadage <https://github.com/diana-hep/yadage>`_
- `CWL <http://www.commonwl.org/>`_
- `EOS <https://github.com/cern-eos/eos>`_

Components
----------

REANA system is composed of multiple separated components that are developed
independently. The components are usually published as Python packages (user
client, administrator cluster management) or as Docker images (internal REANA
components).

reana-client
~~~~~~~~~~~~

REANA command line client for end users.

- source code: `<https://github.com/reanahub/reana-client>`_
- release notes: `<https://github.com/reanahub/reana-client/releases>`_
- known issues: `<https://github.com/reanahub/reana-client/issues>`_
- documentation: `<https://reana-client.readthedocs.io/>`_

reana-cluster
~~~~~~~~~~~~~

REANA component providing utilities to manage cluster instance.

- source code: `<https://github.com/reanahub/reana-cluster>`_
- release notes: `<https://github.com/reanahub/reana-cluster/releases>`_
- known issues: `<https://github.com/reanahub/reana-cluster/issues>`_
- documentation: `<https://reana-cluster.readthedocs.io/>`_

reana-job-controller
~~~~~~~~~~~~~~~~~~~~

REANA component for running and managing jobs.

- source code: `<https://github.com/reanahub/reana-job-controller>`_
- release notes: `<https://github.com/reanahub/reana-job-controller/releases>`_
- known issues: `<https://github.com/reanahub/reana-job-controller/issues>`_
- documentation: `<https://reana-job-controller.readthedocs.io/>`_

reana-message-broker
~~~~~~~~~~~~~~~~~~~~

REANA component for messaging needs.

- source code: `<https://github.com/reanahub/reana-message-broker>`_
- release notes: `<https://github.com/reanahub/reana-message-broker/releases>`_
- known issues: `<https://github.com/reanahub/reana-message-broker/issues>`_
- documentation: `<https://reana-message-broker.readthedocs.io/>`_

reana-server
~~~~~~~~~~~~

REANA component providing API server replying to client queries.

- source code: `<https://github.com/reanahub/reana-server>`_
- release notes: `<https://github.com/reanahub/reana-server/releases>`_
- known issues: `<https://github.com/reanahub/reana-server/issues>`_
- documentation: `<https://reana-server.readthedocs.io/>`_

reana-workflow-controller
~~~~~~~~~~~~~~~~~~~~~~~~~

REANA component for running and managing workflows.

- source code: `<https://github.com/reanahub/reana-workflow-controller>`_
- release notes: `<https://github.com/reanahub/reana-workflow-controller/releases>`_
- known issues: `<https://github.com/reanahub/reana-workflow-controller/issues>`_
- documentation: `<https://reana-workflow-controller.readthedocs.io/>`_

reana-workflow-engine-cwl
~~~~~~~~~~~~~~~~~~~~~~~~~

REANA component for running CWL types of workflows.

- source code: `<https://github.com/reanahub/reana-workflow-engine-cwl>`_
- release notes: `<https://github.com/reanahub/reana-workflow-engine-cwl/releases>`_
- known issues: `<https://github.com/reanahub/reana-workflow-engine-cwl/issues>`_
- documentation: `<https://reana-workflow-engine-cwl.readthedocs.io/>`_

reana-workflow-engine-yadage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

REANA component for running Yadage types of workflows.

- source code: `<https://github.com/reanahub/reana-workflow-engine-yadage>`_
- release notes: `<https://github.com/reanahub/reana-workflow-engine-yadage/releases>`_
- known issues: `<https://github.com/reanahub/reana-workflow-engine-yadage/issues>`_
- documentation: `<https://reana-workflow-engine-yadage.readthedocs.io/>`_

reana-workflow-monitor
~~~~~~~~~~~~~~~~~~~~~~

REANA component permitting to monitor running workflows.

- source code: `<https://github.com/reanahub/reana-workflow-monitor>`_
- release notes: `<https://github.com/reanahub/reana-workflow-monitor/releases>`_
- known issues: `<https://github.com/reanahub/reana-workflow-monitor/issues>`_
- documentation: `<https://reana-workflow-monitor.readthedocs.io/>`_

Deployment
----------

Local deployment using Minikube
-------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

REANA cloud uses `Kubernetes <https://kubernetes.io/>`_ container orchestration
system. The best way to try it out locally is to set up `Minikube
<https://kubernetes.io/docs/getting-started-guides/minikube/>`_ (minikube version 0.23.0 is
known to work the best).
<https://kubernetes.io/docs/getting-started-guides/minikube/>`_ (minikube
version 0.23.0 is known to work the best).

The minikube can be started as follows:

Expand All @@ -25,17 +147,20 @@ script. The typical usage scenario goes as follows:

.. code-block:: console
$ # install reana-cluster utility
$ mkvirtualenv reana-cluster
$ pip install reana-cluster
$ # deploy new cluster and check progress
$ reana-cluster init
$ # wait several minutes...
$ reana-cluster status
$ # set environment variables for reana-client
$ eval $(reana-cluster env)
For more information, please see `reana-cluster documentation
<https://reana-cluster.readthedocs.io/>`_.
For more information, please see `REANA-Cluster's Getting started guide
<http://reana-cluster.readthedocs.io/en/latest/gettingstarted.html>`_.

Next steps
----------

For more information, please see `reana-cluster documentation
For more information, you can explore `REANA-Cluster documentation
<https://reana-cluster.readthedocs.io/>`_.
27 changes: 0 additions & 27 deletions docs/architecture.rst

This file was deleted.

99 changes: 0 additions & 99 deletions docs/components.rst

This file was deleted.

0 comments on commit 080eabb

Please sign in to comment.