Skip to content

Commit

Permalink
Update CQueue demo
Browse files Browse the repository at this point in the history
  • Loading branch information
maystery committed Jan 20, 2021
1 parent 09114f9 commit 70a8ad6
Show file tree
Hide file tree
Showing 8 changed files with 32 additions and 19 deletions.
Binary file removed sphinx/source/images/tutorials/cqueue_cluster.jpg
Binary file not shown.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion sphinx/source/tutorial-bigdata-ai.rst
Original file line number Diff line number Diff line change
Expand Up @@ -778,6 +778,7 @@ For more information on how to use the JupyterLab web-based user interface, visi

If you want Occopus to monitor (health_check) your initiated virtual machine and it is to be deployed in a different network, make sure you assign public (floating) IP to the node.


#. Open the file ``nodes/infra-jupyterlab.yaml`` and edit the variables section labelled by ``variables``. The default username is "jovyan" and the default password is "lpds". Change the value of ``pwd_jupyterlab`` to a safe password!

.. important::
Expand Down Expand Up @@ -823,7 +824,7 @@ For more information on how to use the JupyterLab web-based user interface, visi
14032858-d628-40a2-b611-71381bd463fa
#. You can start using the freshly installed JupyterLab using your web browster at the following URL:
#. You can start using JupyterLab using your web browster at the following URL:

- JupyterLab: ``http://<JupyterLabIP>:8888``

Expand Down
6 changes: 4 additions & 2 deletions sphinx/source/tutorial-building-clusters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@ CQueue stands for "Container Queue". Since Docker does not provide pull model fo

Figure 1 shows, the overall architecture of a CQueue cluster. The CQueue cluster contains one Master node (VM1) and any number of Worker nodes (VM2). Worker nodes can be manually scaled up and down with Occopus. The Master node implements a queue (see "Q" box within VM1), where each item (called task in CQueue) represents the specification of a container execution (image, command, arguments, etc.). The Worker nodes (VM2) fetch the tasks one after the other and execute the container specified by the task (see "A" box within VM2). In each task submission a new Docker container will be launched within at CQueue Worker.

.. figure:: images/tutorials/cqueue_cluster.jpg
.. figure:: images/tutorials/cqueue_cluster.png
:align: center

Figure 1. CQueue cluster architecture
Expand Down Expand Up @@ -559,7 +559,9 @@ The following steps are suggested to be performed:
- you may follow the help on :ref:`collecting the values of the attributes for the plugin <user-doc-collecting-resources>`
- you may find a resource template for the plugin in the :ref:`resource plugin tutorials <tutorial-resource-plugins>`

The downloadable package for this example contains a resource template for the CloudBroker plugin.
.. note::

In this tutorial, we will use nova cloud resources (based on our nova tutorials in the basic tutorial section). However, feel free to use any Occopus-compatible cloud resource for the nodes, but we suggest to instantiate all nodes in the same cloud.

#. Components in the infrastructure connect to each other, therefore several port ranges must be opened for the VMs executing the components. Clouds implement port opening various way (e.g. security groups for OpenStack, etc). Make sure you implement port opening in your cloud for the following port ranges:

Expand Down
Binary file modified tutorials/cqueue-cluster.tar.gz
Binary file not shown.
38 changes: 24 additions & 14 deletions tutorials/cqueue-cluster/nodes/node_definitions.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
'node_def:cqueue_master_node':
-
resource:
type: cloudbroker
endpoint: replace_with_endpoint_of_cloudbroker_interface
description:
deployment_id: replace_with_deployment_id
instance_type_id: replace_with_instance_type_id
key_pair_id: replace_with_keypair_id
opened_port: replace_with_list_of_ports_separated_with_comma
type: nova
endpoint: replace_with_endpoint_of_nova_interface_of_your_cloud
project_id: replace_with_projectid_to_use
user_domain_name: Default
image_id: replace_with_id_of_your_image_on_your_target_cloud
network_id: replace_with_id_of_network_on_your_target_cloud
flavor_name: replace_with_id_of_the_flavor_on_your_target_cloud
key_name: replace_with_name_of_keypair_or_remove
security_groups:
-
replace_with_security_group_to_add_or_remove_section
floating_ip: add_yes_if_you_need_floating_ip_or_remove
floating_ip_pool: replace_with_name_of_floating_ip_pool_or_remove
contextualisation:
type: cloudinit
context_template: !yaml_import
Expand All @@ -21,13 +27,17 @@
'node_def:cqueue_worker_node':
-
resource:
type: cloudbroker
endpoint: replace_with_endpoint_of_cloudbroker_interface
description:
deployment_id: replace_with_deployment_id
instance_type_id: replace_with_instance_type_id
key_pair_id: replace_with_keypair_id
opened_port: replace_with_list_of_ports_separated_with_comma
type: nova
endpoint: replace_with_endpoint_of_nova_interface_of_your_cloud
project_id: replace_with_projectid_to_use
user_domain_name: Default
image_id: replace_with_id_of_your_image_on_your_target_cloud
network_id: replace_with_id_of_network_on_your_target_cloud
flavor_name: replace_with_id_of_the_flavor_on_your_target_cloud
key_name: replace_with_name_of_keypair_or_remove
security_groups:
-
replace_with_security_group_to_add_or_remove_section
contextualisation:
type: cloudinit
context_template: !yaml_import
Expand Down
Binary file modified tutorials/docker-swarm.tar.gz
Binary file not shown.
4 changes: 2 additions & 2 deletions tutorials/docker-swarm/nodes/node_definitions.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
'node_def:dockerswarm_master_node':
-
-
resource:
type: cloudsigma
endpoint: replace_with_endpoint_of_cloudsigma_interface_of_your_cloud
Expand All @@ -26,7 +26,7 @@
- 2375

'node_def:dockerswarm_worker_node':
-
-
resource:
type: cloudsigma
endpoint: replace_with_endpoint_of_cloudsigma_interface_of_your_cloud
Expand Down

0 comments on commit 70a8ad6

Please sign in to comment.