SymbIoteCloud from docker

Vasilis Glykantzis edited this page Feb 14, 2019 · 19 revisions

Cloud Deployment

This document explains how to run the symbIoTe cloud for L1/L2 integration depending on your needs. There are two ways to run docker:

  1. directly (on linux)
  2. docker-machine (previously boot2docker) - in virtual machine

This influences how to expose port in virtual machine to local machine. So, there are 2 ways to run ngrok and 2 ways to expose port in production environment.

1. Preparation steps

1.1. Register user and configure your platform.

When getting the platform configuration you have to enter the following options:

  • Compliance Level: L1 or L2 (depending on the compliance level desired)
  • Deployment Type: Docker
1.2. Install prerequisites:
  • docker (18.03.x),
  • docker-compose (1.21.x),
  • docker-machine (0.14.x),
  • bash,
  • curl,
  • wget
1.3. Create folder for your deployment

IMPORTANT NOTE

Starting from release 3.1.0, there are 2 extra steps needed as noted below.


  • mkdir symbiote-cloud
  • change to that directory (e.g. cd symbiote-cloud)
  • unzip the configuration files to the directory from the .zip downloaded earlier
  • enter the CloudConfigProperties folder, make any changes (if) you want in the properties (e.g. rabbitMQ credentials) and commit the changes (Starting from release 3.1.0):
cd CloudConfigProperties
git commit -am "Platform configuration"
cd ..
  • create a docker volume to hold the CloudConfigProperties. Replace the {docker stack name} (e.g. symbiote-app) in the command below with your selected docker stack name (Starting from release 3.1.0):
docker container run --rm -v $PWD/CloudConfigProperties:/source -v {docker stack name}_symbiote-vol-config:/home/CloudConfigProperties -w /source alpine cp -r . /home/CloudConfigProperties/
2. Configuring and starting components depending on your preferred deployment

Here is table with instructions based on preferred deployment:

Environment docker directly (on linux) docker-machine
production environment section 2.1 section 2.2
hackathon environment (ngrok) section 2.3 section 2.4

Important Note:

The compose files have been configured to block incoming external traffic to components other than the nginx when docker compose is used. However, this is not possible when using docker swarm. Consequently, the Platform Owners need to block the incoming external traffic to components by using other means e.g. iptables in all the swarm nodes.

2.1. Production, without docker-machine (boot2docker)
  1. Obtain certificate as described in https://github.com/symbiote-h2020/SymbioteCloud/wiki/2.1-Configuration-of-NGINX#2111-obtaining-the-ssl-certificate

    • Create dir nginx-certificates
      $ mkdir nginx-certificates

    • Copy certificates to nginx-certificates
      $ sudo cp /etc/letsencrypt/live/{your domain}/fullchain.pem nginx-certificates/
      $ sudo cp /etc/letsencrypt/live/{your domain}/privkey.pem nginx-certificates/

  2. Run docker swarm init if the node is not a swarm manager. We use the swarm mode so that secrets are encrypted during transit and at rest. Docker secrets are only available to swarm services and not to standalone containers.

  3. Run:

    • docker stack deploy -c docker-compose-swarm-L1.yml -c docker-compose-prod-swarm-L1.yml -c docker-compose-swarm-custom.yml symbiote-app to deploy the application in L1 compliance or
    • docker stack deploy -c docker-compose-swarm-L2.yml -c docker-compose-prod-swarm-L2.yml -c docker-compose-swarm-custom.yml symbiote-app for L2 compliance. You can change the name of the service stack (symbiote-app) to be used.
  4. You can run:

    • docker stack ls to list the stack and check the number of services used.
    • docker image ls to check that all images have been created. It may take a while to pull all the images from DockerHub for the first time.
    • docker service ls to list the services and check their status. Wait until the actual number of tasks (replicas) for each service is not 0.
    • docker logs <container_name> -f to get access to and follow the logs of a service. A component is ready when a message similar to 'Started <component_name> in 105.045 seconds (JVM running for 112.933)' appears in the logs of the container.
    • sudo service docker restart to restart the docker service if needed and repeat the steps.
  5. Run docker stack rm symbiote-app to stop the application and remove the service stack. Services, networks, and secrets associated with the stack will be removed.

  6. You can run docker swarm leave --force to leave the swarm.

2.2. Production, with docker-machine (boot2docker)
  1. Obtain certificate as described previously
  2. The following command maps
    • port 443 in default docker-machine (localhost:443) to
    • local computer at port 443 on all interfaces (0.0.0.0:443)
      docker-machine ssh default -L 0.0.0.0:443:localhost:443
  3. Run steps 2-6 as described previously in Section 2.1.
2.3. Hackathon, without docker-machine (boot2docker)

Before starting with hackathon environment you need to install ngrok (https://github.com/symbiote-h2020/SymbioteCloud/wiki/2.1-Configuration-of-NGINX#2121-ngrok).

  1. Run docker swarm init if the node is not a swarm manager. We use the swarm mode so that secrets are encrypted during transit and at rest. Docker secrets are only available to swarm services and not to standalone containers.
  2. Run
    • docker stack deploy -c docker-compose-swarm-L1.yml -c docker-compose-ngrok-swarm-L1.yml -c docker-compose-swarm-custom.yml symbiote-app to deploy the application in L1 compliance or
    • docker stack deploy -c docker-compose-swarm-L2.yml -c docker-compose-ngrok-swarm-L2.yml -c docker-compose-swarm-custom.yml symbiote-app for L2 compliance.
  3. You can run:
    • docker stack ls to list the stack and check the number of services used.
    • docker image ls to check that all images have been created. It may take a while to pull all the images from DockerHub for the first time.
    • docker service ls to list the services and check their status. Wait until the actual number of tasks (replicas) for each service is not 0.
    • docker logs <container_name> -f to get access to and follow the logs of a service. A component is ready when a message similar to 'Started <component_name> in 105.045 seconds (JVM running for 112.933)' appears in the logs of the container.
    • sudo service docker restart to restart the docker service if needed and repeat the steps.
  4. Run docker stack rm symbiote-app to stop the application and remove the service stack. Services, networks, and secrets associated with the stack will be removed.
2.4. Hackathon, with docker-machine (boot2docker)

Before starting with hackathon environment you need to install ngrok (https://github.com/symbiote-h2020/SymbioteCloud/wiki/2.1-Configuration-of-NGINX#2121-ngrok).

  1. Run docker swarm init if the node is not a swarm manager. We use the swarm mode so that secrets are encrypted during transit and at rest. Docker secrets are only available to swarm services and not to standalone containers.
  2. Run docker-machine ssh default -L 0.0.0.0:8102:localhost:8102 for mapping port in virtual machine to this computer.
  3. Run
    • docker stack deploy -c docker-compose-swarm-L1.yml -c docker-compose-ngrok-swarm-L1.yml -c docker-compose-swarm-custom.yml symbiote-app to deploy the application in L1 compliance or
    • docker stack deploy -c docker-compose-swarm-L2.yml -c docker-compose-ngrok-swarm-L2.yml -c docker-compose-swarm-custom.yml symbiote-app for L2 compliance.
  4. You can run:
    • docker stack ls to list the stack and check the number of services used.
    • docker image ls to check that all images have been created. It may take a while to pull all the images from DockerHub for the first time.
    • docker service ls to list the services and check their status. Wait until the actual number of tasks (replicas) for each service is not 0.
    • docker logs <container_name> -f to get access to and follow the logs of a service. A component is ready when a message similar to 'Started <component_name> in 105.045 seconds (JVM running for 112.933)' appears in the logs of the container.
    • sudo service docker restart to restart the docker service if needed and repeat the steps.
  5. Run docker stack rm symbiote-app to stop the application and remove the service stack. Services, networks, and secrets associated with the stack will be removed.
2.5. Manage resources

3. Test integrated resource

3.1. Security
3.2. Search for resources
3.3. Obtaining resource access URL
3.4. Accessing the resource and actuating and invoking service for default (dummy) resources

4. Creating RAP plugin

4.1. Customizing internal RAP plugin
4.2. Using RAP plugin starter
4.3. Creating RAP plugin in other languages
4.4. Development environment

During development and testing of RAP plugin it is convenient to run it in IDE on local machine. In that case follow steps in this document.

4.5. Deploying custom plugin

When you are finished with your custom RAP plugin and you have tested it (explained in section 4.4). Then you need to deploy it in docker environment. You need to do following:

  1. Create docker image from builded jar
  2. Upload image to machine where is running docker implementation
  • you can export image to file, transfer file to machine with docker implementation and import image or
  • you can upload image to docker hub and use image from there
  1. change docker-compose-swarm-custom.yml to use your image
  2. restart docker swarm stack

5. Resource Description Examples

5.1. JSON Description Examples
5.2. RDF Description Examples

Getting Started
Migration to 3.0.0
Migration to Docker

  1. Preparation steps
    1.1. Register user and configure platform in symbIoTe Core
    1.2. Installation of required tools for symbIoTe platform components
    1.3. Downloading jars
    1.4. Downloading sources
  2. Configuring and starting components
    2.1. Configuration of NGINX
    2.2. Starting third party tools that are prerequisite for symbIoTe
    2.3. Starting (generic) symbIoTe Cloud components
    2.4. Configuration of cloud components
    2.4.1. Starting symbIoTe Cloud components
    2.5. Setting up the Platform Authentication and Authorization Manager (PAAM)
    2.6. Starting Registration Handler and resource management
    2.7. Set up of Resource Access Proxy
    2.8. Manage resources
    2.9. Set up of the Monitoring component
    2.10. Other configuration topics
  3. Test integrated resource
    3.1. Security
    3.2. Search for resources
    3.3. Obtaining resource access URL
    3.4. Accessing the resource and actuating and invoking service for default (dummy) resources
  4. Creating RAP plugin
    4.1. Customizing internal RAP plugin
    4.2. Using RAP plugin starter
    4.3. Creating RAP plugin in other languages
  5. Resource Description Examples
    5.1. JSON Description Examples
    5.2. RDF Description Examples
  6. Preparation for L2 compliance
  7. Configuring and starting components for L2
    7.1. Starting Federation Manager
    7.2. Starting Subscription Manager
    7.3. Starting Platform Registry
    7.4. Starting Trust Manager
    7.5. Starting Bartering And Trading
    7.6. Starting SLA Manager
    7.7. Create a federation
    7.8. Manage resources in L2
    7.9. Register Subscription
  8. Test Integrated L2 Resources
  9. Developing symbIoTe enabled apps
Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.