Create a configuration for the Integrator UI

The Integrator UI shows all the existing component dashboards in a unified interface. For each dashboard a menu item is shown in the navigation bar at the top.

The BDE CSS style will be applied automatically on the dashboards. The pilot case partner needs to configure the dashboards and the order in which they should appear in the Integrator UI.

The configuration of the Integrator UI is done through a JSON file. There are two ways of adding new services to the Integrator UI, either using iframes or adding them as ember engines.

How to add new services (Iframe)

• Add the new service(s) to docker-compose.yml of the app-bdi-ide project or the container application.

Configure a VIRTUAL_HOST and VIRTUAL_PORT environment variable for each service that needs to be accessible in the Integrator UI. The value of the VIRTUAL_HOST must be the base URI of the service as configured in the Integrator UI configuration afterwards. The value of the VIRTUAL_PORT variable must be the port on which the component's dashboard is available. For example, the Spark Master may have the following configuration:

spark-master:
  image: bde2020/spark-master:2.2.0-hadoop2.7
  container_name: spark-master
  ports:
    - "8080:8080"
    - "7077:7077"
  environment:
      VIRTUAL_HOST: "spark-master.big-data-europe.aksw.org"
      VIRTUAL_PORT: "8080"
      INIT_DAEMON_STEP: "setup_spark"
      constraint: "node==<yourmasternode>"

spark-worker-1:
  image: bde2020/spark-worker:2.2.0-hadoop2.7
  container_name: spark-worker-1
  depends_on:
    - spark-master
  ports:
    - "8081:8081"
  environment:
      VIRTUAL_HOST: "spark-worker1.big-data-europe.aksw.org"
      VIRTUAL_PORT: "8081"
      INIT_DAEMON_STEP: "setup_spark"
      SPARK_MASTER: "spark://spark-master:7077"
      constraint: "node==<yourmasternode>"

spark-worker-2:
  image: bde2020/spark-worker:2.2.0-hadoop2.7
  container_name: spark-worker-2
  depends_on:
    - spark-master
  ports:
    - "8082:8081"
  environment:
      VIRTUAL_HOST: "spark-worker2.big-data-europe.aksw.org"
      VIRTUAL_PORT: "8082"
      INIT_DAEMON_STEP: "setup_spark"
      SPARK_MASTER: "spark://spark-master:7077"
      constraint: "node==<yourmasternode>"

• Add an entry in /etc/hosts to point the url to localhost (or wherever your service is running) (e.g):

127.0.0.1 workflow-builder.big-data-europe.aksw.org
127.0.0.1 swarm-ui.big-data-europe.aksw.org
127.0.0.1 kibana.big-data-europe.aksw.org
(..)
127.0.0.1 spark-master.big-data-europe.aksw.org
127.0.0.1 spark-worker1.big-data-europe.aksw.org
127.0.0.1 spark-worker2.big-data-europe.aksw.org

• Modify the file user-interfaces (in app-bdi-ide it is in the config/integrator-ui/user-interfaces path) to add a link to the new service in the integrator UI.

{
  "data": [
    ...etc .. ,
    {
      "id": 1,
      "type": "user-interfaces",
      "attributes": {
        "engine": false,
        "repo-name": "",
        "label": "Spark Master",
        "base-url": "http://spark-master.big-data-europe.aksw.org/",
        "append-path": ""
      }
    },
    {
      "id": 2,
      "type": "user-interfaces",
      "attributes": {
        "engine": false,
        "repo-name": "",
        "label": "Spark Worker 1",
        "base-url": "http://spark-worker1.big-data-europe.aksw.org/",
        "append-path": ""
      }
    },
    {
      "id": 3,
      "type": "user-interfaces",
      "attributes": {
        "engine": false,
        "repo-name": "",
        "label": "Spark Worker 2",
        "base-url": "http://spark-worker2.big-data-europe.aksw.org/",
        "append-path": ""
      }
    }

  ]
}

How to add new services (ember-engines)

If the new service frontend to be added is an ember-engine instead of a standalone frontend image, the process is a little different. Since the engine is part of the integrator-ui (added as an addon), there is no longer need of adding a new service into the docker-compose.yml file, since it will be using the integrator-ui's nginx server.

Adding a new service involves changing the config/integrator-ui/user-interfaces file to instruct the integrator-ui that that piece is an engine.

{
  "data": [
    ...etc .. ,
    {
      "id": 1,
      "type": "user-interfaces",
      "attributes": {
        "engine": true,
        "repo-name": "docker-spark-hypothetical-engine-repo",
        "label": "Spark Engine",
        "base-url": "",
        "append-path": ""
      }
    }
  ]
}

You will also need to fork the integrator-ui repository and add the proper git url for your ember-engine in the package.json then replace the default integrator-ui by your forked version:

{
  "devDependencies": {
    "ember-pipeline-builder-engine": "https://github.com/big-data-europe/ember-pipeline-builder-engine.git#v1.2.1"
  }
}

Some remarks:

• engine: it is set to true to specify this frontend piece is an engine.
• repo-name: this is a little tricky. This name must match the name you use in the router.js file of your application, for it to create a link that matches with the name of the engine.
• base-url: for engines it must be empty for the engines.

---

Implementing Pilot on BDE Stack

1. List the required images and dependencies
2. Add support for the init daemon
3. Build a pipeline flow
4. Create a configuration for the Integrator UI
5. Construct a docker-compose.yml file