#24 FIWARE further doc and setup

docs/fiware.rst

@@ -1,11 +1,10 @@
-.. _rio:
+.. _fiwareeval:
 
-====================
-FI-WARE - Evaluation
-====================
+===================
+FIWARE - Evaluation
+===================
 
-Starting in october 2015 the SOSPilot platform was
-extended to use FI-WARE
+Starting in october 2015 the SOSPilot platform was extended to use and deploy FIWARE.
 
 The Plan
 ========
@@ -17,6 +16,33 @@ The Plan
 5. get temperatures from Orion CB
 6. show in Wirecloud Mashup
 
+The architecture used for this setup is depicted below. This
+architecture emerged dynamically as described below.
+
+This setup is similar
+as in this `FIWARE presentation <http://www.slideshare.net/FI-WARE/fiware-iotidasintroul20v2>`_.
+
+.. figure:: _static/fiware/sospilot-fiware-arch1.jpg
+   :align: center
+
+   *Architecture for FIWARE within SOSPilot platform*
+
+From bottom to top the setup is as follows:
+
+* Sensors or manual input use the UltraLight 2.0 (UL2.0) protocol for managing services and devices and sending observations
+* A client library is used to facilitate using the UL2.0 protocol
+* The client may reside anywhere on the Internet
+* the server ``sensors.geonovum.nl`` hosts the FIWARE components ``Orion Context Broker`` and the ``IoTAgentCpp``
+* all persistence for these components is done in ``MongoDB``
+* the ``IoTAgentCpp`` translates requests from the UL2.0 clients to Orion CB NGSI requests
+* ``WireCloud`` (WC), the Mashup environment runs in the FIWARE Lab server at lab.fiware.org
+* WC communicates to (any) OCB using the NGSI10 protocol
+* within WC mashups are produced in order to view and interact with the sensor data
+
+NGSI10 is a specification from the Open Mobile Alliance (OMA).
+The FI-WARE version of the OMA NGSI 10 interface is a
+`RESTful API via HTTP <https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/FI-WARE_NGSI-10_Open_RESTful_API_Specification>`.
+
 FIWARE Docs
 ===========
 
@@ -734,9 +760,14 @@ Within the ``mongodb`` Docker container we can inspect the persisted data in the
 Display Values with WireCloud
 -----------------------------
 
-WireCloud http://conwet.fi.upm.es/wirecloud/ is a MashUp framework within FIWARE with instance at FIWARE Lab: https://mashup.lab.fiware.org
+WireCloud http://conwet.fi.upm.es/wirecloud/ is a Mashup framework within FIWARE with instance at FIWARE Lab: https://mashup.lab.fiware.org
 
-Here we can create Widgets to get data from the Orion CB. Trying simple the NGSI Browser, but did not succeed (help mail
+Here we can create Widgets to get data from the Orion CB, so indirectly observations sent to the IoTAgent from our devices.
+
+First Steps
+~~~~~~~~~~~
+
+Trying simple the NGSI Browser, but did not succeed (help mail
 sent to ``fiware-lab-help@lists.fiware.org`` :  ::
 
 	Trying to connect to my OCB  which has entities created via IDAS. Both are of latest Docker version.
@@ -762,6 +793,50 @@ sent to ``fiware-lab-help@lists.fiware.org`` :  ::
 
 This problem was posted at StackOverflow : http://stackoverflow.com/questions/33452246/fiware-wirecloud-fiware-service-http-header-needed-for-orion-cb
 
+A quick local solution is to manually add the HTTP header using the browser's Developer Tools to a WireCloud Widget JavaScript (usually ``main.js``)
+where the HTTP request to the Orion NGSI API is created. The WC NGSI connector supports adding extra HTTP headers as per the documentation:
+ https://wirecloud.readthedocs.org/en/latest/development/ngsi_api/#ngsiconnection. See for example here below where the Chrome developer tools is used
+to modify the NGSIConnection :
+
+.. figure:: _static/fiware/wc-fwheader-hack.jpg
+   :align: center
+
+   *Quick Fix: modify the HTTP header locally in browser*
+
+Another possibility is to download the widget, modify its config.xml and main.js to support configuring the FIWARE-Service header.
+This worked as well, but the real fixes should be done within the component source code. The issue affects all WC components
+(Operators, Widgets) that interact with Orion NSGI. The following issues have been opened:
+
+* https://github.com/wirecloud-fiware/ngsi-browser-widget/issues/1  (fixed)
+* https://github.com/wirecloud-fiware/ngsi-source-operator/issues/3
+* https://github.com/wirecloud-fiware/ngsi-updater-widget/issues/1
+* https://github.com/wirecloud-fiware/ngsi-type-browse-widget/issues/1
+
+As the NGSI Browser Widget was fixed and a new version was available, a first test could be performed.
+
+Temperature in POI on Map
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Using the `NGSI Browser Widget <https://github.com/wirecloud-fiware/ngsi-browser-widget>` (fixed v1.0.1) with an
+NSGI Entity to POI Converter
+connected to a Map Widget
+the temperature could be made visible on a map. The result of the mashup is below.
+
+.. figure:: _static/fiware/wc-geonovum-temp1.jpg
+   :align: center
+
+   *First WireCloud Mashup: show Temperature from sensor as POI*
+
+The wiring for these components was as depicted below.
+
+
+.. figure:: _static/fiware/wc-geonovum-temp1-wiring.jpg
+   :align: center
+
+   *First WireCloud Mashup: wiring view in editor*
+
+
+
 Installing FIWARE - from Source
 ===============================
 
@@ -780,8 +855,8 @@ but later found: https://github.com/telefonicaid/fiware-orion/blob/develop/scrip
 
 Install build deps. ::
 
-    apt-get  install cmake scons libmicrohttpd-dev libboost-all-dev
-    # what about libcurl-dev gcc-c++ ???
+	apt-get  install cmake scons libmicrohttpd-dev libboost-all-dev
+	# what about libcurl-dev gcc-c++ ?
 	apt-get -y install make cmake build-essential scons git libmicrohttpd-dev libboost-dev libboost-thread-dev libboost-filesystem-dev libboost-program-options-dev  libcurl4-gnutls-dev clang libcunit1-dev mongodb-server python python-flask python-jinja2 curl libxml2 netcat-openbsd mongodb valgrind libxslt1.1 libssl-dev libcrypto++-dev
 
 	Setting up libboost1.54-dev (1.54.0-4ubuntu3.1) ...

src/fiware/docker/Dockerfile

@@ -0,0 +1,77 @@
+FROM centos:6.6
+
+MAINTAINER David 
+
+ENV GIT_URL_IOTACPP https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus.git
+ENV GIT_REV_IOTACPP develop
+ENV GIT_PATH_IOTACPP /opt/iotacpp
+ENV IOTACPP_BUILD_TYPE Release
+ENV IOTACPP_USER iotagent
+
+RUN yum -y install tar gzip unzip file \
+    	   	   cpp gcc gcc-c++ \
+           automake autoconf libtool \
+           git scons cmake \
+           openssl-devel bzip2-devel zlib-devel \
+           rpm-build && \
+    git clone ${GIT_URL_IOTACPP} ${GIT_PATH_IOTACPP} && \
+    cd ${GIT_PATH_IOTACPP} && \
+    sed -i CMakeLists.txt \
+    	-e 's|^add_test|#add_test|g' \
+    -e 's|^add_subdirectory(tests)|#add_subdirectory(tests)|g' \
+    -e 's|^enable_testing|#enable_testing|g' \
+        -e 's|git@github.com:mongodb/mongo-cxx-driver.git|https://github.com/mongodb/mongo-cxx-driver|g' && \
+    source tools/get_version_string.sh && \
+    mkdir -p  build/${IOTACPP_BUILD_TYPE} && \
+    cd build/${IOTACPP_BUILD_TYPE} && \
+    cmake -DGIT_VERSION=`get_rpm_version_string | cut -d ' ' -f 1` \
+    	  -DGIT_COMMIT=`get_rpm_version_string | cut -d ' ' -f 2` \
+      -DMQTT=ON \
+      -DCMAKE_BUILD_TYPE=${IOTACPP_BUILD_TYPE} ../../ && \
+    make install && \
+    make package && \
+    cd pack/Linux/RPM 
+
+RUN yum clean all
+
+RUN yum -y remove autoconf automake bzip2-devel \
+          cmake cpp file gcc gcc-c++ git \
+         libtool openssl-devel rpm-build \
+         scons zlib-devel bzip2 cloog-ppl \
+         elfutils elfutils-libs gdb glibc-devel \
+         glibc-headers kernel-headers \
+         keyutils-libs-devel krb5-devel \
+         libarchive libcom_err-devel libgomp \
+         libselinux-devel libsepol-devel \
+         libstdc++-devel m4 mpfr patch \
+         perl-Error perl-Git perl-Module-Pluggable \
+         perl-Pod-Escapes perl-Pod-Simple \
+         perl-libs perl-version  \
+         ppl rsync
+
+
+ RUN cd ${GIT_PATH_IOTACPP} && \
+    cd build/${IOTACPP_BUILD_TYPE} && \
+    cd pack/Linux/RPM && \
+    curl -kOL http://download.opensuse.org/repositories/home:/oojah:/mqtt/CentOS_CentOS-6/home:oojah:mqtt.repo && \
+    mv home:oojah:mqtt.repo /etc/yum.repos.d/ && \
+    yum localinstall -y iot-agent-*.rpm && \
+    yum clean all && \
+    chkconfig iotagent on
+
+RUN chkconfig mosquitto on
+
+COPY config.json /etc/iot/
+RUN chown ${IOTACPP_USER}:${IOTACPP_USER} /etc/iot/config.json
+
+COPY docker-entrypoint.sh /
+RUN chmod +x /docker-entrypoint.sh
+ENTRYPOINT ["/docker-entrypoint.sh"]
+
+EXPOSE 8080
+EXPOSE 8081
+EXPOSE 1883
+
+
+
+

src/fiware/docker/config.json

@@ -0,0 +1,34 @@
+{
+    "ngsi_url": {
+        "cbroker": "http://ORION_HOSTNAME:ORION_PORT",
+        "updateContext": "/NGSI10/updateContext",
+        "registerContext": "/NGSI9/registerContext",
+        "queryContext": "/NGSI10/queryContext"
+    },
+    "timeout": 10,
+    "http_proxy": "PUBLIC_PROXY_PORT",
+    "public_ip": "PUBLIC_IP_PORT",
+    "dir_log": "/var/log/iot/",
+    "timezones": "/etc/iot/date_time_zonespec.csv",
+    "storage": {
+        "host": "MONGODB_HOSTNAME",
+        "type": "mongodb",
+        "port": "MONGODB_PORT",
+        "dbname": "iot"
+    },
+   "resources": [
+        {
+            "resource": "/iot/d",
+            "options": {
+                "FileName": "UL20Service"
+            }
+        },
+        {
+            "resource": "/iot/mqtt",
+            "options": {
+                "ConfigFile" : "/etc/iot/MqttService.xml",
+                "FileName": "MqttService"
+            }
+         }
+   ]
+}

src/fiware/docker/docker-entrypoint.sh

@@ -0,0 +1,43 @@
+#!/bin/sh
+
+if [[ ${MONGODB_PORT} =~ ^tcp://(.*):(.*)+$ ]] ; then
+    export MONGODB_PORT=${BASH_REMATCH[2]}
+    export MONGODB_HOSTNAME=${BASH_REMATCH[1]}
+
+fi
+
+if [[ ${ORION_PORT} =~ ^tcp://(.*):(.*)+$ ]] ; then
+    export ORION_PORT=${BASH_REMATCH[2]}
+    export ORION_HOSTNAME=${BASH_REMATCH[1]}
+fi
+
+echo $PUBLIC_PROXY_PORT
+echo $PUBLIC_IP_PORT
+echo $MONGODB_HOSTNAME
+echo $MONGODB_PORT
+echo $ORION_HOSTNAME
+echo $ORION_PORT
+
+sed -i /etc/init.d/mosquitto \
+    -e "s|etc/mosquitto/mosquitto.conf|etc/iot/mosquitto.conf|g"
+
+sed -i /etc/iot/mosquitto.conf \
+    -e "s|user root|user iotagent|g"
+
+sed -i /etc/iot/config.json \
+	-e "s|ORION_HOSTNAME|${ORION_HOSTNAME}|g" \
+	-e "s|ORION_PORT|${ORION_PORT}|g" \
+	-e "s|MONGODB_HOSTNAME|${MONGODB_HOSTNAME}|g" \
+    -e "s|MONGODB_PORT|${MONGODB_PORT}|g" \
+    -e "s|PUBLIC_PROXY_PORT|${PUBLIC_PROXY_PORT}|g" \
+    -e "s|PUBLIC_IP_PORT|${PUBLIC_IP_PORT}|g"
+
+#cat /etc/iot/config.json
+
+exec /sbin/init
+
+
+
+
+#/bin/bash
+

src/fiware/docker/readme.md

@@ -0,0 +1,17 @@
+#Introduction
+Dockerfile to create a container with IoTAgent (MQTT and UL20). Using Docker-compose (iota.yaml fie) you could run multiple containers with IoTAgent, Orion Context-Broker and MongoDB Database all interconnected.
+
+#Use
+
+Run this command:
+```
+docker-compose -f iota.yaml up -d iotacpp
+```
+This command will build the IoTAgent docker image and it will run three different containers with Orion ContextBroker, MongoDB and IoTAgent all of them linked. The IoTAgent container will expose the 8080 (HTTP) port and 1883 for MQTT protocol. 
+
+#Build just IoTAgent docker image
+```
+docker build .
+```
+
+This will create an image of the IoTAgent, but in order to properly start it, you need to run other containers for ContextBroker and MongoDB, hence the convenience of having a docker-compose file that will start up all three elements together. 

src/fiware/docker/restart.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+docker-compose -f iota.yaml stop
+docker rm -v $(docker ps -a -q -f status=exited)
+docker-compose -f iota.yaml up -d
+docker ps
+