From 8a4c452bb3cc1f3e5c0c9f3274cb1cc4f17c1562 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pablo=20Rodr=C3=ADguez?= Date: Tue, 11 Aug 2015 12:32:22 +0200 Subject: [PATCH] Amend end-to-end and acceptance testing --- README.rst | 31 ++--- doc/manuals/admin/README.rst | 41 ++++--- ngsi_adapter/src/test/acceptance/README.rst | 118 +++++++++++++------- 3 files changed, 115 insertions(+), 75 deletions(-) diff --git a/README.rst b/README.rst index e787d6a..7891dd4 100644 --- a/README.rst +++ b/README.rst @@ -276,25 +276,8 @@ Testing End-to-end tests ---------------- -Use the commands of the monitoring framework being used (for example, Nagios) -to reschedule some probe execution and force the generation of new monitoring -data. - -Check NGSI Adapter logs for incoming requests with raw data, and check the -outgoing Context Broker requests as NGSI updateContext() operations: - -:: - - $ cat /var/log/ngsi_adapter/ngsi_adapter.log - time=... | lvl=INFO | trans=ci2627bx00000b42g8m2pxw3z | op=POST | msg=Request on resource /check_xxx with params id=xxx&type=xxx - time=... | lvl=INFO | trans=ci2627bx00000b42g8m2pxw3z | op=POST | msg=Response status 200 OK - time=... | lvl=INFO | trans=ci2627bx00000b42g8m2pxw3z | op=UpdateContext | msg=Request to ContextBroker at http://host:1026/... - - -Finally, query Context Broker API to check whether entity attributes have been -updated according to the new monitoring data (see details here__) - -__ `FIWARE Orion Context Broker`_ +Please refer to the `Installation and administration guide +`_ for details. Unit tests @@ -318,6 +301,16 @@ information about how to prepare the environment to run the unit tests. +Acceptance tests +---------------- + +In the following documents you will find a business readable description of the +features provided by the components of the Monitoring GE, as well as automated +tests for them: + +- `NGSI Adapter acceptance tests `_ + + Advanced topics =============== diff --git a/doc/manuals/admin/README.rst b/doc/manuals/admin/README.rst index 9b60866..4b4ab9b 100644 --- a/doc/manuals/admin/README.rst +++ b/doc/manuals/admin/README.rst @@ -117,8 +117,8 @@ distributed filesystem storage (usually HDFS from Hadoop_). Historically the here__), although from March 2014 this component is deprecated and a brand new **Cygnus** implementation (installation details here__) is available. -__ `ngsi2cosmos`_ -__ `Cygnus`_ +__ ngsi2cosmos_ +__ Cygnus_ Running the monitoring components @@ -183,20 +183,24 @@ to unit tests, integration tests and user validation. End to end testing ------------------ -- Use the commands of the monitoring framework being used (for example, Nagios) - to reschedule some probe execution and force the generation of new monitoring - data. +Use the commands of the monitoring framework being used (for example, Nagios) +to reschedule some probe execution and force the generation of new monitoring +data: -- Check NGSI Adapter logs for incoming requests with raw data, and check the - outgoing Context Broker requests as NGSI updateContext() operations: +- Check the logs of the framework (i.e. ``/var/log/nagios/nagios.log``) for + a new probe execution detected by the *collector*:: + + $ cat /var/log/nagios/nagios.log + [1439283831] lvl=INFO | trans=rdPmJ/uHE62a | comp=fiware-monitoring-ngsi-event-broker | op=NGSIAdapter | msg=Request sent to http://host:1337/check_xxx?id=xxx&type=host -.. code:: - $ cat /var/log/ngsi_adapter/ngsi_adapter.log - time=... | lvl=INFO | trans=ci2627bx00000b42g8m2pxw3z | op=POST | msg=Request on resource /check_xxx with params id=xxx&type=xxx - time=... | lvl=INFO | trans=ci2627bx00000b42g8m2pxw3z | op=POST | msg=Response status 200 OK - time=... | lvl=INFO | trans=ci2627bx00000b42g8m2pxw3z | op=UpdateContext | msg=Request to ContextBroker at http://host:1026/... +- Check NGSI Adapter logs for incoming requests with raw data, and for the + corresponding updateContext() request to Context Broker:: + $ cat /var/log/ngsi_adapter/ngsi_adapter.log + time=... | lvl=INFO | trans=rdPmJ/uHE62a | op=POST | msg=Request on resource /check_xxx with params id=xxx&type=xxx + time=... | lvl=INFO | trans=rdPmJ/uHE62a | op=POST | msg=Response status 200 OK + time=... | lvl=INFO | trans=rdPmJ/uHE62a | op=UpdateContext | msg=Request to ContextBroker at http://host:1026/... - Finally, query Context Broker API to check whether entity attributes have been updated according to the new monitoring data (see details here__) @@ -211,16 +215,16 @@ A ``node`` process running the "adapter" server should be up and running, e.g.: .. code:: - $ ps -C node -f | grep adapter - fiware   21930     1  0 Mar28 ?        00:06:06 node /opt/fiware/ngsi_adapter/src/adapter + $ ps -C node -f | grep adapter + fiware   21930     1  0 Mar28 ?        00:06:06 node /opt/fiware/ngsi_adapter/src/adapter Alternatively, we can check if service is running, e.g.: .. code:: - $ service ngsi_adapter status - * ngsi_adapter is running + $ service ngsi_adapter status + * ngsi_adapter is running Network interfaces Up & Open @@ -284,9 +288,10 @@ I/O flows Figure at `installation section`__ shows the I/O flows among the different monitoring components: -__ `Installation`_ +__ Installation_ -- Probes send requests to NGSI Adapter with raw monitoring data +- Probes send requests to NGSI Adapter with raw monitoring data, by means + of a custom *collector* component (for example, NGSI Event Broker) - NGSI Adapter sends request to Context Broker in terms of context updates of the monitored resources diff --git a/ngsi_adapter/src/test/acceptance/README.rst b/ngsi_adapter/src/test/acceptance/README.rst index 101106a..601a581 100644 --- a/ngsi_adapter/src/test/acceptance/README.rst +++ b/ngsi_adapter/src/test/acceptance/README.rst @@ -1,19 +1,20 @@ -========================================================== -FIWARE-MONITORING | NGSI-Adapter | Acceptance test project -========================================================== +=============================== + NGSI Adapter acceptance tests +=============================== -This project contains the NGSI-Adapter acceptance tests (component, integration and E2E testing). -All test cases have been defined using Gherkin that it is a Business Readable, Domain Specific Language that lets you -describe software’s behaviour without detailing how that behaviour is implemented. -Gherkin has the purpose of serving documentation of test cases. +This project contains the NGSI Adapter acceptance tests (component, integration +and E2E testing). All test cases have been defined using Gherkin_, that it is a +Business Readable, Domain Specific Language that lets you describe software’s +behaviour without detailing how that behaviour is implemented. Gherkin has the +purpose of serving documentation of test cases. +Test case implementation has been performed using Python_ and Lettuce_. -Test case implementation has been performed using `Python `_ and -`Lettuce `_. Acceptance Project Structure ----------------------------- - :: +============================ + +:: ├───acceptance │ ├───commons @@ -29,34 +30,35 @@ Acceptance Project Structure FIWARE Monitoring Automation Framework ---------------------------------------- +====================================== Features: - Lettuce-Tools support - Settings using json files and Lettuce-Tools utility - Test report using Lettuce-Tools XUnit output -- NGSI-Adapter Client +- NGSI Adapter Client - Logging -- Remote NGSI-Adapter log capturing +- Remote NGSI Adapter log capturing - Test data management using templates (resources) -Acceptance test execution -------------------------- +Acceptance tests execution +========================== Execute the following command in the test project root directory: :: - $> cd ngsi_adapter/src/test/acceptance - $> lettuce_tools -ft send_data_api_resource -ts comp -sd features/ --tags=-skip -en dev + $ cd ngsi_adapter/src/test/acceptance + $ lettuce_tools -ft send_data_api_resource -ts comp -sd features/ --tags=-skip -en dev With this command, you will execute: -- components Test Cases in the 'Development' environment configured in settings/dev-properties.json -- the send_data_api_resource feature -- Skipping all Scenarios with tagged with "skip" +- Components test cases in the "Development" environment configured in file + ``settings/dev-properties.json`` +- The "send_data_api_resource" feature +- Skipping all scenarios tagged with ``"skip"`` **Prerequisites** @@ -64,32 +66,72 @@ With this command, you will execute: - Python 2.7 or newer (2.x) (https://www.python.org/downloads/) - pip (https://pypi.python.org/pypi/pip) - virtualenv (https://pypi.python.org/pypi/virtualenv) -- Monitoring [NGSI-Adapter] (`Download NGSI-Adapter `_) +- NGSI Adapter from FIWARE Monitoring (`download sources`__) + +__ `NGSI Adapter sources`_ **Test case execution using virtualenv** -1. Create a virtual environment somewhere *(virtualenv $WORKON_HOME/venv)* -#. Activate the virtual environment *(source $WORKON_HOME/venv/bin/activate)* -#. Go to *ngsi_adapter/src/test/acceptance* folder in the project -#. Install the requirements for the acceptance tests in the virtual environment *(pip install -r requirements.txt --allow-all-external)* +1. Create a virtual environment somewhere:: + + $ virtualenv $WORKON_HOME/venv + +#. Activate the virtual environment:: + + $ source $WORKON_HOME/venv/bin/activate + +#. Go to the acceptance tests folder in the project:: + + $ cd ngsi_adapter/src/test/acceptance + +#. Install requirements for the acceptance tests in the virtual environment:: + + $ pip install -r requirements.txt --allow-all-external **Test case execution using Vagrant (optional)** -Instead of using virtualenv, you can use the provided Vagrantfile to deploy a local VM using `Vagrant `_, -that will provide all environment configurations for launching test cases. +Instead of using ``virtualenv``, you can use Vagrant_ to deploy a local VM from +the given *Vagrantfile*, providing all environment configurations to launch the +test cases. -1. Download and install Vagrant (https://www.vagrantup.com/downloads.html) -#. Go to *ngsi_adapter/src/test/acceptance* folder in the project -#. Execute *vagrant up* to launch a VM based on Vagrantfile provided. -#. After Vagrant provision, your VM is properly configured to launch acceptance tests. You have to access to the VM using *vagrant ssh* and change to */vagrant* directory that will have mounted your workspace *(test/acceptance)*. +As a prerequisite, first download and install Vagrant +(https://www.vagrantup.com/downloads.html) -If you need more information about how to use Vagrant, you can see -`Vagrant Getting Started `_ +1. Go to the acceptance tests folder in the project:: + + $ cd ngsi_adapter/src/test/acceptance + +#. Launch a VM from the provided *Vagrantfile*:: + + $ vagrant up + +#. After Vagrant provision, your VM is properly configured to launch acceptance + tests. You have to access the VM and change to the Vagrant directory mapping + the *test/acceptance* workspace:: + + $ vagrant ssh + $ cd /vagrant + +For more information about how to use Vagrant, please check `this document`__. + +__ `Vagrant Getting Started`_ **Settings** -Before executing the acceptance tests, you will need configure the properties file. To execute acceptance test on the -experimentation environment, you will have to configured the file *settings/dev-properties*. +Before executing the acceptance tests, you will have configure the properties +file ``settings/dev-properties.json`` and setup the attributes to run the +acceptance tests on the experimentation environment. + +You will also need a valid private key (*private_key_location*) to connect to +NGSI Adapter host to capture remote logs. This way you will be able to execute +scenarios that require the logs capturing for test validations. + + +.. REFERENCES -You will need a valid private key (*private_key_location*) to connect to NGSI-Adapter Host to capture remote logs. -In this way, you will be able to execute Scenarios that require the logs capturing for test validations. +.. _Gherkin: https://github.com/cucumber/cucumber/wiki/Gherkin +.. _Lettuce: http://lettuce.it/ +.. _Python: http://www.python.org/ +.. _Vagrant: https://www.vagrantup.com/ +.. _Vagrant Getting Started: https://docs.vagrantup.com/v2/getting-started/index.html +.. _NGSI Adapter sources: https://github.com/telefonicaid/fiware-monitoring/