Skip to content

Commit

Permalink
"testando arquivos po"
Browse files Browse the repository at this point in the history
  • Loading branch information
marciofunes committed Dec 9, 2020
1 parent 484fd6b commit 3679e3c
Show file tree
Hide file tree
Showing 19 changed files with 2,567 additions and 2,655 deletions.
30 changes: 12 additions & 18 deletions source/architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,10 @@ Using dojot is as follows: a user configures IoT devices through the GUI or
directly using the REST APIs provided by the API Gateway. Data processing flows
might be also configured - these entities can perform a variety of actions,
such as generate notifications when a particular device attribute reaches a
certain threshold or save all data generated by a device onto an external
database. As devices start sending their readings to dojot, a user can:
certain threshold or send data generated by a device to an external service.
As devices start sending their readings to dojot, a user can:

- receive these readings in real time by socket.io channels;
- receive these readings in real time by socket.io or websocket channels;
- consolidate all data into virtual devices;
- gather all data from historical database, and so on.

Expand All @@ -48,7 +48,7 @@ easy way to perform management operations for all entities related to the
platform (users, devices, templates and flows) and can also be used to check if
everything is working fine.

The user contexts are isolated and there are no data sharing, the access
The tenant contexts are isolated and there are no data sharing, the access
credentials are validated by the authorization service for each and every
operation (API Request). Therefore, a user belonging to a particular context
(tenant) cannot reach any data (including devices, templates, flows or any
Expand All @@ -75,8 +75,8 @@ comparison with other open-source messaging solutions, Kafka seems to be more
appropriate to fulfil *dojot*'s architectural requirements (responsibility
isolation, simplicity, and so on).

In Kafka, a specialized topics structure is used to insure isolation between
different users and applications data, enabling a multi-tenant infrastructure.
In Kafka, a specialized topics structure is used to insure isolation among
different tenants and applications data, enabling a multi-tenant infrastructure.

The DataBroker service makes use of an in-memory database for efficiency. It
adds context to Apache Kafka, making it possible that internal or even external
Expand All @@ -92,7 +92,7 @@ templates data models. It is also responsible for publishing any updates to all
interested components through Kafka.

This service is stateless, having its data persisted to a database, with data
isolation for users and applications, making possible a multi-tenant
isolation for tenants and applications, making possible a multi-tenant
architecture for the middleware.

IoT Agent
Expand All @@ -104,9 +104,7 @@ devices. The *dojot* platform can have multiple iot-agents, each one of them
being specialized in a specific protocol like, for instance, MQTT/JSON,
CoAP/LWM2M, Lora/ATC, Sigfox/WDN and HTTP/JSON.

It is also responsible to ensure that it communicates with devices using secure
channels.

Communication via secure channels with devices is also the responsibility of IoT agents.

User Authorization Service
**************************
Expand Down Expand Up @@ -142,14 +140,10 @@ Cron is a dojot's microservice that allows you to schedule events to be emitted
Kafka2Ftp
*********

The kafka2ftp service provides a connector solution for
forwarding messages from Apache Kafka to FTP servers.
It subscribes to the topic ``tenant.dojot.ftp``
(*tenant* is defined in the environment variable).
In these topics there are messages with information about the file name,
encoding format and file content,
these messages are published by some service or
the `Publish in FTP topic` node from Flowbroker.
The kafka2ftp service allows forwarding messages from Apache Kafka
to FTP servers. It subscribes to Kafka’s topic ``tenant.dojot.ftp``, where the
messages must follow a specific schema. Messages can be redirected to these
topics using a specific node in the flowbroker.


Persister/History
Expand Down
16 changes: 8 additions & 8 deletions source/faq/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ It’s a brazilian IoT platform launched as open source software with aims to
ease the development of solutions and the IoT ecosystem with local resources
geared towards brazilians needs.

It takes a role as an enabler platform with:
The dojot acts as a facilitating platform with:

- Open APIs which makes the access to the platform resources easy.
- Capacity to store large volumes of data in different formats.
Expand All @@ -46,12 +46,12 @@ There are 3 main ones:
- `<https://github.com/dojot/dojot>`_: this is where we keep track of all the
things related to this project as issues and there are also some component codes here.

- `<https://github.com/dojot/docker-compose>`_: repository for Docker compose
files and configurations. This is what we would recommend to use to start
with.
- `<https://github.com/dojot/docker-compose>`_: repository with the files and settings
to perform dojot in a Docker environment using the * docker-compose * tool. This is the
repository we recommend to start with dojot.

- `<https://github.com/dojot/ansible-dojot>`_: repository for ansible (kubernetes)
files and configurations.
- `<https://github.com/dojot/ansible-dojot>`_: repository with files and settings to perform
dojot in * Kubernetes * environment.

See how to use *docker-compose* and *ansible-dojot* repositories in :doc:`../installation-guide`.

Expand Down Expand Up @@ -215,7 +215,7 @@ things to keep in mind:

- When using Mosca, the topic should look like ``/<tenant>/<device-id>/attrs`` (e.g.:
``/admin/efac/attrs``). Depending on how IoT agent MQTT was started (more strict), the client ID
must also be set to ``<tenant>:<deviceid>``, such as ``admin:efac``.
must also be set to ``<tenant>:<device-id>``, such as ``admin:efac``.

- When using VerneMQ, the topic should look like ``<tenant>:<device-id>/attrs`` (e.g.:
``admin:efac/attrs``). You must also set the username for the client as ``<tenant>:<device-id>``, such
Expand Down Expand Up @@ -417,7 +417,7 @@ How can I add a new node type to its menu?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There is a tutorial on how to add new nodes and two examples of node too,
check the `flowbroker library`_ for more details.
check the `GitHub flowbroker repository`_ for more details.


Applications
Expand Down
3 changes: 2 additions & 1 deletion source/installation-guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ This page contains information about how to deploy dojot using Docker Compose an
Hardware requirements
---------------------

The estimated hardware requirements for 500 devices with updates every 15s are:
When choosing hardware configurations for Dojot *deployment*, the number of devices and the message interval they will be
configured with should be considered. As an example, the estimated hardware requirements for 500 devices with a message interval every 15s are:

.. list-table:: Hardware requirements for 500 devices
:header-rows: 1
Expand Down
14 changes: 7 additions & 7 deletions source/internal-communication.rst
Original file line number Diff line number Diff line change
Expand Up @@ -281,9 +281,9 @@ Auth + API gateway (Kong)
-------------------------

Auth is a service deeply connected to Kong. It is responsible for user
management, authentication and authorization. As such, it is invoked by Kong
whenever an request is received by one of its registered endpoints. This
section will detail how this is performed and how they work together.
management, authentication and authorization. As such, it is invoked by
Kong whenever a request is received by one of its registered endpoints.
This section will detail how this is performed and how they work together.

Kong configuration
++++++++++++++++++
Expand Down Expand Up @@ -314,7 +314,7 @@ after starting Kong. Its only purpose is to register endpoints in Kong, such as:
These commands will register the endpoint `/device/*/latest` and `/subscription`
and all requests to it are going to be forwarded to `http//data-broker:80`. You
and all requests to it are going to be forwarded to `http://data-broker:80`. You
can check the documentation on how to add endpoints in Kong's documentation.
The links are in the :doc:`./components-and-apis` page.

Expand All @@ -323,7 +323,7 @@ selected endpoints:

#. JWT generation. The documentation for this plugin is available at `Kong JWT
plugin page`_.
#. Configuration a plugin which will forward all policies requests to Auth
#. Configures a plugin which will forward all policies requests to Auth
in order to authenticate requests. This plugin is available
inside the `Kong repository`_.

Expand Down Expand Up @@ -457,8 +457,8 @@ Persister is a very simple service which only purpose is to receive messages
from devices (using ``device-data`` subject) and store them into MongoDB. For
that, the bootstrapping procedure (detailed in `Bootstrapping tenants`_) is
performed and, whenever a new message is received, it will create a new Mongo
document and store it into the device's collection. This is shown in
:numref:`Persister`, in this case we are using tenant admin.
document and store it into the device's collection. The following image in
:numref:`Persister`, shows an example of this flow using the.

.. _Persister:
.. uml::
Expand Down
10 changes: 5 additions & 5 deletions source/iotagent-architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,9 @@ agent and they are categorized in the following groups:
able to enable or disable message processing from a particular device and
detect device liveness.

An extra feature that an IoT agent might implement is firmware updates.
Depending on is underlying protocol, it might be possible to do such thing
in a easy, secure and reliable way.
An extra feature that an IoT agent might implement is firmware updates.
Depending on its underlying protocol, it might be possible to do such
thing in an easy, secure and reliable way.

Each one of these groups is going to be detailed in the following sections.

Expand Down Expand Up @@ -515,13 +515,13 @@ and vice-versa.
Libraries to assist the development of new IotAgents
====================================================

We have libraries that abstract some points describe in previous topics
We have libraries that abstract some points described in previous topics
to facilitate the development of an IotAgent.

There are two libraries:

- node.js **recommended** (https://www.npmjs.com/package/@dojot/iotagent-nodejs)
- java (https://github.com/dojot/iotagent-java)
- java (https://jitpack.io/#dojot/iotagent-java)



Expand Down
41 changes: 19 additions & 22 deletions source/load-testing-dojot-platform.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Setting the environment up
First of all, you will need a running dojot environment. Check the :doc:`./installation-guide` for
more info.

Download dojot's Locust implementation in your machine directly from dojot's repository and change
to the corresponding version of your dojot environment:
To access the Locust implementation, download the dojot repository on your machine and switch to
the same version as your current environment:

.. code:: shell
Expand All @@ -35,10 +35,9 @@ Enter in Locust directory:
Running a simple test
---------------------

In this section, will be shown how to configure and run Locust and generate certs script in order
to execute the load test. This is a simple test where will be created 100 clients to send messages
to dojot.

In this section, it will be shown how to configure and run Locust, and also how to generate
certificates with the generate_certs script in order to execute the load test. This is a simple
test where will be created 100 clients to send messages to dojot.

Configuration
^^^^^^^^^^^^^
Expand Down Expand Up @@ -90,8 +89,7 @@ variables:
Generating the certificates
^^^^^^^^^^^^^^^^^^^^^^^^^^^

As said before, the communication from Locust to dojot is secure, so there is the need to create
certificates.
As said before, the communication from Locust to dojot is secure, then, it is necessary to use certificates.

There are two ways of simulating devices: you can create fake devices (will not show up in
dojot's GUI) or real devices. In this part of the tutorial, we will create real devices, so you can
Expand Down Expand Up @@ -278,8 +276,8 @@ variables:
REDIS_PORT: "6379"
As you can see, the configurations changed a little bit, with the changes being only about the
master and Redis location.
As you can see, the configurations have changed a little bit, with the changes
being only about the master and Redis location.


Generating certificates
Expand All @@ -295,9 +293,7 @@ Before running the script, we need to initialize the Locust master. Inside Locus
docker-compose -f Docker/docker-compose-master.yml up
After its initialization, initialize the ``generate_certs`` script docker compose in the **primary**
machine and go inside it:
After its initialization, initialize the docker compose ``generate_certs`` script in the **primary** machine and enter in it:

.. code:: shell
Expand Down Expand Up @@ -361,9 +357,8 @@ Type ``1000`` in ``Number of users to simulate``, ``10`` in ``Hatch rate`` and c
Using Grafana's Locust dashboard
--------------------------------

The Locust web interface is easy to use and simple, but there are some downsides. The major one is
the persistence: it does not store any data if you exit the page, even refreshing it removes all of
its history.
The Locust web interface is easy and simple to use, but there are some downsides. The major
one is the persistence: The history data will be deleted as soon as you close or refresh the page.

To solve this problem, we chose to add the Locust Exporter image to the master docker compose file,
allowing us to export all of its metrics in a Prometheus-compatible format. That way we can persist
Expand Down Expand Up @@ -430,17 +425,19 @@ For dojot platform:
+---------------+--------------------------+------+------+
| Machine name | Hosted services | CPU | RAM |
+===============+==========================+======+======+
| dojot-verne-1 | VerneMQ, K2V and V2K | 7 | 16GB |
| dojot-verne-1 | VerneMQ, K2V and V2K | 8 | 8GB |
+---------------+--------------------------+------+------+
| dojot-verne-2 | VerneMQ, K2V and V2K | 8 | 8GB |
+---------------+--------------------------+------+------+
| dojot-verne-2 | VerneMQ, K2V and V2K | 7 | 16GB |
| dojot-verne-3 | VerneMQ, K2V and V2K | 8 | 8GB |
+---------------+--------------------------+------+------+
| dojot-verne-3 | VerneMQ, K2V and V2K | 7 | 16GB |
| dojot-x509 | x509 identity manager | 4 | 4GB |
+---------------+--------------------------+------+------+
| dojot-ejbca | x509 identity manager | 6 | 8GB |
| dojot-kafka | Kafka and Zookeeper | 6 | 6GB |
+---------------+--------------------------+------+------+
| dojot-kafka | Kafka and Zookeeper | 6 | 12GB |
| dojot-dojot | The rest of the services | 4 | 4GB |
+---------------+--------------------------+------+------+
| dojot-main | The rest of the services | 6 | 8GB |
| haproxy | Load balancer | 4 | 4GB |
+---------------+--------------------------+------+------+

For Locust, we will use 5 replicas of the same machine, with **14GB** of RAM and **9** CPUs.
Expand Down

0 comments on commit 3679e3c

Please sign in to comment.