Merge pull request #90 from jonaphael/doc-kafkaws

[release] kafka-ws documentation update

source/architecture.rst

@@ -209,6 +209,12 @@ communicate securely with the dojot platform, as the data collected by the
 device is transmitted over a secure (encrypted) channel and it is also possible
 to guarantee the integrity of the data.
 
+Kafka WS
+***************
+This component is responsible for retrieving data from Apache Kafka through pure WebSocket 
+connections. It was designed to allow dojot users to retrieve realtime raw and/or processed 
+data from dojot devices.
+
 Infrastructure
 --------------
 

source/components-and-apis.rst

@@ -110,6 +110,10 @@ Components
     - `GitHub - Kafka2Ftp`_
     -
     -
+  * - Kafka WS
+    - `GitHub - Kafka WS`_
+    -
+    - `API - kafka-ws`_
 
 
 Exposed APIs (API Gateway)
@@ -229,6 +233,16 @@ See more about how using APIs in :doc:`./using-api-interface`.
      - **/api/v1**
      - `API - x509-identity-mgmt`_
      - Yes
+   * - Kafka WS
+     - **/kafka-ws/v1/ticket**
+     -  /v1/ticket
+     -
+     - Yes
+   * - Kafka WS
+     - **/kafka-ws/v1**
+     - /v1
+     -
+     - No
 
 **NOTE: Some of the components' endpoints aren't exposed, but are used internally.**
 
@@ -297,6 +311,14 @@ TCP port 8000), see the following table.
    * - X.509 Identity Management
      - host:3000/api/v1
      - host:8000/x509/v1
+   * - Kafka WS
+     - host:8080/v1/ticket
+     - host:8000/kafka-ws/v1/ticket
+   * - Kafka WS
+     - host:8080/v1/topics
+     - host:8000/kafka-ws/v1/topics
+
+
 
 Kafka messages
 --------------
@@ -384,3 +406,6 @@ dojot.
 .. _API - x509-identity-mgmt: https://dojot.github.io/dojot/x509-identity-mgmt/apiary_v0.5.0.html
 
 .. _GitHub - Kafka2Ftp: https://github.com/dojot/dojot/tree/v0.5.0/connector/kafka2ftp
+
+.. _GitHub - Kafka WS: https://github.com/dojot/dojot/tree/v0.5.0/subscription-engine/kafka-ws
+.. _API - kafka-ws: https://dojot.github.io/dojot/kafka-ws/apiary_v0.5.0.html

source/faq/faq.rst

@@ -443,8 +443,8 @@ could be integrated with dojot:
   - **Retrieving historical data**: you might want to periodically read all
     historical data related to a device. This can be done by using the *history* API
   - **Retrieving real time data**: you might want to read device-related real time data.
-    This can be done by using *kafka-ws*, a *websocket* based implementation.
-    To better understand how to use *kafka-ws* check :ref:`Kafka-WS Internal`.
+    This can be done by using *Kafka WS*, a *websocket* based implementation.
+    To better understand how to use *Kafka WS* check :ref:`Kafka-WS Internal`.
   - **Using flowbroker to pre-process data**: for more flexible ways of data manipulation, you can use flows.
     They can process/transform data so they can
     be properly sent to your application via HTTP request, or stored
@@ -454,7 +454,7 @@ could be integrated with dojot:
 
 All these endpoints should bear an access token, see more :ref:`Getting access token`.
 
-Check the documentation API for  *history* and *kafka-ws* in :doc:`./../components-and-apis`.
+Check the documentation API for  *History* and *Kafka WS* in :doc:`./../components-and-apis`.
 And to a tutorial on how to use the flow, check :doc:`../flow`.
 
 

source/internal-communication.rst

@@ -7,7 +7,7 @@ This page describes how each service in dojot communicate with each other.
 Components
 ----------
 
-The main components current in dojot are shown in :numref:`dojot_components`.
+The main components that are currently in `dojot platform`_ are :numref:`dojot_components`.
 
 .. _dojot_components:
 .. uml::
@@ -523,7 +523,7 @@ check DataBroker documentation in :doc:`./components-and-apis`.
 
 .. NOTE::
    The real time socket.io connections via Data Broker will be discontinued in
-   future releases. Use `Kafka-WS`_ instead.
+   future releases. Use `Kafka WS`_ instead.
 
 Certificate authority
 ---------------------
@@ -540,40 +540,80 @@ certificate-related services for devices.
 
 .. _Kafka-WS Internal:
 
-Kafka-WS
+Kafka WS
 --------
 
-*Kafka WebSocket* service allows the users to retrieve conditional and/or
-partial real time data from a given dojot topic in its internal Kafka cluster.
+The *Kafka WS* service allows users to retrieve conditional and/or
+partial real time data from a given dojot topic in a Kafka cluster.
+It works with pure websocket connections, so you can create websocket 
+clients in any language you want as long as they support RFC 6455.
+
+Connecting to the service
+++++++++++++++++++++++++++
+
+The connection is done in two steps: you must first obtain a single-use ticket 
+through a HTTP request, then connect to the service via websocket passing it
+as a parameter.
+
+First step: Get the single-use ticket
+*************************************
+A ticket allows the user to subscribe to a dojot topic. To obtain it is necessary 
+to have a JWT access token that is issued by the platform's Authentication/Authorization 
+service. The ticket must be retrieved via a HTTP request using the GET verb to the 
+`<base-url>/kafka-ws/v1/ticket` endpoint. The request must contain the `Authorization` 
+header with the previously retrieved JWT token as a value. Example:
+
+| `GET <base-url>/kafka-ws/v1/ticket`
+| `Authorization: Bearer [Encoded JWT]`
+
+The component responds with the following syntax:
+
+| `HTTP/1.1 200 OK`
+| `Content-type: application/json`
+.. code-block:: json
+
+  {
+    "ticket": "[an opaque ticket of 64 hexadecimal characters]"
+  }
+
+Note: In the context of a dojot deployment the JWT Token is provided by the Auth service, 
+and is validated by the API Gateway before redirecting the connection to the *Kafka WS*. 
+So, no validations are done by the Kafka WS.
+
+Second step: Establish a websocket connection
+**********************************************
+The connection is done via pure websockets using the URI `<base-url>/kafka-ws/v1/topics/:topic`.
+You must pass the previously generated ticket as a parameter of this URI. It is also possible 
+to pass conditional and filter options as parameters of the URI.
 
 Behavior when requesting a ticket and a websocket connection
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
-Below we can understand the behavior of the Kafka-WS service when a user
-(through a `user agent`_) requests a ticket in order to establish a
-communication via websocket with Kafka-WS.
+Below we can understand the behavior of the Kafka WS service when a user
+(through an `user agent`_) requests a ticket in order to establish a
+communication via websocket with Kafka WS.
 
-Note that when the user requests a new ticket, Kafka-WS extracts some
+Note that when the user requests a new ticket, Kafka WS extracts some
 information from the *user's access token* (`JWT`_) and generates a
 *signed payload*, to be used later in the decision to authorize (or not)
 the websocket connection. From the payload a *ticket* is generated and
 both are stored in Redis, where the ticket is the key to obtain the payload.
-A `TTL`_ is defined by Kafka-WS, so the user has to use the ticket within the
+A `TTL`_ is defined by Kafka WS, so the user has to use the ticket within the
 established time, otherwise, Redis automatically deletes the ticket and payload.
 
-After obtaining the ticket, the user makes an HTTP request to Kafka-WS
+After obtaining the ticket, the user makes an HTTP request to Kafka WS
 requesting an upgrade to communicate via *websocket*. As the specification of
 this HTTP request limits the use of additional headers, it is necessary to send
-the ticket through the URL, so that it can be validated by Kafka-WS before
+the ticket through the URL, so that it can be validated by Kafka WS before
 authorizing the upgrade.
 
 Since the ticket is valid, that is, it corresponds to an entry on Redis,
-Kafka-WS retrieves the payload related to the ticket, verifies the integrity
+Kafka WS retrieves the payload related to the ticket, verifies the integrity
 of the payload and deletes that entry on Redis so that the ticket cannot be
 used again.
 
 With the payload it is possible to make the decision to authorize the upgrade
-to websocket or not. If authorization is granted, Kafka-WS opens a subscription
+to websocket or not. If authorization is granted, Kafka WS opens a subscription
 channel based on a specific topic in Kafka. From there, the upgrade to websocket
 is established and the user starts to receive data as they are being published
 in Kafka.
@@ -628,3 +668,4 @@ in Kafka.
 .. _Kafka's official documentation: https://kafka.apache.org/documentation/
 .. _Kong JWT plugin page: https://docs.konghq.com/hub/kong-inc/jwt/
 .. _Kong repository: https://github.com/dojot/kong
+.. _dojot platform: http://dojot.com.br/

source/locale/pt_BR/LC_MESSAGES/architecture.po

@@ -7,8 +7,8 @@ msgid ""
 msgstr ""
 "Project-Id-Version: dojot 0.0.0\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2020-09-14 12:19-0300\n"
-"PO-Revision-Date: 2020-09-14 12:19-0300\n"
+"POT-Creation-Date: 2020-09-17 10:30-0300\n"
+"PO-Revision-Date: 2020-09-22 11:58-0300\n"
 "Last-Translator: \n"
 "Language: pt_BR\n"
 "Language-Team: \n"
@@ -502,22 +502,37 @@ msgstr ""
 "integridade dos dados."
 
 #: ../../source/architecture.rst:213
+msgid "Kafka WS"
+msgstr "Kafka WS"
+
+#: ../../source/architecture.rst:214
+msgid ""
+"This component is responsible for retrieving data from Apache Kafka through "
+"pure WebSocket connections. It was designed to allow dojot users to "
+"retrieve realtime raw and/or processed data from dojot devices."
+msgstr ""
+"Este componente é responsável por obter dados do Apache Kafka via uma "
+"conexão Websocket pura. Foi desenhado para permitir que usuários da dojot "
+"possam recuperar dados brutos e/ou processados em tempo real de dispositivos da "
+"dojot."
+
+#: ../../source/architecture.rst:219
 msgid "Infrastructure"
 msgstr "Infraestrutura"
 
-#: ../../source/architecture.rst:215
+#: ../../source/architecture.rst:221
 msgid "A few extra components are used in dojot,they are:"
 msgstr "Alguns outros componentes são utilizados na dojot, são eles:"
 
-#: ../../source/architecture.rst:217
+#: ../../source/architecture.rst:223
 msgid ""
 "postgres: this database is used to persist data from many components, such "
 "as Device Manager."
 msgstr ""
 "postgres: esse banco de dados é utilizado para persistir informações de "
 "vários componentes, como do gerenciador de dispositivos."
 
-#: ../../source/architecture.rst:220
+#: ../../source/architecture.rst:226
 msgid ""
 "redis: in-memory database used as cache in many components, such as service "
 "orchestrator, subscription manager, IoT agents, and so on. It is very light "
@@ -527,7 +542,7 @@ msgstr ""
 "componentes, como o serviço de orquestração, gerenciador de subscrição, "
 "agentes IoT e outros. É bem leve e fácil de usar."
 
-#: ../../source/architecture.rst:224
+#: ../../source/architecture.rst:230
 msgid ""
 "rabbitMQ: message broker used in service orchestrator in order to implement "
 "action flows related that should be applied to messages received from "
@@ -537,27 +552,27 @@ msgstr ""
 "implementar fluxos de ação relacionados que devem ser aplicados a mensagens "
 "recebidas de componentes."
 
-#: ../../source/architecture.rst:228
+#: ../../source/architecture.rst:234
 msgid ""
 "mongo database: widely used database solution that is easy to use and "
 "doesn't add a considerable access overhead (where it was employed in dojot)."
 msgstr ""
 "mongo: solução de banco de dados amplamente utilizada, fácil de usar e não "
 "adiciona *overhead* considerável (nos locais onde foi empregado na dojot)."
 
-#: ../../source/architecture.rst:231
+#: ../../source/architecture.rst:237
 msgid "zookeeper: keeps replicated services within a cluster under control."
 msgstr "zookeeper: mantém sob controle serviços replicados em cluster."
 
-#: ../../source/architecture.rst:235
+#: ../../source/architecture.rst:241
 msgid "Communications"
 msgstr "Comunicação"
 
-#: ../../source/architecture.rst:237
+#: ../../source/architecture.rst:243
 msgid "All components communicate with each other in two ways:"
 msgstr "Todos os componentes se comunicam de duas maneiras:"
 
-#: ../../source/architecture.rst:239
+#: ../../source/architecture.rst:245
 msgid ""
 "Using HTTP requests: if one component needs to retrieve data from other "
 "one, say an IoT agent needs the list of currently configured devices from "
@@ -568,7 +583,7 @@ msgstr ""
 "do gerenciador de dispositivos, ele pode enviar uma requisição HTTP para o "
 "componente apropriado."
 
-#: ../../source/architecture.rst:243
+#: ../../source/architecture.rst:249
 msgid ""
 "Using Kafka messages: if one component needs to send new information about "
 "a resource controlled by it (such as new devices created in Device "