adjust architecture overview and remove concierge from docs

* also adjust changed piggyback "names"

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

documentation/src/main/resources/_data/sidebars/ditto_sidebar.yml

@@ -292,9 +292,6 @@ entries:
                   - title: Connectivity
                     url: /architecture-services-connectivity.html
                     output: web
-                  - title: Concierge
-                    url: /architecture-services-concierge.html
-                    output: web
                   - title: Gateway
                     url: /architecture-services-gateway.html
                     output: web

documentation/src/main/resources/architecture/DADR-0005-semantic-versioning.md

@@ -4,6 +4,7 @@ Date: 27.01.2020
 
 * Enhancement of adding `@since` in javadoc: 21.02.2020
 * Adjusted [Modules considered API](#modules-considered-api) with Ditto 2.0 in 05.2021
+* Adjusted [Modules considered API](#modules-considered-api) with Ditto 3.0 in 08.2022
 
 ## Status
 
@@ -63,13 +64,14 @@ ditto-jwt-model
 ditto-rql-model
 ditto-rql-query
 ditto-rql-parser
+ditto-rql-search-option-parser
 ditto-policies-model
 ditto-things-model
 ditto-thingsearch-model
-ditto-concierge-model
 ditto-connectivity-model
+ditto-placeholders
 ditto-protocol
-ditto-utils
+ditto-utils-jsr305
 ```
 
 ### Javadoc documentation using `@since`

documentation/src/main/resources/pages/ditto/architecture-overview.md

@@ -13,20 +13,22 @@ which responsibilities.
 This overview shows the Ditto services (components), the externally provided and consumed API endpoints, 
 the external dependencies (MongoDB and nginx) and the relations of the services to each other.
 
-{% include image.html file="pages/architecture/context-overview.png" alt="Ditto services and context" caption="Ditto services in blue and context with nginx as reverse proxy and MongoDB" max-width=800 %}
+{% include image.html file="pages/architecture/ditto-architecture-overview-2022.png" alt="Ditto services and context" caption="Ditto services in blue and context with nginx as reverse proxy and MongoDB" max-width=800 %}
 
 The components have the following tasks:
 
-* [Policies](architecture-services-policies.html): persistence of [Policies](basic-policy.html)
-* [Things](architecture-services-things.html): persistence of [Things](basic-thing.html) and [Features](basic-feature.html)
+* [Policies](architecture-services-policies.html): persistence + enforcement (authorization) of [Policies](basic-policy.html)
+* [Things](architecture-services-things.html): persistence + enforcement (authorization) of [Things](basic-thing.html) 
+  and [Features](basic-feature.html)
 * [Things-Search](architecture-services-things-search.html): tracking changes to `Things`, `Features`, `Policies` and 
   updating an optimized search index + executes queries on this search index
-* [Concierge](architecture-services-concierge.html): orchestrates and authorizes the backing persistence services
 * [Gateway](architecture-services-gateway.html): provides HTTP and WebSocket API
 * [Connectivity](architecture-services-connectivity.html):
-  sends [Ditto Protocol](protocol-overview.html) messages to external message brokers and receives messages from them. <br>
-  Supported transport protocols are AMQP 1.0 (e.g. [Eclipse Hono](https://eclipse.org/hono/)),
-  AMQP 0.9.1 (e.g. RabbitMQ), MQTT 3.1.1 (e.g. [Eclipse Mosquitto](https://www.eclipse.org/mosquitto/)), plain HTTP or Apache Kafka 2.x.
+   * persistence of [Connections](basic-connections.html)
+   * sends [Ditto Protocol](protocol-overview.html) messages to external message brokers and receives messages from them
+
+All services run in the same [Akka cluster](https://doc.akka.io/docs/akka/current/typed/cluster-concepts.html) and can
+reach each other via TCP without the need for an additional message broker in between.
 
 ## Components
 
@@ -44,8 +46,6 @@ All microservices can communicate asynchronously in a Ditto cluster. Communicati
 [Akka remoting](https://doc.akka.io/docs/akka/current/general/remoting.html) which means that each service acts as server, 
 providing a TCP endpoint, as well as client sending data to other services.
 
-Therefore, it is required that all Ditto microservices can reach each others port `2551`.
-
-Another consequence is that all messages which are sent between Ditto microservices are in a way serializable and deserializable.
+All messages which are sent between Ditto microservices must in a way be serializable and deserializable.  
 All Ditto [signals](basic-signals.html) can be serialized from Java objects to JSON representation and deserialized back 
 from JSON to Java objects. 

documentation/src/main/resources/pages/ditto/architecture-services-connectivity.md

@@ -24,17 +24,22 @@ The model of the connectivity service is defined around the entity `Connection`:
 
 Other services can communicate with the connectivity service via:
 
-* [commands](https://github.com/eclipse/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model/signals/commands):
-  containing commands and command responses which are processed by this service
-* [events](https://github.com/eclipse/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model/signals/events):
-  containing events which are emitted when entities managed by this service were modified
+* [ConnectivityCommands](https://github.com/eclipse/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model/signals/commands/ConnectivityCommand.java):
+  implementing classes provide commands which are processed by this service
+* [ConnectivityEvents](https://github.com/eclipse/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model/signals/events/ConnectivityEvent.java):
+  implementing classes represent events which are emitted when entities managed by this service were modified
 
 ## Persistence
 
 The connectivity service uses [Akka persistence](https://doc.akka.io/docs/akka/current/persistence.html?language=java) and 
 with that [Event sourcing](basic-signals.html#architectural-style) in order to persist changes to 
 and restore persisted [connections](basic-connections.html).
 
+## Enforcement
+
+The connectivity service does not enforce/authorize [connection signals](#signals) by a [policy](basic-policy.html) as 
+the connection does not contain a `policyId`.
+
 ## Tasks
 
 * create/remove connections (by persisting them)
@@ -48,7 +53,6 @@ and restore persisted [connections](basic-connections.html).
 
 
 
-[AMQP 1.0]: connectivity-protocol-bindings-amqp10.html
 [Ditto Protocol]: protocol-overview.html
 [Ditto Protocol Message]: protocol-specification-things-messages.html
 [payload mapping]: protocol-specification-things-messages.html

documentation/src/main/resources/pages/ditto/architecture-services-policies.md

@@ -18,15 +18,17 @@ The model of the policies service is defined around the entity `Policy`:
 
 Other services can communicate with the policies service via:
 
-
-* [commands](https://github.com/eclipse/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model/signals/commands):
-  containing commands and command responses which are processed by this service
-* [events](https://github.com/eclipse/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model/signals/events):
-  containing events which are emitted when entities managed by this service were modified
+* [PolicyCommands](https://github.com/eclipse/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model/signals/commands/PolicyCommand.java):
+  implementing classes provide commands which are processed by this service
+* [PolicyEvents](https://github.com/eclipse/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model/signals/events/PolicyEvent.java):
+  implementing classes represent events which are emitted when entities managed by this service were modified
 
 ## Persistence
 
 The policies service uses [Akka persistence](https://doc.akka.io/docs/akka/current/persistence.html?language=java) and 
 with that [Event sourcing](basic-signals.html#architectural-style) in order to persist changes to  
 and restore persisted [policies](basic-policy.html).
 
+## Enforcement
+
+The policies service enforces/authorizes [policy signals](#signals) by the "own" [policy](basic-policy.html).

documentation/src/main/resources/pages/ditto/architecture-services-things-search.md

@@ -7,11 +7,11 @@ permalink: architecture-services-things-search.html
 
 The "things-search" service takes care of:
 
-
 * updating an optimized search index of `Things` based on the [events](basic-signals-event.html) emitted by the 
   [things](architecture-services-things.html) and [policies](architecture-services-policies.html) services when entities
   are changed there
-* executing search queries against the search index in order to find out which `Things` match a given search
+* executing search queries against the search index in order to find out which `Things` (which `thingId`s) match a 
+  given search
 
 ## Model
 
@@ -21,7 +21,6 @@ and [policies](architecture-services-policies.html) services.
 It however contains a model which can transform an <a href="#" data-toggle="tooltip" data-original-title="{{site.data.glossary.rql}}">RQL</a> 
 search query into a Java domain model which is defined here:
 
-
 * [rql parser ast](https://github.com/eclipse/ditto/tree/master/rql/model/src/main/java/org/eclipse/ditto/rql/model/predicates/ast)
 
 ## Signals
@@ -39,31 +38,7 @@ provide a full search on arbitrary `Thing` data.
 
 Things-Search creates the following MongoDB collections:
 
-* `searchThings`: The search index.
-* `searchThingsSyncThings`: A single-document capped collection containing the instant until which `Thing` events are
-indexed for sure; expected to be 30 minutes before the current time.
-* `searchThingsSyncPolicies`: A single-document capped collection containing the instant until which `Policy` events
-are indexed for sure; expected to be 30 minutes before the current time.
-
-## Migration from Ditto 0.9.0-M1
-
-The index schema has changed since Ditto version 0.9.0-M1. Data migration is obligatory to upgrade an existing
-installation running Ditto version 0.9.0-M1 or earlier. Expected duration of data migration is 1/60th of the lifetime
-of the Ditto installation.
-
-1. *After* stopping the cluster of Ditto 0.9.0-M1, drop unnecessary collections:
-```javascript
-db.getCollection('thingEntities').drop();
-db.getCollection('policyBasedSearchIndex').drop();
-db.getCollection('thingsSearchSyncStatePolicies').drop();
-```
-
-2. *Before* starting the upgraded Ditto cluster, write into `searchThingsSyncThings` the timestamp when the Ditto cluster started for the first time:
-```javascript
-var startingTimestamp = new Date(TIMESTAMP-WHEN-DITTO-CLUSTER-STARTED-FOR-THE-FIRST-TIME); // e.g. new Date('2019-01-01T00:00:00.000Z')
-db.getCollection('thingsSearchSyncStateThings').renameCollection('searchThingsSyncThings');
-db.getCollection('searchThingsSyncThings').insert({'ts':startingTimestamp});
-```
+* `search`: The search index.
+* `searchSync`: A single-document capped collection containing the instant until which `Thing` events are
+indexed for sure.
 
-3. Start the upgraded Ditto cluster. All `Thing` events persisted after the timestamp in `searchThingsSyncThings` 
-will be indexed in the background.

documentation/src/main/resources/pages/ditto/architecture-services-things.md

@@ -17,14 +17,18 @@ The model of the things service is defined around the entities `Thing` and `Feat
 
 Other services can communicate with the things service via:
 
-* [commands](https://github.com/eclipse/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model/signals/commands):
-  containing commands and command responses which are processed by this service
-* [events](https://github.com/eclipse/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model/signals/events):
-  containing events which are emitted when entities managed by this service were modified
+* [ThingCommands](https://github.com/eclipse/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model/signals/commands/ThingCommand.java):
+  implementing classes provide commands which are processed by this service
+* [ThingEvents](https://github.com/eclipse/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model/signals/events/ThingEvent.java):
+  implementing classes represent events which are emitted when entities managed by this service were modified
 
 ## Persistence
 
 The things service uses [Akka persistence](https://doc.akka.io/docs/akka/current/persistence.html?language=java) and 
 with that [Event sourcing](basic-signals.html#architectural-style) in order to persist changes to 
 and restore persisted [things](basic-thing.html).
 
+## Enforcement
+
+The things service enforces/authorizes [thing signals](#signals) by the via `policyId` referenced 
+[policy](basic-policy.html).

documentation/src/main/resources/pages/ditto/connectivity-manage-connections.md

@@ -27,7 +27,6 @@ See [Basic Authentication](basic-auth.html#authorization-context-in-devops-comma
 
 The following commands are available in order to manage connections:
 
-
 * [create](#create-connection)
 * [modify](#modify-connection)
 * [retrieve](#retrieve-connection)
@@ -42,7 +41,8 @@ Create a new connection by sending the following DevOps command.
   "targetActorSelection": "/system/sharding/connection",
   "headers": {
     "aggregate": false,
-    "is-group-topic": true
+    "is-group-topic": true,
+    "ditto-sudo": true
   },
   "piggybackCommand": {
     "type": "connectivity.commands:createConnection",
@@ -65,7 +65,8 @@ Modify an existing connection by sending the following DevOps command.
   "targetActorSelection": "/system/sharding/connection",
   "headers": {
     "aggregate": false,
-    "is-group-topic": true
+    "is-group-topic": true,
+    "ditto-sudo": true
   },
   "piggybackCommand": {
     "type": "connectivity.commands:modifyConnection",
@@ -86,7 +87,8 @@ The only parameter necessary for retrieving a connection is the `connectionId`.
   "targetActorSelection": "/system/sharding/connection",
   "headers": {
     "aggregate": false,
-    "is-group-topic": true
+    "is-group-topic": true,
+    "ditto-sudo": true
   },
   "piggybackCommand": {
     "type": "connectivity.commands:retrieveConnection",
@@ -153,7 +155,7 @@ The only parameter necessary for deleting a connection is the `connectionId`.
 
 ## Helper commands
 
-The following commands are available to help creating connections and retrieving the status of existing connections:
+The following commands are available to help to create connections and retrieving the status of existing connections:
 
 * [test connection](#test-connection)
 * [retrieve ids of all connections](#retrieve-ids-of-all-connections)