added releasenotes and blogpost for Ditto 2.4.0

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
eclipse-ditto · Mar 30, 2022 · 06fa787 · 06fa787
1 parent 27ac06d
commit 06fa787
Show file tree

Hide file tree

Showing 4 changed files with 216 additions and 12 deletions.
diff --git a/documentation/src/main/resources/_posts/2022-04-06-release-announcement-240.md b/documentation/src/main/resources/_posts/2022-04-06-release-announcement-240.md
@@ -0,0 +1,82 @@
+---
+title: "Announcing Eclipse Ditto Release 2.4.0"
+published: true
+permalink: 2022-04-06-release-announcement-240.html
+layout: post
+author: thomas_jaeckle
+tags: [blog]
+hide_sidebar: true
+sidebar: false
+toc: false
+---
+
+The Eclipse Ditto teams announces availability of Eclipse Ditto [2.4.0](https://projects.eclipse.org/projects/iot.ditto/releases/2.4.0).
+
+The main topics in this release were the move from Java 11 to Java 17 (and the switch in Ditto's pre-built Docker containers from OpenJ9 to Hotspot runtime), 
+W3C WoT (Web of Things) integration and enhanced placeholder capabilities, e.g. used in JWT claim extraction and signal 
+enrichment.
+
+## Adoption
+
+Companies are willing to show their adoption of Eclipse Ditto publicly: 
+[https://iot.eclipse.org/adopters/?#iot.ditto](https://iot.eclipse.org/adopters/?#iot.ditto)
+
+From our various [feedback channels](feedback.html) we however know of more adoption.  
+If you are making use of Eclipse Ditto, it would be great to show this by adding your company name to that list of 
+known adopters.  
+In the end, that's one main way of measuring the success of the project.
+
+
+## Changelog
+
+The main improvements and additions of Ditto 2.4.0 are:
+
+* W3C WoT (Web of Things) integration
+* SSE (ServerSentEvent) API for subscribing to messages
+* Recovery status for connections indicating when e.g. recovery is no longer tried after max backoff
+* Enhance placeholders to resolve to multiple values
+* Advanced JWT placeholder operations
+* Support for a wildcard/placeholder identifying the changed feature in order to enrich e.g. its definition
+
+The following notable fixes are included:
+
+* Several fixes and improvements regarding consistency and performance of search updates
+* Don't publish messages with failed enrichments and issue failed ack
+* Filter for incorrect element types in jsonArray of feature definitions
+* Fix of placeholder resolvment in "commandHeaders" of "ImplicitThingCreation" mapper
+* Fix `fn:substring-after()` function returning incorrect data
+
+The following non-functional work is also included:
+
+* Upgrade of compiler target level for service modules from Java 11 to Java 17
+* Switch of used Java runtime in pre-built Docker containers from OpenJ9 to Hotspot
+* Publication of pre-built multi-architecture Docker images for `linux/amd64` (as always) and now in addition `linux/arm64`
+* Removal of rate limiting / throttling limits as default
+* Update of several used dependencies
+
+Please have a look at the [2.4.0 release notes](release_notes_240.html) for a more detailed information on the release.
+
+
+## Artifacts
+
+The new Java artifacts have been published at the [Eclipse Maven repository](https://repo.eclipse.org/content/repositories/ditto/)
+as well as [Maven central](https://repo1.maven.org/maven2/org/eclipse/ditto/).
+
+The Ditto JavaScript client release was published on [npmjs.com](https://www.npmjs.com/~eclipse_ditto):
+* [@eclipse-ditto/ditto-javascript-client-dom](https://www.npmjs.com/package/@eclipse-ditto/ditto-javascript-client-dom)
+* [@eclipse-ditto/ditto-javascript-client-node](https://www.npmjs.com/package/@eclipse-ditto/ditto-javascript-client-node)
+
+
+The Docker images have been pushed to Docker Hub:
+* [eclipse/ditto-policies](https://hub.docker.com/r/eclipse/ditto-policies/)
+* [eclipse/ditto-things](https://hub.docker.com/r/eclipse/ditto-things/)
+* [eclipse/ditto-things-search](https://hub.docker.com/r/eclipse/ditto-things-search/)
+* [eclipse/ditto-gateway](https://hub.docker.com/r/eclipse/ditto-gateway/)
+* [eclipse/ditto-connectivity](https://hub.docker.com/r/eclipse/ditto-connectivity/)
+* [eclipse/ditto-concierge](https://hub.docker.com/r/eclipse/ditto-concierge/)
+
+<br/>
+<br/>
+{% include image.html file="ditto.svg" alt="Ditto" max-width=500 %}
+--<br/>
+The Eclipse Ditto team
diff --git a/documentation/src/main/resources/pages/ditto/basic-placeholders.md b/documentation/src/main/resources/pages/ditto/basic-placeholders.md
@@ -124,7 +124,7 @@ In [connections](basic-connections.html), the following placeholders are availab
 | `{%raw%}{{ feature:id }}{%endraw%}`            | the ID of the feature (only available if the processed signal was related to a feature)                                                                                                         |
 | `{%raw%}{{ header:<header-name> }}{%endraw%}`  | external header value for connection sources, or Ditto protocol header value for targets and reply-targets (both case-insensitive)                                                              |
 | `{%raw%}{{ request:subjectId }}{%endraw%}`     | primary authorization subject of a command, or primary authorization subject that caused an event                                                                                               |
-| `{%raw%}{{ topic:full }}{%endraw%}`            | full [Ditto Protocol topic path](protocol-specification-topic.html)<br/>in the form `{namespace}/{entityId}/{group}/`<br/>`{channel}/{criterion}/{action-subject}`                              |
+| `{%raw%}{{ topic:full }}{%endraw%}`            | full [Ditto Protocol topic path](protocol-specification-topic.html)<br/>in the form `{namespace}/{entityName}/{group}/`<br/>`{channel}/{criterion}/{action-subject}`                            |
 | `{%raw%}{{ topic:namespace }}{%endraw%}`       | Ditto Protocol [topic namespace](protocol-specification-topic.html#namespace)                                                                                                                   |
 | `{%raw%}{{ topic:entityName }}{%endraw%}`      | Ditto Protocol [topic entity name](protocol-specification-topic.html#entity-name)                                                                                                               |
 | `{%raw%}{{ topic:group }}{%endraw%}`           | Ditto Protocol [topic group](protocol-specification-topic.html#group)                                                                                                                           |

diff --git a/documentation/src/main/resources/pages/ditto/release_notes_240.md b/documentation/src/main/resources/pages/ditto/release_notes_240.md
@@ -14,54 +14,161 @@ to prior Eclipse Ditto 2.x versions.
 
 Eclipse Ditto 2.4.0 includes the following topics/enhancements:
 
+* W3C WoT (Web of Things) integration
+* SSE (ServerSentEvent) API for subscribing to messages
+* Recovery status for connections indicating when e.g. recovery is no longer tried after max backoff
+* Enhance placeholders to resolve to multiple values
+* Advanced JWT placeholder operations
+* Support for a wildcard/placeholder identifying the changed feature in order to enrich e.g. its definition
+
 The following notable fixes are included:
 
+* Several fixes and improvements regarding consistency and performance of search updates
+* Don't publish messages with failed enrichments and issue failed ack
+* Filter for incorrect element types in jsonArray of feature definitions
+* Fix of placeholder resolvment in "commandHeaders" of "ImplicitThingCreation" mapper
+* Fix `fn:substring-after()` function returning incorrect data
+
 The following non-functional work is also included:
 
+* Upgrade of compiler target level for service modules from Java 11 to Java 17
+* Switch of used Java runtime in pre-built Docker containers from OpenJ9 to Hotspot
+* Publication of pre-built multi-architecture Docker images for `linux/amd64` (as always) and now in addition `linux/arm64`
+* Removal of rate limiting / throttling limits as default
+* Update of several used dependencies
+
 <br/>
 
 For a complete list of all merged PRs, inspect the following milestones:
+* [merged pull requests for milestone 2.4.0-M1](https://github.com/eclipse/ditto/pulls?q=is:pr+milestone:2.4.0-M1)
 * [merged pull requests for milestone 2.4.0](https://github.com/eclipse/ditto/pulls?q=is:pr+milestone:2.4.0)
 
 <br/>
 <br/>
 
 Compared to the latest release [2.3.0](release_notes_230.html), the following most notable changes, new features and
-bugfixes were added.
+bugfixes were added:
+
 
 ### Changes
 
-#### [Upgrade to Java 17](https://github.com/eclipse/ditto/issues/1283)
+#### [Upgrade to Java 17 + change of Java runtime to Hotspot](https://github.com/eclipse/ditto/issues/1283)
 
 We upgraded the compiler target level for our service modules from 11 to 17 and also use a Java 17 runtime environment
-for our service containers. Please note that the Ditto model still remains compatible to Java 8.
+for our service containers. Please note that the Ditto model still remains compatible to Java 8.  
 This change only affects you when you're building and deploying Ditto on your own.
 
+#### Publication of pre-built multi-architecture Docker images
+
+Acknowledging the raise of the ARM processor architecture, starting with Ditto 2.4.0, 
+Docker images for the following architectures will be published to docker.io:
+* `linux/amd64`
+* `linux/arm64` (new)
+
+#### [Removal of rate limiting / throttling limits as default](https://github.com/eclipse/ditto/pull/1324)
+
+By default, Ditto had configurations in place to rate limit consumption of messages received via:
+* AMQP 1.0 connections
+* Apache Kafka connections
+* the WebSocket endpoint
+
+These limitations are now by default disabled (as they mainly make sense for a multi-tenant environment) 
+and can be enabled manually, as mentioned in the [configuration - rate limiting section](installation-operating.html#rate-limiting).
+
+
 ### New features
 
+#### [W3C WoT (Web of Things) integration](https://github.com/eclipse/ditto/issues/1034)
+
+Ditto adds and optional (and currently *experimental*) integration of the 
+[Web of Things (WoT) Thing Description 1.1 - W3C Working Draft 11 March 2022](https://www.w3.org/TR/2022/WD-wot-thing-description11-20220311/) specification.
+
+Digital twins managed by Ditto can reference to WoT "Thing Models" (being accessible via an HTTP URL) in which the
+capabilities of the twin are defined.  
+Using this linked model, Ditto can generate a WoT "Thing Description" on a thing instance level and e.g. generate a JSON
+skeleton upon creation of a thing.  
+For more details, please have a look at the [blogpost](2022-03-03-wot-integration.html) and the 
+[WoT integration documentation](basic-wot-integration.html).
+
+#### [SSE (ServerSentEvent) API for subscribing to messages](https://github.com/eclipse/ditto/issues/1186)
+
+Messages to or from a digital twin can now be subscribed to with the [SSE endpoint](httpapi-sse.html), either on 
+[Thing level](httpapi-sse.html#subscribe-for-messages-for-a-specific-thing) or for a specific 
+[Feature](httpapi-sse.html#subscribe-for-messages-of-a-specific-feature-of-a-specific-thing)
+
+#### [Recovery status for connections indicating when e.g. recovery is no longer tried after max backoff](https://github.com/eclipse/ditto/pull/1336)
+
+The new recovery status contains one of the values:
+* ongoing
+* succeeded
+* backOffLimitReached
+* unknown
+
+and can be used to find out whether an automatic failover is still ongoing or if the max amount of configured reconnects 
+applying backoff was reached and that recovery is no longer happening.
+
+#### [Enhance placeholders to resolve to multiple values](https://github.com/eclipse/ditto/pull/1331)
+
+Placeholders may now resolve to multiple values instead of only a single one which enables e.g. applying 
+[placeholder functions](basic-placeholders.html#function-expressions) to each element of an array.
+
+#### [Advanced JWT placeholder operations](https://github.com/eclipse/ditto/pull/1309)
+
+Using the above feature of placeholders being resolved to multiple values, the JWT placeholder, which can be used
+in [scope of the OpenID connect configuration](basic-placeholders.html#scope-openid-connect-configuration), can now 
+be used with [functions](basic-placeholders.html#function-expressions).  
+This can e.g. be used in order to filter out unwanted subjects in the 
+[OpenId connect configuration](installation-operating.html#openid-connect), or to additionally split a JWT claim into 
+several values using the new `fn:split(' ')` function.
+
+Example extracting only subjects from a JSON array "roles" contained in a JWT ending with "moderator": 
+```
+{%raw%}{{ jwt:extra/roles | fn:filter('like','*moderator') }}{%endraw%}
+```
+
+#### [Support for a wildcard/placeholder identifying the changed feature in order to enrich e.g. its definition](https://github.com/eclipse/ditto/issues/710)
+
+Using the above feature of placeholders being resolved to multiple values, it is now possible to use a placeholder 
+`{%raw%}{{ feature:id }}{%endraw%}` as part of an [enrichment `extraFields` pointer](basic-enrichment.html) resolving
+to all affected feature ids of a change.
+
+This can e.g. be used to enrich all the feature `definition`s of all modified features in an event which should be 
+published via websocket or a connection:
+```
+{%raw%}extraFields=features/{{feature:id}}/definition{%endraw%}
+```
+
 
 ### Bugfixes
 
-Several bugs in Ditto 2.2.x were fixed for 2.4.0.
+Several bugs in Ditto 2.3.x were fixed for 2.4.0.
 This is a complete list of the
-[merged pull requests](https://github.com/eclipse/ditto/pulls?q=is%3Apr+milestone%3A2.4.0), including the fixed bugs.
+[merged pull requests for 2.4.0-M1](https://github.com/eclipse/ditto/pulls?q=is%3Apr+milestone%3A2.4.0-M1) and 
+[merged pull requests for 2.4.0](https://github.com/eclipse/ditto/pulls?q=is%3Apr+milestone%3A2.4.0), 
+including the fixed bugs.
 
 
 ## Migration notes
 
-Migrations required updating from Ditto 2.3.x:
-* With this release we not only switched from Java 11 to Java 17 but also from OpenJ9 to Hotspot.
-  This means that the environment variable OPENJ9_JAVA_OPTIONS needs to be renamed to JAVA_TOOL_OPTIONS.
+Migrations required updating from Ditto 2.3.x or earlier versions:
+* With this release we not only switched from Java 11 to Java 17 but also from OpenJ9 to Hotspot runtime.
+  This means that the environment variable `OPENJ9_JAVA_OPTIONS` needs to be renamed to `JAVA_TOOL_OPTIONS` and that 
+  options specific to the OpenJ9 runtime are no longer effective.  
   Make sure that all options that are defined are valid for the Hotspot JVM.
 
+
 ## Ditto clients
 
 For a complete list of all merged client PRs, inspect the following milestones:
-* [merged pull requests for milestone 2.4.0](https://github.com/eclipse/ditto-clients/pulls?q=is:pr+milestone:2.4.0)
+* [merged pull requests for milestone 2.4.0-M1](https://github.com/eclipse/ditto-clients/pulls?q=is:pr+milestone:2.4.0-M1)
 
 ### Ditto Java SDK
 
-No mentionable changes/enhancements/bugfixes.
+#### [Fix returning the revision of a Policy when retrieved via the client](https://github.com/eclipse/ditto-clients/pull/182)
+
+When using the `client.policies().retrievePolicy(PolicyId)` functionality in the Ditto Java client, the `getRevision()`
+method of the returned policy was always empty.  
+The revision will now be included.
 
 ### Ditto JavaScript SDK
 
@@ -71,3 +178,18 @@ See separate [Changelog](https://github.com/eclipse/ditto-clients/blob/master/ja
 ## Roadmap
 
 Looking forward, the current plans for Ditto 2.5.0 are:
+* Enhance MQTT (3.1.1 and 5) connections to use reactive MQTT driver (with backpressure support)
+* Improvements of MongoDB search index of [things-search](architecture-services-things-search.html), using a new index 
+  structure:
+  * smaller index size for large thing / policy combinations
+  * fewer used CPU resources on MongoDB
+  * faster search queries
+  * better vertical scalability of search index
+* Finalize the experimental WoT (Web of Things) integration when the "Thing Description 1.1" is published as W3C Recommendation
+
+Looking even more ahead, early rough plans for Ditto 3.0.0 are:
+* Simplify Ditto's architecture by:
+  * removing the [concierge service](architecture-services-concierge.html)
+  * potentially even removing the [gateway service](architecture-services-gateway.html) and merging its 
+    functionality (providing HTTP + WS endpoints) into the 
+    [connectivity service](architecture-services-connectivity.html)
diff --git a/.../src/main/java/org/eclipse/ditto/protocol/placeholders/ImmutableTopicPathPlaceholder.java b/.../src/main/java/org/eclipse/ditto/protocol/placeholders/ImmutableTopicPathPlaceholder.java
@@ -25,7 +25,7 @@
 /**
  * Placeholder implementation that replaces:
  * <ul>
- * <li>{@code topic:full} -> {@code {namespace}/{entityId}/{group}/{channel}/{criterion}/{action-subject}}</li>
+ * <li>{@code topic:full} -> {@code {namespace}/{entityName}/{group}/{channel}/{criterion}/{action-subject}}</li>
  * <li>{@code topic:namespace}</li>
  * <li>{@code topic:entityName}</li>
  * <li>{@code topic:group}</li>