Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/mqtt connectivity #225

Merged
merged 66 commits into from
Sep 5, 2018
Merged

Feature/mqtt connectivity #225

merged 66 commits into from
Sep 5, 2018

Conversation

yufei-cai
Copy link
Contributor

Add some MQTT support #220 using Alpakka.

Changes:

  • Replaced custom regex for connection URI by standard java.net.URI to support URI schemes used by Alpakka to determine the transport protocol.
  • Added connection type mqtt and the client actor, publisher, subscriber and connection factory for it.
  • Split Source and Target of connections into protocol-specific implimentations for configurations.
  • Added Quality of Service configuration for Source and Target of MQTT connections.
  • Added topic filters to Source of MQTT connections to check consistency between MQTT topics and Ditto protocol topics. The check is implemented by ThingIdEnforcement.
  • ConnectionActor now reads configuration from a ConnectionConfigReader instead of directly from the actor system config so that it needn't know all the protocol-specific config keys.

Authentication is unchanged. For Amazon IoT integration #220 we still need certificate-based authentication.

Binary interface breakage since release 0.8.0-M1:

  • Removed Connection.UriRegex in favor of java.net.URI
  • Removed methods newSource, sourceFromJson, targetFromJson from ConnectivityModelFactory
  • Added method getThingIdEnforcement to interface ExternalMessage
  • Added method withThingIdEnforcement to interface ExternalMessageBuilder
  • Added method withAddress to interface Target

dguggemos and others added 30 commits August 6, 2018 11:20
@dguggemos
initial version of mqtt connectivity
857d4e1 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@yufei-cai
add mqtt[s] protocol to connections model; remove [:] from username r…
a338bfd 
…egex

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@dguggemos
Merge branch 'feature/mqtt-connectivity'
161c4a4 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@dguggemos
fix compile error
150d87f 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@dguggemos
read provided username/password from configuration and apply to mqtt …
4d81373 
…connection

Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@yufei-cai
add protocol-specific config to connection source & target
e304077 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
Connection: remove URI regexes & permit all valid URI at construction…
7e2e2dd 
… time

Protocol-specific URI validations should be performed elsewhere.

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@dguggemos
respond to metrics requests in mqtt actors
04c05b8 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@dguggemos
fix unit tests after merge
96ee3a3 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@dguggemos
reduce unit test execution time
76d44dd 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@dguggemos
avoid actor name conflicts when creating consumer/publisher actors in…
403edce 
… MqttClientActor

Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@yufei-cai
move URI scheme validation to protocol level
32881a6 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@dguggemos
initialize mqtt connection with default ssl context
e505fb8 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@yufei-cai
validate specific configs of MQTT sources and targets
e505fb5 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
translate SocketFactoryExtension from scala to java
673cc51 
Reason: It confused Maven.

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@dguggemos
use ssl context only for secure connections
d535ab4 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@dguggemos
- move creation of MqttConnectionSettings to factory
102abde 
- add trust manager that accepts any certificate for testing purposes
- send OpenConnection instead CreateConnection on recovery

Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@dguggemos
Merge branch 'master' into feature/mqtt-connectivity
cf4e061
@dguggemos
fix initialization of custom ssl context
60b50ab 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@yufei-cai
use async client in MQTT connection test to not block actor; use open…
08558e7 
… port in testTestConnectionFails

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
increment message counters in MqttClientActor thread
32a4dfd 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
adjust formatting; enter moquette version in ditto-bom
af5f395 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@dguggemos
fix some sonar findings
7f0ea80 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@yufei-cai
delete unused Connection.UriRegex
88fc10c 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@dguggemos
validate configured mqtt topics
c8a25de 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@yufei-cai
split protocol-specific source & target validation; configure Mqtt co…
af8e90f 
…nsumers by QoS of sources

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
set QoS in MqttPublisherActor & start streams as children in MqttClie…
49fde3f 
…ntActor

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
set QoS of MQTT-Subscribe messages by source config
4416559 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
move MQTT sink stream to MqttPublisherActor; delay ClientConnected un…
4ba67a4 
…til publisher stream is ready

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
MqttClientActor: add javadoc & start 1 stream per source instead of p…
68af041 
…er consumer

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
yufei-cai and others added 27 commits August 17, 2018 18:03
@yufei-cai
Merge branch 'master' into feature/mqtt-connectivity
4cfff81 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
fix ImmutableExternalMessageTest
6aa39a2 
- fixed 'testHashCodeAndEquals'
- ignored 'assertImmutability' until it is fixed

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
fix incorrect I.MqttSource constructor call due to overloading
8b62b8b 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
disable binary compatibility check for now
a8fbab5 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
rename ImmutableExternalMessage to UnmodifiableE.M. and document corr…
5155f68 
…ect usage

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
ReconnectActor: stop pinging corrupted & nonexistent connections
6572a20 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
fix unintended disconnect of freshly created connections
9668adf 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
give MQTT source default QOS=0 for convenience; improve comment
e19d522 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
unify connection establishment and testing; removed 1 deadletter from…
91fd517 
… ConnectionActor

ConnectionActor
- Cancel STOP_SELF_IF_DELETED post-stop

BaseClientActor
- Accept only Status.Status as test connection result if it
  is sent from self

Mqtt
- Add dryRun mode to publisher and subscriber actors
- Instead of starting a new Paho client, start publishers and
  subscribers in dryRun mode to test the connection. Thus is
  test result consistent with normal operation.

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
add Target.withAddress; remove unnecessary collection of responses in…
6ddfae8 
… ConnectionActor

Instead of collecting pending responses and flush them in one go,
schedule the responses with fixed delay.

Reason: Sending CreateConnectionResponse on arrival of first
event is not correct because pub/sub may not be distributed
on all cluster nodes. Thus all responses are sent after fixed
delay, obviating their retention outside the scheduler.

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
MQTT: remove moquette dependency & dead code; increment message count…
7b9228d 
…ers regardless of state

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
validate MQTT source topic filters
a4797b1 
Also verified that Alpakka sources work with QoS 2.

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
MQTT: fix TODOs
191314b 
- Source buffer size is configured via ConnectionConfigReader.mqtt().

- Source is wrapped in RestartSource when failover is enabled.

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
MQTT: remove RestartSource because auto-reconnect of Paho should be s…
e5a8280 
…ufficient

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
fix connectivity.conf
81e938d 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
Merge branch 'master' into feature/mqtt-connectivity
81c6843 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
document MQTT 3.1.1 connections
7ff3f03 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
set default qos of MQTT source to 2 so that Ditto get all messages
810fc1f 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@marianne-klein
reviewed Ditto MQTT docu
9c5da12 
Signed-off-by: marianne-klein <marianne.klein@bosch-si.com>
@yufei-cai
Merge branches 'feature/mqtt-connectivity' and 'master'
f2d73c4 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
fix timeout retrieving metrics of connections with empty source or ta…
d024f03 
…rget addresses

- Immutable{Address|Source|Target}Metrics: tolerate empty Json key during serialization
- BaseClientActor: do not read message counters in future.
- ConnectionValidator: reject connections with empty source or target addresses

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
fix MqttClientActorTest
7bc1b5e 
testRetrieveConnectionMetrics did not send OpenConnection.
Don't know why it wasn't failing.

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
improve error message of topic filter mismatch; do not disclose topic…
3d874d9 
… filters; add javadoc

Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
fix jekyll errors
4a2033f 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai
document that events are not published on origin connection
e31c041 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@dguggemos
added some tests for model classes
aea7422 
Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch-si.com>
@yufei-cai
improve description of no-same-origin policy
d5c04f1 
Signed-off-by: Cai Yufei (INST/ECS1) <yufei.cai@bosch-si.com>
@yufei-cai yufei-cai merged commit 9d9e763 into eclipse-ditto:master Sep 5, 2018
@yufei-cai yufei-cai deleted the feature/mqtt-connectivity branch September 5, 2018 10:20
@thjaeckle thjaeckle added this to the 0.8.0-M2 milestone Sep 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.8.0-M2
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@yufei-cai @thjaeckle @dguggemos @marianne-klein