Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic end-to-end acknowledgement handling for connection sources and targets #661

Closed
thjaeckle opened this issue Apr 27, 2020 · 0 comments · Fixed by #737
Closed

Automatic end-to-end acknowledgement handling for connection sources and targets #661

thjaeckle opened this issue Apr 27, 2020 · 0 comments · Fixed by #737
Milestone

Comments

@thjaeckle
Copy link
Member

Building on the work on #611 the in Ditto managed connections shall automatically

  • request acknowledgements when configured in sources
  • delivering acknowledgements when configured in targets

Connection sources get an additional "required-acks" configuration.
Connection targets get an additional "delivered-acks" configuration.

For consuming messages via sources, the technical acknowledgement/settlement of the underlying protocols (AMQP 1.0, AMQP 0.9.1, MQTT 3, MQTT 5, Apache Kafka, HTTP) shall be applied accordingly:

  • only technically ack/settle a message as "successfully processed" when all configured "required-acks" were successful
  • either issue a technical nack or timeout for unsuccessful processings
  • protocols shall redeliver the message when the message was sent with a certain QoS (e.g. "at least once")

For publishing messages via targets, wait for the technical confirmation of the client in order to automatically create an acknowledgement from it.
Reasonably fill the "status", "payload" and "headers" of the automatically created acknowledgement, e.g. for HTTP connections, the acknowledgement shall contain the HTTP status code as "status" and the response payload as "payload", also the HTTP response headers as "headers".

@thjaeckle thjaeckle modified the milestones: 1.1.0, 1.2.0 Apr 27, 2020
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…AMQP, MQTT, HTTP.

TODO: Add this method for Kafka, RabbitMQ.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
… return futures with pubblishMessage.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…a client producer.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…model to support e2e ack for connections

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…ments to n<=1

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…sher actors for auto acknowledgements.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…essage-acks in MessageMappingProcessorActor. Changed props of MessageMappingProcessorActor to access connection sources.

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Fixed an exception when receiving a thing response without correlation ID.

Fixed the lack of concurrency between processing of different messages.

Fixed removal of Ditto internal headers from ExternalMessage.getInternalHeaders().

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
… deserialization.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
… back to Gateway.

Also fix a logging bug in AbstractRoute.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…nsumerActor.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
… to emit acknowledgements

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…or; fix and enable RabbitMQPublisherActorTest.testAutoAck().

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…herActor.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…blisherActor

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…k in AmqpPublisherActor

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
TODO: negative acks + tests.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
… 5 PublisherActor

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…nual ack; add TODOs.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…erActor for support of connection-acks

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…eaders to assure processing idependent from header-mapping set to connection source

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…ements for managed Ditto connections

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…eption retriability; fix TODOs.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…e precise.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
* adjusted some javadoc
* minor code (mainly style) adjustments

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
* changed default MQTT headerMapping behavior:
** when source has no header mapping defined, map the 3 MQTT specific headers "mqtt.topic", "mqtt.qos" and "mqtt.retain"
** when source has a header mapping defined, don't map the 3 MQTT specific headers by default but take the source header mapping

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…iver" condition; improve comment.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…heading

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…into a config.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
… on sources FilteredAcknowledgementRequest.filter to MessageMappingProcessorActor

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…out mapping multiple commands with different requested-acks

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…ture for re-assigning

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…y renaming headings

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…which do not go through the client actor.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…lize responseType as lowercase in JSON

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk pushed a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…y-mode" for MQTT3

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…edged messages

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 20, 2020
…h-messages

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
thjaeckle added a commit that referenced this issue Jul 21, 2020
#661 End to end acknowledgement for connections
DerSchwilk added a commit to bosch-io/ditto that referenced this issue Jul 22, 2020
Signed-off-by: David Schwilk <david.schwilk@bosch.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant