Switch branches/tags
Nothing to show
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


Flowing Retail / REST

This folder contains services that connect via REST. Currently this is reduced to showcasing resilience patterns.

Sample service demonstrating stateful resilience patterns in a REST environment

This sample (micro-)service can retrieve payments and therefor needs to be called via REST. It requires an upstream REST service to charge credit cards.

REST callstack

This simple call-chain is great to demonstrate various important resilience patterns.

There are the following technology choices available for the code demos:

  • Java + Spring, Hystrix, Camunda
  • C# + Polly, Camunda

There is a stripped down version available for:


See Fail fast is not enough: https://blog.bernd-ruecker.com/fail-fast-is-not-enough-84645d6864d3

Assume the credit card service goes nuts, meaning it still responds, but very slow. Having no pattern in place this is the worst thing that can happen - as now the payment service will call the credit card service and block until he gets a response. As this take a long time all threads from the payment service are hold hostile and the payment service will eventually time out for its clients. Tiny failures somewhere in your system might blow up your whole system:


Fail fast

The least you have to do is to apply a fail fast pattern like circuit breaker. In this example I use Netflix Hystrix. If a service responds to slow the circuit breaker interrupts and the payment service gets a failure right away. This way you make sure the overall system is still responding, even if functionality degrades (I cannot charge credit cards).


Fail fast is not enough

But failing fast is not enough. Very often a retry after the credit card service has been fixed resolves the situation. This retry needs to be stateful to not only retry right away but again in a couple of minutes, hours or even days. Having this stateful retrying as possibility keeps the failure handling local to payment making the whole architecture less complex.


In the example I use the Camunda workflow engine to do the stateful retry reliably.

Keep synchronous responses

The processing just got asynchronous, which is often not wanted. In this scenario you could very well return a synchronous response whenever the credit card service is available, but switch to asynchronicity only if it is not.


HTTP supports this by differntiating the return code (200 OK means all OK, 202 ACCEPTED means I call you back later).

Sync vs. async

Asynchronous work distribution without messaging

An alternative to synchronously call an upstream service is to communicate asynchronously. The default would be messaging.

In this example I show one alternative which a lot of customers use very sucessfully: Using the workflow engine as work distribution, behaving like a queue. Therefor I leveraged a concept called External Tasks.


Business transactions using compensation

The last part of the example adds compensation to the game. In distributed systems ACID transactions are not applicable (or at least do not scale well). Using compensation is the alternative - meaning that you reliably undo already executed work if something later on fails.


See payment6.bpmn / Java and PaymentV6.bpmn / C# for the workflow

How-to run

You have to startup both services:

  • Stripe Fake Server
  • Payment Service

This varies:

Now the different versions of the payment service are available:

You now can issue a PUT with an empty body:

curl \
-H "Content-Type: application/json" \
-X PUT \
-d '{}' \

Hint on using Camunda Enterprise Edition

For Camunda there is an enterprise edition available with [https://camunda.com/products/cockpit/#/features](additional features in Cockpit) (the monitoring tool). It is quite handy to use this when playing around with the example. You can easily switch to use enterprise edition:

Note that you do not need the enterprise edition to run the examples, the community edition will also do fine. But because of less features you do not see historical workflow instances - and that means you do not see that much in Camunda Cockpit if everything runs smoothly.