FINERACT-1795: Improve resilience of command processing service

[vidakovic]

Improve the resilience of the command processing service and make retry (and other parameters) configurable (provide reasonable defaults).

The current retry mechanics in place are causing various congestion issues in high traffic deployments. Besides that, we always have to lookup the tenant to retrieve the retry configuration; Spring Boot's application.properties are the proper place for that.

Retries are used currently with 4 service function calls and the code used to implement retries is very repetitive and not very adjustable.

[vidakovic]

I've investigated that failure a bit and can't make any sense of it. The server side exception mapping seems to be OK as far as I can tell (investigated all execution paths). Nevertheless, on the integration test we get a HTTP 204 instead of 404. As far as I can see this is not related to my PR (made no changes in the area where things are breaking).

[vidakovic]

I think I found it... tricky. Needed to re-throw exception in logCommand fallback method; it was swallowing the exceptions and that resulted utlimately in the HTTP 204 code.

[vidakovic]

Adjusting the tests took more time than should be necessary... the whole tenant based database setup is impossible to mock for testing... there are just to many factories inside of factories calling static functions in some obscure utility classes.
Found a solution for now and left a comment.

[vidakovic]

Alright... I think that's it. Removed some overly complex error handling for some retry scenarios and fixed a hidden issue in one of the retry fallback functions (missing return type leads to wrong method signature).

[vidakovic]

Added documentation and more cleanups.