Durg | Kirtimukh

The Guardian of Thresholds

Introduction

As a part of constant efforts towards Stability & Reliability, what every Software Engineer look for is High Precisions & Predictability. Especially when margin for errors & recovery gets tinier.

Why does Fault Tolerance/Resiliency matter?

Imagine...

• You are in an overloaded elevator, which keeps returning to the ground floor in an honest attempt to lift entire load
• Everytime you turned on Air Conditioner & Television, it results in power-cut in your entire area
• A Civil Engineer receiving a call in the middle of the night that flood gates have opened unexpectedly
• You are mid-air and flight's landing gears appear to have jammed because all passengers are watching movies

As an engineer, we all do understand that systems can give-up or fail anytime and they might need regular servicing/tuning.

However the experience is frustrating as well as scary at the same time, isn't it?

Resiliency & Microservices

Resiliency is the key thing to look after in Microservices Architecture.

Circuit Breaker

One of the most important or moreover the critical/key aspects of Microservices Architecture is fault tolerance using Circuit Breakers. One may choose between FailFast or FailSafe approaches, purely depends on the use-cases. While the FailFast approach effectively provides fast failing, faster recovery & effectively better utilisation of resources, FailSafe approach tolerates fluctuations.

Throttling

While most of the Architectures rely completely on Circuit Breaker Pattern & Retries, which gives the complete control of The Stability to the client side. They do perform well initially in the small setup or closely working small number of microservices. As organisations grow bigger & bigger, clients may not have fine-tuned parameters, resulting in creating back-pressures.

And then comes those completely eventful moments when in the event of _Outages due to Back

Pressure_, we end up searching for which client caused this.

Throttling essentially provides a mechanism for the flow control in the event of unexpected increase in incoming traffic or increase in latencies from downstream.

About Durg | Kirtimukh

Durg means Fortress or something difficult to conquer.

About Kirtimukh

Kirtimukh is the guardian of the thresholds.

Current Version

How to use throttling?

A Throttling bundle for DropWizard.

Rate-Limiting/Throttling Strategies

All strategies can be applied on one single command or bucket of commands. This enables us bucketizing similar API calls which are being served under different versions. This also helps in providing a mechanism to traffic structuring on different windows.

Quota based

Quota based rate-limiting can be used where one needs to restrict number of request volumes based timed windows e.g. a second, a minute, 5 seconds or 10 minutes.

Leaky Bucket

Wish to serve high requests when systems are healthy but throttle when downstream becomes latent or unhealthy, this strategy shall help. This is the preferred strategy for rate-limiting as well as traffic shaping/flow control if downstream is latent due to external factors which are not in our control.

Priority Bucket

Unsure about traffic patterns but want to give preference to certain commands/buckets, in an unexpected event or things start falling apart, Priority Bucket is the right thing to do.

Custom Controller

Define your own Centralised or Decentralised way to handle rate-limiting.

How to integrate?

Maven

<dependency>
    <groupId>io.durg.kirtimukh.dw</groupId>
    <artifactId>throttling-bundle</artifactId>
    <version>0.0.4</version>
</dependency>

Gradle

compile 'io.durg.kirtimukh.dw:throttling-bundle:0.0.4'

Sample bundle initialisation

    bootstrap.addBundle(new ThrottlingBundle<ApplicationConfiguration, ThrottlingKeyType>() {
        @Override
        protected ThrottlingBundleConfiguration getThrottlingConfiguration(ApplicationConfiguration appConfig) {
            return appConfig.getThrottlingConfig();
        }

        @Override
        protected CustomThrottlingController<ThrottlingKeyType> getCustomController() {
            // Introduce custom controller logic here
            return null;
        }

        @Override
        protected ThrottlingExceptionTranslator<ApplicationException> getExceptionTranslator() {
            return new ThrottlingExceptionTranslator<ApplicationException>() {
                @Override
                public ApplicationException throwable(ThrottlingException e) {
                    return new ApplicationException(ResponseCode.TOO_MANY_REQUESTS, "Too Many Requests");
                }
            };
        }
    });

Sample config

throttlingConfig:
  defaultStrategyConfig:
    type: LEAKY_BUCKET
    threshold: 64
  commandStrategyConfigs:
    EventIngestor.publish:
      type: LEAKY_BUCKET
      threshold: 16
    STATUS_APIS_BUCKET:
      type: QUOTA
      unit: SECOND
      threshold: 64

Sample code

// Single API level throttling
@Throttle
public Response publish(EventPublishRequest request)

// API Bucket level throttling
@Throttle(bucket = STATUS_APIS_BUCKET)
public Response statusV1(StatusRequest request)

@Throttle(bucket = STATUS_APIS_BUCKET)
public Response statusV2(StatusRequest request)

@Throttle(bucket = STATUS_APIS_BUCKET)
public Response statusV3(StatusRequest request)

License

This project has been released under an Apache License v2.

ASF v2.0

 Copyright (c) 2020 Pradeep A. Dalvi <prad@apache.org>

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.