rate limiter as middleware

[alperhankendi]

rate limiter middleware usage;
The default config;
the same request is limited with 100 in a minute.

ps: Distributed RateLimiter implementation is in progress. I'm thinking about to use Redis for it.

e := echo.New()
e.GET("/", func(c echo.Context) error {
    return c.String(http.StatusOK, "Hello, World!")
})
//e.Use(middleware.RateLimiter())
e.Use(middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
    Max:2,
    Duration:time.Minute*1,
    Prefix:"test",
}))
e.Logger.Fatal(e.Start(":1323"))

for 200-response these fields will be added to the response header.

x-ratelimit-limit →2
x-ratelimit-remaining →1
x-ratelimit-reset →1559074005

for 429 - to many request code these fields will be added to the response header.

retry-after →4
x-ratelimit-limit →2
x-ratelimit-remaining →-1
x-ratelimit-reset →1559074005

TODO:

• Distributed rate limiter
• IP, Header, Method and cookie-based limitation.
• Extensible rate limit policies

[codecov]

Codecov Report

Merging #1343 into master will increase coverage by 0.63%.
The diff coverage is 92.92%.

@@            Coverage Diff             @@
##           master    #1343      +/-   ##
==========================================
+ Coverage   84.39%   85.03%   +0.63%     
==========================================
  Files          27       28       +1     
  Lines        2019     2305     +286     
==========================================
+ Hits         1704     1960     +256     
- Misses        205      225      +20     
- Partials      110      120      +10

Impacted Files	Coverage Δ
middleware/ratelimiter.go	92.92% <92.92%> (ø)
middleware/jwt.go	76.84% <0%> (-3.89%)	⬇️
middleware/key_auth.go	67.79% <0%> (-3.64%)	⬇️
bind.go	87.09% <0%> (-1.93%)	⬇️
router.go	94.61% <0%> (-0.65%)	⬇️
echo.go	85.34% <0%> (-0.28%)	⬇️
middleware/proxy_1_11.go	100% <0%> (ø)	⬆️
context.go	91.12% <0%> (+0.41%)	⬆️
middleware/secure.go	93.75% <0%> (+0.41%)	⬆️
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fbb7286...72c3e32. Read the comment docs.

[alperhankendi]

@vishr if u review the code, I'll continue for distributed rate limiter feature. I'm thinking distributed rate limiter with Redis or Memcached it will be configurable and don't want to depend on it directly. Do u have any comment about it?

[vishr]

@alperhankendi I definitely want it. However, couple of points:

• We want a solution that learns from existing and provides the best. For this we should have comparison. Programming language should not be a barrier.
• We want to minimize external dependencies, I believe using an interface you can get rid of it.

[alperhankendi]

Distributed rate limiter has been added. @vishr According to your comment the middleware is just know the RedisClient interface and the client imp. will be injected outside the Echo.

	RedisClient interface {
		DeleteKey(string) error
		EvalulateSha(string, []string, ...interface{}) (interface{}, error)
		LuaScriptLoad(string) (string, error)
	}

middleware usage;

import (
	"github.com/labstack/echo"
	"github.com/labstack/echo/middleware"
	"net/http"
	"time"
	"gopkg.in/redis.v5"
)

type redisClient struct {
	*redis.Client
}

func (c *redisClient) DeleteKey(key string) error {
	return c.Del(key).Err()
}
func (c *redisClient) EvalulateSha(sha1 string, keys []string, args ...interface{}) (interface{}, error) {
	return c.EvalSha(sha1, keys, args...).Result()
}
func (c *redisClient) LuaScriptLoad(script string) (string, error) {
	return c.ScriptLoad(middleware.LuaScriptForRedis).Result()
}
func main() {

	client := redis.NewClient(&redis.Options{
		Addr: "localhost:6379",
	})
	e := echo.New()
	e.GET("/", func(c echo.Context) error {
		return c.String(http.StatusOK, "Hello, World!")
	})
	e.Use(middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
		Max:100,
		Duration:time.Minute*1,
		Prefix:"keyPrefix_",
		Client: &redisClient{client},
		SkipRateLimiterInternalError:false,
	}))
	e.Logger.Fatal(e.Start(":1323"))
}

[nghtstr]

Out of curiosity @vishr, why hasn't this been merged to the main branch yet? This looks rather solid.

[alperhankendi]

I have changed the limitation logic. now there are Strategy options. you can limit your API based on client-ip or header key.

middleware using;

e.Use(middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
		LimitConfig:middleware.LimiterConfig{
			Max:100,
			Duration:time.Minute*1,
			Strategy:"header",
			Key:"Client-Token",
			//or 
			//Strategy:"ip",
		},
		Prefix:"keyPrefix_",
		SkipRateLimiterInternalError:false,
	}))

TODO:

• Distributed rate limiter
• IP, Header limitation.

@vishr it seems unit test coverage failed. I have tried to pump the coverage rate. rate_limiter.go file coverage is about %92 waiting for your comment

[dilshat]

Looking forward for this feature

[jotto]

To support the use-case of creating multiple rate limiters (1 for minute, hour, day, etc...) can we move this from global to instance based?

[alperhankendi]

imo it makes thinks complex. What you have in your mind, can you give more specific example?

[alperhankendi]

this is a way to support multiple rate limiters.

rateLimiter1:=middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
		LimitConfig: middleware.LimiterConfig{
			Max:                         2,
			Strategy:                    "header",
			Duration:                    time.Minute,
			Key:                         "test",
			HeaderTokenExtractorHandler: func(headerValue string) string {
				return headerValue
			},
		},
	})
	rateLimiter2:=middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
		LimitConfig: middleware.LimiterConfig{
			Max:                         2,
			Strategy:                    "ip",
			Duration:                    time.Minute,
		},})
	
	e.GET("/", func(c echo.Context) error {
		return c.String(http.StatusOK, "Hello, World!")
	},rateLimiter1)

	e.GET("/status", func(c echo.Context) error {
		return c.String(http.StatusOK, "Hello, World!")
	},rateLimiter2)

[jotto]

Are you sure? I think in your example here, rateLimiter2's config would overwrite rateLimiter1, so yes, you'd have 2 rate limiters, but both would be using the same limiterImp

[jotto]

can you give more specific example

The most specific example I can think of is:

e.GET("/", func(c echo.Context) error {
    return c.String(http.StatusOK, "Hello, World!")
}, rateLimiter1Min, rateLimiter60Min, rateLimiter1440Min)

For example, we could set 10,000 requests max per day, but we don't want all 10,000 to happen in the first minute, so the minute rate limiter could be 60.

[jotto]

What I've presented above, is simple/clean, and I've seen in some node.js libraries, but there's another approach that's even more interesting.

Dynamic rate limits per request.

For example:

e.GET("/", func(c echo.Context) error {
    return c.String(http.StatusOK, "Hello, World!")
}, createRateLimiterMiddleware(func(c *echo.Context) ([]middleware.RateLimiterConfig, error) {
  user := extractUse(c)
  path := extractPath(c)

  if user.role == "premium" {
    return []middleware.RateLimiterConfig{middleware.RateLimiterConfig{Max: 1000, Duration: time.Hour}}, nil
  }

  return defaultRateLimiterConfigs(), nil
}))

[alperhankendi]

Are you sure? I think in your example here, rateLimiter2's config would overwrite rateLimiter1, so yes, you'd have 2 rate limiters, but both would be using the same limiterImp

you're right. there is only one limiterimp so first registered middleware is working and rest of the limiters will be ignored as you said it should work with many limiters so Im gonna change it to support multiple limiters. many thanks

[jotto]

If you decide to support multiple instances, these header names could be modified like:

fmt.Sprintf("X-Ratelimit-Remaining-%s", config.Prefix)

[alperhankendi]

A prefix is used for the cache keys. We may use hostName instead of prefix.
something like;

hostName,_ :=  os.HostName()
fmt.Sprintf("X-Ratelimit-Remaining-%s", hostName)

or
fmt.Sprintf("X-Ratelimit-Host-%s", hostName)
I think new header is more proper.

[ioggstream]

It would be great if this patch implements the ratelimit I-D here https://tools.ietf.org/html/draft-polli-ratelimit-headers-01.

You just need to return a delta-seconds instead of a unix-timestamp. See comments for further infos.

[alperhankendi]

thanks @jotto for the valuation feedback. can check the new commit

[ioggstream]

Hi @alperhankendi, it would be great if this middleware aligns to the new ratelimit standardization proposal.

The proposal was:

• recently presented at the HTTP Workgroup during the IETF Meeting
• implemented by Kong, EnvoyProxy, Redhat 3scale
• configurable in express-gateway, azure gateway

The proposal

Very similar to your current implementation, but header names have no X- and Reset is in delta-seconds instead of timestamp. The rationale for the choice is in FAQ 5.

RateLimit-Limit: 60
RateLimit-Remaining: 50
RateLimit-Reset: 5

Moreover, you're welcome to contribute to the spec and add your implementation here ioggstream/draft-polli-ratelimit-headers#1

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[amegianeg]

Hi there! what is the status of this implementation? @alperhankendi are you going to adapt this PR to #1343 (comment) ?

Thanks a lot!

[tanvir-retailai]

HI,

Any chance to get this feature soon?

[Demetri0]

HI,

Any chance to get this feature soon?

Look into github.com/didip/tollbooth maybe it will be good for you.

[tanvir-retailai]

Thank you @Demetri0

[shrivastavshubham34]

When will this be pushed as a feature?

[Demetri0]

When will this be pushed as a feature?

why do you think it should?

[shrivastavshubham34]

@jotto @alperhankendi
please let us know, we can continue with something else meanwhile, thanks!