feat: cluster key rate limit #1002

hanxiantao · 2024-05-26T01:10:01Z

Ⅰ. Describe what this PR did

cluster key rate limit

Ⅱ. Does this pull request fix one issue?

fixes #851

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

1）、识别请求参数 apikey，进行区别限流

wasmplugin.yam：

apiVersion: extensions.higress.io/v1alpha1
kind: WasmPlugin
metadata:
  name: test
  namespace: higress-system
spec:
  defaultConfig:
    rule_name: default_limit_by_param_apikey
    limit_by_param: apikey
    limit_keys:
    - key: 9a342114-ba8a-11ec-b1bf-00163e1250b5
      query_per_minute: 10
    - key: a6a6d7f2-ba8a-11ec-bec2-00163e1250b5
      query_per_hour: 100
    redis:
      service_name: redis
      service_host: redis
      service_port: 80
      service_source: ip
    show_limit_quota_header: true
  url: oci://registry.cn-hangzhou.aliyuncs.com/wasm-plugin/wasm-plugin:0.0.51
  imagePullSecret: aliyun

1）根据第一个apikey进行限流

一分钟内请求三次：

curl -kvv -X GET 'http://localhost:8082/test?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5' \
-H 'Host: www.test.com'

响应头中x-ratelimit-limit为10（限制的总请求数），x-ratelimit-remaining为7（剩余还可以发送的请求数）

触发限流后：

2）根据第二个apikey限流

请求三次：

curl -kvv -X GET 'http://localhost:8082/test?apikey=a6a6d7f2-ba8a-11ec-bec2-00163e1250b5' \
-H 'Host: www.test.com'

redis中数据：

2）、识别请求头 x-ca-key，进行区别限流

wasmplugin.yam：

apiVersion: extensions.higress.io/v1alpha1
kind: WasmPlugin
metadata:
  name: test
  namespace: higress-system
spec:
  defaultConfig:
    rule_name: default_limit_by_param_x-ca-key
    limit_by_header: x-ca-key
    limit_keys:
    - key: 102234
      query_per_minute: 10
    - key: 308239
      query_per_hour: 100
    redis:
      service_name: redis
      service_host: redis
      service_port: 80
      service_source: ip
    show_limit_quota_header: true  
  url: oci://registry.cn-hangzhou.aliyuncs.com/wasm-plugin/wasm-plugin:0.0.51
  imagePullSecret: aliyun

1）根据第一个x-ca-key限流

一分钟内请求三次：

curl -kvv -X GET 'http://localhost:8082/test' \
-H 'Host: www.test.com' \
-H 'x-ca-key: 102234'

响应头中x-ratelimit-limit为10（限制的总请求数），x-ratelimit-remaining为7（剩余还可以发送的请求数）

触发限流后：

3）、根据请求头 x-forwarded-for 获取对端IP，进行区别限流

apiVersion: extensions.higress.io/v1alpha1
kind: WasmPlugin
metadata:
  name: test
  namespace: higress-system
spec:
  defaultConfig:
    limit_by_per_ip: from-header-x-forwarded-for
    rule_name: limit_by_per_ip
    redis:
      service_name: redis
      service_host: redis
      service_port: 80
      service_source: ip
    show_limit_quota_header: true
    limit_keys:
      - key: 1.1.1.1
        query_per_day: 10
      - key: 1.1.1.0/24
        query_per_day: 100
      - key: 0.0.0.0/0
        query_per_day: 1000
  url: oci://registry.cn-hangzhou.aliyuncs.com/wasm-plugin/wasm-plugin:0.0.51
  imagePullSecret: aliyun

1）命中1.1.1.1

请求三次：

curl -kvv -X GET 'http://localhost:8082/test' \
-H 'Host: www.test.com' \
-H 'x-forwarded-for: 1.1.1.1'

2）命中1.1.1.0/24

请求四次：

curl -kvv -X GET 'http://localhost:8082/test' \
-H 'Host: www.test.com' \
-H 'x-forwarded-for: 1.1.1.2'

redis中数据（key名称包含从x-forwarded-for中获取的IP 1.1.1.2）：

Ⅴ. Special notes for reviews

plugins/wasm-go/extensions/key-cluster-rate-limit/README.md

hanxiantao · 2024-06-03T04:23:41Z

@johnlanni 代码已调整，麻烦再帮忙review下

johnlanni · 2024-06-06T02:26:46Z

plugins/wasm-go/extensions/cluster-key-rate-limit/README.md

+
+| 配置项         | 类型   | 必填 | 默认值 | 说明                                            |
+| -------------- | ------ | ---- | ------ | ----------------------------------------------- |
+| service_source | string | 必填 | -      | 类型为固定ip或者DNS，输入redis服务的注册来源    |


https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-cache/main.go

麻烦参考下这里的redis配置解析和client初始化逻辑，使用FQDNCluster，简化用户的配置方式

https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-cache/main.go

麻烦参考下这里的redis配置解析和client初始化逻辑，使用FQDNCluster，简化用户的配置方式

好的，这里我再调整下

johnlanni · 2024-06-06T02:27:01Z

plugins/wasm-go/extensions/cluster-key-rate-limit/README.md

+| service_name   | string | 必填 | -      | 输入redis服务的注册名称                         |
+| service_port   | int    | 必填 | -      | 输入redis服务的服务端口                         |
+| service_host   | string | 必填 | -      | 当类型为固定ip时必须填写，输入redis服务的主机名 |
+| service_domain | string | 必填 | -      | 当类型为DNS时必须填写，输入redis服务的domain    |


其实redis 不需要设置domain

johnlanni · 2024-06-06T02:27:47Z

plugins/wasm-go/extensions/cluster-key-rate-limit/README.md

+rule_name: limit_by_per_ip_from-header-x-forwarded-for
+limit_by_per_ip: from-header-x-forwarded-for
+limit_keys:
+	# 精确ip


这里有一段空白符

johnlanni · 2024-06-06T02:36:21Z

plugins/wasm-go/extensions/cluster-key-rate-limit/README.md

+| 配置项                  | 类型   | 必填 | 默认值 | 说明 |
+| ----------------------- | ------ | ---- | ------ | ---- |
+| rule_name               | string | 是 | - | 限流规则名称，根据限流规则名称和限流的客户端IP段来拼装redis key |
+| limit_by_header         | string          | 否，`limit_by_header`,`limit_by_param`,`limit_by_per_ip` 中选填一项 | -                 | 配置获取限流键值的来源 http 请求头名称                       |


可以考虑加一个 limit_by_consumer, 现在通过 x-mse-consumer 这个request header可以取出 consumer 的名字

可以考虑加一个 limit_by_consumer, 现在通过 x-mse-consumer 这个request header可以取出 consumer 的名字

这个应该是要配合认证的插件一起使用对吧

是的，如果用户开了 key-auth 等认证插件，就可以配合基于 consumer 做流控了，在openapi场景下挺有用的

是的，如果用户开了 key-auth 等认证插件，就可以配合基于 consumer 做流控了，在openapi场景下挺有用的

好的

可以考虑加一个 limit_by_consumer, 现在通过 x-mse-consumer 这个request header可以取出 consumer 的名字

这个目前根据请求头限流的应该可以覆盖这个场景了，limit_by_header直接配置成x-mse-consumer就可以了，如果单独再实现一个limit_by_consumer会有点奇怪，在limit_by_header使用文档中添加下对x-mse-consumer的使用场景的说明会更好些

limit_by_consumer 更好一些，目前通过header传递consumer信息的方式后续可能会调整，有了limit_by_consumer，用户就不感知这个调整

limit_by_consumer 更好一些，目前通过header传递consumer信息的方式后续可能会调整，有了limit_by_consumer，用户就不感知这个调整

好的，limit_by_consumer实现的时候也会和limit_by_per_header、limit_by_per_param一样支持配置一个正则表达式

johnlanni · 2024-06-06T03:06:29Z

plugins/wasm-go/extensions/cluster-key-rate-limit/README.md

+| show_limit_quota_header | bool | 否 | false | 响应头中是否显示`X-RateLimit-Limit`（限制的总请求数）和`X-RateLimit-Remaining`（剩余还可以发送的请求数） |
+| rejected_code           | int | 否 | 429 | 请求被限流时，返回的HTTP状态码 |
+| rejected_msg            | string | 否 | Too many requests | 请求被限流时，返回的响应体 |
+| redis                   | object          | 是                                                           | -                 | redis相关配置                                                |


可以进一步扩展一下，类似 per ip 这种，用于不可枚举场景的限流：
limit_by_per_header
limit_by_per_param

key 可以支持配置一个正则表达式，符合正则匹配的进行限制

可以进一步扩展一下，类似 per ip 这种，用于不可枚举场景的限流： limit_by_per_header limit_by_per_param

key 可以支持配置一个正则表达式，符合正则匹配的进行限制

key以regexp-开头的标识根据正则表达式匹配，示例如下：

rule_name: limit_by_param_x-ca-key limit_by_header: x-ca-key limit_keys: - key: 102234 query_per_second: 10 - key: regexp-\d+ query_per_hour: 10

regexp:\d+

用冒号分隔看上去自然一些

hanxiantao · 2024-06-11T07:33:13Z

plugins/wasm-go/extensions/cluster-key-rate-limit/main.go

+	}
+
+	// 构建redis限流key和参数
+	limitKey := fmt.Sprintf(ClusterRateLimitFormat, config.ruleName, key)


redis key add ruleType

johnlanni

LGTM

feat: cluster rate limit by client ip

0f828be

hanxiantao requested review from johnlanni, WeixinX and CH3CHO as code owners May 26, 2024 01:10

johnlanni requested changes May 27, 2024

View reviewed changes

plugins/wasm-go/extensions/key-cluster-rate-limit/README.md Outdated Show resolved Hide resolved

plugins/wasm-go/extensions/key-cluster-rate-limit/README.md Outdated Show resolved Hide resolved

plugins/wasm-go/extensions/key-cluster-rate-limit/README.md Outdated Show resolved Hide resolved

hanxiantao changed the title ~~feat: cluster rate limit by client ip~~ feat: cluster key rate limit Jun 1, 2024

hanxiantao requested a review from johnlanni June 1, 2024 07:59

hanxiantao force-pushed the key-cluster-rate-limit branch 2 times, most recently from 148a8cb to 83cb0ac Compare June 2, 2024 05:27

feat: cluster key rate limit

6c07b58

hanxiantao force-pushed the key-cluster-rate-limit branch from 156e1db to 6c07b58 Compare June 2, 2024 05:33

Merge branch 'alibaba:main' into key-cluster-rate-limit

55b5c04

Merge branch 'main' into key-cluster-rate-limit

d91c337

johnlanni requested changes Jun 6, 2024

View reviewed changes

johnlanni reviewed Jun 6, 2024

View reviewed changes

simplify the redis configuration method

a335fc6

hanxiantao commented Jun 11, 2024

View reviewed changes

johnlanni approved these changes Jun 12, 2024

View reviewed changes

johnlanni merged commit 12cc44b into alibaba:main Jun 12, 2024
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: cluster key rate limit #1002

feat: cluster key rate limit #1002

hanxiantao commented May 26, 2024 •

edited

Loading

hanxiantao commented Jun 3, 2024

johnlanni Jun 6, 2024

hanxiantao Jun 6, 2024

johnlanni Jun 6, 2024

johnlanni Jun 6, 2024

johnlanni Jun 6, 2024

hanxiantao Jun 6, 2024

johnlanni Jun 6, 2024

hanxiantao Jun 6, 2024

hanxiantao Jun 6, 2024

johnlanni Jun 11, 2024

hanxiantao Jun 11, 2024

johnlanni Jun 6, 2024

hanxiantao Jun 6, 2024 •

edited

Loading

johnlanni Jun 11, 2024

hanxiantao Jun 11, 2024

johnlanni left a comment

feat: cluster key rate limit #1002

feat: cluster key rate limit #1002

Conversation

hanxiantao commented May 26, 2024 • edited Loading

Ⅰ. Describe what this PR did

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

hanxiantao commented Jun 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hanxiantao Jun 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johnlanni left a comment

Choose a reason for hiding this comment

hanxiantao commented May 26, 2024 •

edited

Loading

hanxiantao Jun 6, 2024 •

edited

Loading