Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature]Adaptive Flow Control with Q-Learning algorithm #1686

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

mastertiller
Copy link

@mastertiller mastertiller commented Aug 20, 2020

Describe what this PR does / why we need it

在服务数量多,拓扑复杂,处理能力逐渐变化的情况下,使用固定的最大并发会带来巨大的测试工作量,并且用户需要手动地设置很多流控规则。自适应限流就是为了解决这个问题。我们希望使用Q-Learning算法,引入智能的自适应流控策略进行限流,在最大化吞吐量的同时保障系统服务的稳定。相关issue可见#748 #1641

Does this pull request fix one issue?

#748

Describe how you did it

使用强化学习Q-Learning算法实现自适应流控,算法详细介绍见 结项报告#1641
算法流程图如下:

image

QLearningLearner.java实现了算法的迭代更新。
QLearningMetric.java实现了算法的主要方法与参数设置。
QTableStorage.java实现QTable的读写与存储。
QInfo.java存储了决策前状态(state)、行为(action)、效用(utility)的信息。

Describe how to verify it

sentinel-demo/sentinel-demo-basic/src/main/java/com/alibaba/csp/sentinel/demo/qlearning/UserPeakCpuDemo.java 实现了简单的峰值流量场景测试。
测试时,isTraining为false,不更新QTable。
训练时,将isTraining设为true,可以更新QTable。
启用QLearning的开关为isQLearning,设置为false时,不使用QLearning算法进行自适应流控。

Special notes for reviews

@sczyh30 sczyh30 added the area/adaptive-traffic-shaping Issues or PRs related to adaptive traffic shaping label Aug 21, 2020
@sczyh30 sczyh30 added to-review To review size/XXL Indicate a PR that changes 1000+ lines. labels Sep 1, 2020
@sczyh30 sczyh30 added this to the 2.0.0 milestone Dec 9, 2020
@sczyh30
Copy link
Member

sczyh30 commented Dec 9, 2020

This will be resolved after #1889 has been done.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/adaptive-traffic-shaping Issues or PRs related to adaptive traffic shaping size/XXL Indicate a PR that changes 1000+ lines. to-review To review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants