[feature]Adaptive Flow Control with Q-Learning algorithm #1686
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe what this PR does / why we need it
在服务数量多,拓扑复杂,处理能力逐渐变化的情况下,使用固定的最大并发会带来巨大的测试工作量,并且用户需要手动地设置很多流控规则。自适应限流就是为了解决这个问题。我们希望使用Q-Learning算法,引入智能的自适应流控策略进行限流,在最大化吞吐量的同时保障系统服务的稳定。相关issue可见#748 #1641
Does this pull request fix one issue?
#748
Describe how you did it
使用强化学习Q-Learning算法实现自适应流控,算法详细介绍见 结项报告 ,#1641
算法流程图如下:
QLearningLearner.java实现了算法的迭代更新。
QLearningMetric.java实现了算法的主要方法与参数设置。
QTableStorage.java实现QTable的读写与存储。
QInfo.java存储了决策前状态(state)、行为(action)、效用(utility)的信息。
Describe how to verify it
sentinel-demo/sentinel-demo-basic/src/main/java/com/alibaba/csp/sentinel/demo/qlearning/UserPeakCpuDemo.java 实现了简单的峰值流量场景测试。
测试时,
isTraining
为false,不更新QTable。训练时,将
isTraining
设为true,可以更新QTable。启用QLearning的开关为
isQLearning
,设置为false时,不使用QLearning算法进行自适应流控。Special notes for reviews