feat:separation of fast and slow commands #2162

dingxiaoshuai123 · 2023-11-24T04:55:32Z

issue : #2085
issue : #2174

1 separation of fast and slow commands
2 modify the codis proxy & pika
3 add slow command queue and slow command thread pool in pika

codis/config/proxy.toml

dingxiaoshuai123 · 2023-11-24T09:43:04Z

codis/pkg/proxy/backend.go

@@ -491,18 +507,23 @@ func (s *sharedBackendConn) BackendConn(database int32, seed uint, must bool) *B
 type sharedBackendConnPool struct {
 	config   *Config
 	parallel int
+	quick    int


补充注释

dingxiaoshuai123 · 2023-11-24T09:43:44Z

codis/pkg/proxy/config.go

-backend_primary_parallel = 1
-backend_replica_parallel = 1
+backend_primary_parallel = 2
+backend_primary_quick = 1


补充注释

codis/pkg/proxy/config.go

dingxiaoshuai123 · 2023-11-24T09:47:06Z

codis/pkg/proxy/mapper.go

+		opTable[r.Name] = r
+	}
+
+	if cmdlist == "" {


调用len判断字符串为空

codis/pkg/proxy/router.go

dingxiaoshuai123 · 2023-11-24T09:53:12Z

conf/pika.conf

@@ -23,6 +23,13 @@ thread-num : 1
 # are dedicated to handling user requests.
 thread-pool-size : 12

+# Size of the low level thread pool, The threads within this pool
+# are dedicated to handling slow user requests.
+low-level-thread-pool-size : 4


改为slowcmdthreadpool

conf/pika.conf

include/pika_conf.h

dingxiaoshuai123 · 2023-11-24T09:59:29Z

src/pika_admin.cc

+  } else if (set_item == "slow-cmd-list") {
+    g_pika_conf->SetSlowCmd(value);
+    ret = "+OK\r\n";
+  }else if (set_item == "max-cache-files") {


src/pika_client_conn.cc

AlexStocks · 2023-11-24T12:53:39Z

todo:
1 seperate the read and write command;
2 elastic the slow command thread pool

AlexStocks · 2023-11-29T04:21:54Z

codis/pkg/proxy/mapper.go

@@ -6,8 +6,10 @@ package proxy
 import (
 	"bytes"
 	"hash/crc32"
+	"pika/codis/v2/pkg/utils/log"


重置下位置

include/pika_conf.h

src/pika_server.cc

codis/pkg/proxy/backend.go

include/pika_conf.h

Mixficsol · 2023-12-04T06:19:38Z

src/pstd/include/pstd_string.h

@@ -50,6 +51,8 @@ int string2int(const char* s, size_t slen, unsigned long* lval);
 int d2string(char* buf, size_t len, double value);
 int string2d(const char* s, size_t slen, double* dval);
 std::vector<std::string>& StringSplit(const std::string& s, char delim, std::vector<std::string>& elems);
+void StringSplit2Set(const std::string&s, char delim, std::unordered_set<std::string>& elems);


elems这个换成 * 把，一般需要修改的值的用引用+const修饰，需要修改值的用 *

src/pstd/include/pstd_string.h

Mixficsol · 2023-12-04T06:21:08Z

src/pstd/include/pstd_string.h

@@ -50,6 +51,8 @@ int string2int(const char* s, size_t slen, unsigned long* lval);
 int d2string(char* buf, size_t len, double value);
 int string2d(const char* s, size_t slen, double* dval);
 std::vector<std::string>& StringSplit(const std::string& s, char delim, std::vector<std::string>& elems);
+void StringSplit2Set(const std::string&s, char delim, std::unordered_set<std::string>& elems);


这个delim也尽量不要值传递把，看下用指针还是引用

src/pstd/include/pstd_string.h

codis/config/proxy.toml

codis/pkg/proxy/config.go

include/pika_conf.h

yes

wangshao1 · 2023-12-04T12:32:12Z

src/pika_server.cc

+    // Print the current low level queue size if it exceeds QUEUE_SIZE_THRESHOLD_PERCENTAGE/100 of the maximum queue size.
+    size_t low_level_queue_cur_size = LowLevelThreadPoolCurQueueSize();
+    size_t low_level_queue_max_size = LowLevelThreadPoolMaxQueueSize();
+    size_t low_level_queue_thread_hold = (low_level_queue_max_size / 100) * QUEUE_SIZE_THRESHOLD_PERCENTAGE;


变量名是想写low_level_queue_thresh_hold？

这里的名字都要改成slow_cmd相关，下次提交的时候改

src/pika_client_conn.cc

wangshao1 · 2023-12-04T13:16:03Z

codis/pkg/proxy/backend.go

+	if quick := s.owner.quick; quick > 0 {
+		if isQuick {
+			i = seed % uint(quick)
+			if bc := parallel[i]; bc.IsConnected() {


这个地方，原来是在一个循环里，如果一个bc无效会继续找下一个，这里是不是也该加一下。

这里传入的seed是用key做的hash，为了保证pipeline中相同key的有序，所以没有找下一个连接。之前不知道为什么是进行了遍历，所以新添加的代码没有进行循环找conn

wangshao1 · 2023-12-04T13:29:37Z

codis/pkg/proxy/mapper.go

@@ -55,10 +61,14 @@ const (
 	FlagMasterOnly
 	FlagMayWrite
 	FlagNotAllow
+	FlagQuick


Quick和Slow不是互斥的吗？需要两个都定义吗？

理论上只需要FlagSlow即可，但是为了简化一些逻辑就加了个FlagQuick，这样在一些打印快慢列表的函数中逻辑更清晰点，不需要的话删除也行

AlexStocks · 2023-12-08T12:10:53Z

继续改进下，解决下文件冲突

chejinge · 2024-01-18T03:45:31Z

codis/pkg/proxy/forward.go

 					return bc
 				}
 			}
 		}
 	}
-	return s.backend.bc.BackendConn(database, seed, true)
+	//  fix:https://github.com/OpenAtomFoundation/pika/issues/2174


这行去掉吧可以放在comment中

chejinge · 2024-01-18T03:46:46Z

codis/pkg/proxy/mapper.go

+	flagString := "FlagQuick"
+	if flag&FlagSlow != 0 {
+		reverseFlag = FlagQuick
+		flagString = "FlagSlow"


为啥FlagQuick 没有引号 FlagSlow有引号这是有什么特殊的考虑吗

* split fast and slow cmd in codis * separation of fast and slow commands * Resolve formatting issues. * use the unordered_set instead of unordered_map * fix:disorderly execution of the same key when using pipeline * update go 1.18 to go 1.21 * change proxy.go & mapper.go

split fast and slow cmd in codis

1838b51

github-actions bot added the Invalid PR Title label Nov 24, 2023

dingxiaoshuai123 force-pushed the split_fast_slow_cmd_codis branch from 1aeb603 to daa8a12 Compare November 24, 2023 04:57

dingxiaoshuai123 changed the title ~~separation of fast and slow commands~~ feat : separation of fast and slow commands Nov 24, 2023

dingxiaoshuai123 changed the title ~~feat : separation of fast and slow commands~~ feat:separation of fast and slow commands Nov 24, 2023

github-actions bot removed the Invalid PR Title label Nov 24, 2023

dingxiaoshuai123 closed this Nov 24, 2023

dingxiaoshuai123 reopened this Nov 24, 2023

github-actions bot added the ✏️ Feature New feature or request label Nov 24, 2023

separation of fast and slow commands

f9ed848

dingxiaoshuai123 force-pushed the split_fast_slow_cmd_codis branch from daa8a12 to f9ed848 Compare November 24, 2023 05:16