Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pika 3.3.6 的线上服务,会出现很多 hmset 的慢查询日志,pika HMSET HMGET有点吃cpu #1716

Closed
luky116 opened this issue Jul 11, 2023 · 2 comments

Comments

@luky116
Copy link
Collaborator

luky116 commented Jul 11, 2023

pika_client_conn.cc:158] table: db0, command: "HMGET" "d3d01a35ee5a87c909bdd60a52187779" "gnid" "gzh_id" "title_vecstr" "btag_json" "store_news_type" "dupid" "kws_title" "tbtag" "title" "tag" "btag" "cluster" "content_length" "content_pic_count" "image" "pdate" "duration" "store_vvctag" "vbtcls" "store_vctag" "s", command_size: 222, arguments: 23, start_time(s): 1682823831, duration(us): 2161166
pika_client_conn.cc:158] table: db0, command: "HMGET" "d3d01a35ee5a87c909bdd60a52187779" "gnid" "gzh_id" "title_vecstr" "btag_json" "store_news_type" "dupid" "kws_title" "pdate" "cluster" "v_qt" "store_lt", command_size: 133, arguments: 13, start_time(s): 1682823831, duration(us): 2004331

机器配置:
Linux 693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
10 核 256G

@chejinge
Copy link
Collaborator

HMSET HMGET需要scan吃CPU是必然的,想要不吃cpu的话有两种方案:

1.加个缓存层抗一抗

2.改格式,编成一行自然就不吃cpu了

加编写成一行的hash_v2接口,新的业务用新的,旧的业务慢慢切,编一行+blobdb把scan干掉

当然这样也是有缺点的大key肯定不行,假设一条1k也就1万条,极限了,但是有的业务要是达到1亿呢这种情况需要考虑到

编写成一行,需要pb/fbs编码得业务配合把大key修改成为拉链的,但是这样的话业务方基本不怎么愿意的

scan高cpu对于大key是比较艰难的优先做缓存层吧

@chejinge chejinge changed the title pika 3.3.6 的线上服务,会出现很多 hmset 的慢查询日志,确认最新的代码是否会有类似问题 pika 3.3.6 的线上服务,会出现很多 hmset 的慢查询日志,pika HMSET HMGET有点吃cpu Jul 28, 2023
@luky116
Copy link
Collaborator Author

luky116 commented Sep 2, 2023

缓存层解决

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants