Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] 同步插件进程状态优化 #1887

Closed
ZhuoZhuoCrayon opened this issue Oct 25, 2023 · 0 comments
Closed

[FEATURE] 同步插件进程状态优化 #1887

ZhuoZhuoCrayon opened this issue Oct 25, 2023 · 0 comments
Assignees
Labels
done 已上线到正式环境并验收通过 kind/feature 功能
Projects

Comments

@ZhuoZhuoCrayon
Copy link
Member

背景

为什么需要

现在先遍历业务,再分页异步查询的逻辑,在业务数量多(> 3000)的情况下,会产生非常多的 celery 任务和第三方请求,这里需要做个优化,对业务列表进行分批,比如之前是 1,2,3,4 遍历四次,现在设置窗口为 2,改为 [1, 2]、[3, 4] 遍历 2 次

image

在实际代码中,窗口大小设置为 15

功能

你想要什么功能

对查询的业务进行分片

功能实现

建议的方案

分片逻辑:apps.utils.basic.chunk_lists

熟悉 python manage.py sync_plugin_status 命令

实现方案

功能自测

代码变更覆盖功能点需要自测并截图

功能点 1

描述代码变更涉及功能点及自测截图

功能点 2

描述代码变更涉及功能点及自测截图

...

@ZhuoZhuoCrayon ZhuoZhuoCrayon added kind/feature 功能 backlog 需求初始状态,等待产品进行评估 labels Oct 25, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Oct 27, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Oct 27, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 2, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 2, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 2, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 2, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 2, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 2, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 3, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 3, 2023
Huayeaaa added a commit to Huayeaaa/bk-nodeman that referenced this issue Nov 3, 2023
@ZhuoZhuoCrayon ZhuoZhuoCrayon added doing 已经接受处理,正在开发实验 and removed backlog 需求初始状态,等待产品进行评估 labels Nov 6, 2023
@ZhuoZhuoCrayon ZhuoZhuoCrayon added for test 可以在测试环境进行验收 and removed doing 已经接受处理,正在开发实验 labels Nov 13, 2023
ping15 pushed a commit to ping15/bk-nodeman that referenced this issue Nov 15, 2023
@wyyalt wyyalt added this to In progress in 2.4.4 Nov 17, 2023
@wyyalt wyyalt added doing 已经接受处理,正在开发实验 for test 可以在测试环境进行验收 tested 已经在测试环境验收通过 done 已上线到正式环境并验收通过 and removed for test 可以在测试环境进行验收 doing 已经接受处理,正在开发实验 tested 已经在测试环境验收通过 labels Nov 17, 2023
@wyyalt wyyalt closed this as completed in 946a580 Mar 1, 2024
2.4.4 automation moved this from In progress to Done Mar 1, 2024
@wyyalt wyyalt moved this from Done to In progress in 2.4.4 Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
done 已上线到正式环境并验收通过 kind/feature 功能
Projects
2.4.4
In progress
Development

Successfully merging a pull request may close this issue.

3 participants