fix: 主机AgentId更新后，小概率出现使用旧AgentId下发任务 #2142 #2178

jsonwan · 2023-06-29T07:24:25Z

更新/删除主机前，对比cmdb主机数据获取时间与当前DB数据最后更新时间，防止旧数据覆盖新数据；
去除批量更新主机长事务；
去除主机数据更新与主机关系数据更新间的耦合；
全量更新改为增量更新；
去除对主机表中拓扑数据的引用，后续去除冗余拓扑字段。

1.更新/删除主机前，对比cmdb主机数据获取时间与当前DB数据最后更新时间，防止旧数据覆盖新数据； 2.去除批量更新主机长事务；

1.去除主机数据更新与主机关系数据更新间的耦合； 2.全量更新改为增量更新； 3.去除对主机表中拓扑数据的引用，后续去除冗余拓扑字段。

wangyu096 · 2023-06-30T02:35:49Z

support-files/sql/job-manage/0031_job_manage_20230618-1000_V3.7.4_mysql.sql

+                  WHERE TABLE_SCHEMA = db
+                    AND TABLE_NAME = 'host'
+                    AND COLUMN_NAME = 'last_time') THEN
+    ALTER TABLE host ADD COLUMN last_time BIGINT(20) NOT NULL DEFAULT 0 COMMENT 'CMDB中的数据最后修改时间';


数据库字段最好不要有中文注释

wangyu096 · 2023-06-30T02:35:55Z

support-files/sql/job-manage/0031_job_manage_20230618-1000_V3.7.4_mysql.sql

+                  WHERE TABLE_SCHEMA = db
+                    AND TABLE_NAME = 'host_topo'
+                    AND COLUMN_NAME = 'last_time') THEN
+    ALTER TABLE host_topo ADD COLUMN last_time BIGINT(20) NOT NULL DEFAULT 0 COMMENT 'CMDB中的数据最后修改时间' AFTER `app_id`;


数据库字段最好不要有中文注释

wangyu096 · 2023-06-30T02:53:14Z

src/backend/commons/common/src/main/java/com/tencent/bk/job/common/util/TimeUtil.java

@@ -155,4 +154,14 @@ public static String formatTime(long timeMills, String format) {
        DateFormat formatter = new SimpleDateFormat(format);
        return formatter.format(date);
    }
+
+    public static Long parseZonedTime(String zonedTimeStr) {


基础工具需要添加注释以及对应的单元测试代码。另外，如果这个 dateformat 如果只支持 ISO_DATE_TIME 的话，也需要在注释中说明限制

已添加，方法名称已改为ISO专用

wangyu096 · 2023-06-30T02:56:27Z

...age/service-job-manage/src/main/java/com/tencent/bk/job/manage/dao/impl/HostTopoDAOImpl.java

+        if (CollectionUtils.isEmpty(hostTopoList)) {
+            return 0;
+        }
+        int batchSize = 1000;


分批可以使用 CollectionUtil.partitionList() 方法

wangyu096 · 2023-06-30T02:57:23Z

...age/service-job-manage/src/main/java/com/tencent/bk/job/manage/dao/impl/HostTopoDAOImpl.java

+        return affectedNum;
+    }
+
+    public int updateBeforeLastTime(HostTopoDTO hostTopo) {


private ? 如果是继承的话，添加 @OverRide

wangyu096 · 2023-06-30T02:58:11Z

...age/service-job-manage/src/main/java/com/tencent/bk/job/manage/dao/impl/HostTopoDAOImpl.java

+        if (CollectionUtils.isEmpty(hostTopoList)) {
+            return 0;
+        }
+        int batchSize = 1000;


同上，分批使用工具方法

wangyu096 · 2023-06-30T03:00:21Z

...ce-job-manage/src/main/java/com/tencent/bk/job/manage/service/host/impl/HostServiceImpl.java

+        hostInfoList.forEach(hostCache::addOrUpdateHost);
+        watch.stop();
+        if (watch.getTotalTimeMillis() < 10_000L) {
+            log.debug("Performance:batchUpdateHostsBeforeLastTime:{}", watch.prettyPrint());


isDebugEnabled

wangyu096 · 2023-06-30T03:00:30Z

...ce-job-manage/src/main/java/com/tencent/bk/job/manage/service/host/impl/HostServiceImpl.java

-        }
-        log.debug("Performance:insertHostsToApp:appId={},{}", bizId, watch.prettyPrint());
-        return insertFailHostIds;
+        log.debug("Performance:insertHosts:{}", watch.prettyPrint());


isDebugEnabled

review意见处理

fix: 主机AgentId更新后，小概率出现使用旧AgentId下发任务 TencentBlueKing#2142

b6a498b

1.更新/删除主机前，对比cmdb主机数据获取时间与当前DB数据最后更新时间，防止旧数据覆盖新数据； 2.去除批量更新主机长事务；

jsonwan force-pushed the github_fix/sync branch 4 times, most recently from b4cfca5 to d893312 Compare June 29, 2023 14:30

fix: 主机AgentId更新后，小概率出现使用旧AgentId下发任务 TencentBlueKing#2142

e8d0468

1.去除主机数据更新与主机关系数据更新间的耦合； 2.全量更新改为增量更新； 3.去除对主机表中拓扑数据的引用，后续去除冗余拓扑字段。

jsonwan force-pushed the github_fix/sync branch from d893312 to e8d0468 Compare June 30, 2023 02:05

wangyu096 requested changes Jun 30, 2023

View reviewed changes

fix: 主机AgentId更新后，小概率出现使用旧AgentId下发任务 TencentBlueKing#2142

6906819

review意见处理

wangyu096 approved these changes Jun 30, 2023

View reviewed changes

wangyu096 merged commit 16a8286 into TencentBlueKing:master Jun 30, 2023
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: 主机AgentId更新后，小概率出现使用旧AgentId下发任务 #2142 #2178

fix: 主机AgentId更新后，小概率出现使用旧AgentId下发任务 #2142 #2178

jsonwan commented Jun 29, 2023

wangyu096 Jun 30, 2023

jsonwan Jun 30, 2023

wangyu096 Jun 30, 2023

jsonwan Jun 30, 2023

wangyu096 Jun 30, 2023

jsonwan Jun 30, 2023

wangyu096 Jun 30, 2023

jsonwan Jun 30, 2023

wangyu096 Jun 30, 2023

jsonwan Jun 30, 2023

wangyu096 Jun 30, 2023

jsonwan Jun 30, 2023

wangyu096 Jun 30, 2023

jsonwan Jun 30, 2023

wangyu096 Jun 30, 2023

jsonwan Jun 30, 2023

fix: 主机AgentId更新后，小概率出现使用旧AgentId下发任务 #2142 #2178

fix: 主机AgentId更新后，小概率出现使用旧AgentId下发任务 #2142 #2178

Conversation

jsonwan commented Jun 29, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment