Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/executors有数据丢失的风险 #186

Closed
idle-ape opened this issue Nov 5, 2020 · 1 comment
Closed

core/executors有数据丢失的风险 #186

idle-ape opened this issue Nov 5, 2020 · 1 comment

Comments

@idle-ape
Copy link

idle-ape commented Nov 5, 2020

语鹊的文档地址:https://www.yuque.com/tal-tech/go-zero/zig6zy

文中说的“带上 dts.insertExecutor.Wait(),当然要等待全部的 goroutine task 完成”,我做了个实验,代码如下:
executor := executors.NewBulkExecutor(
myExecutor,
executors.WithBulkInterval(time.Second*3), // 3s会自动刷一次container中task去执行
executors.WithBulkTasks(16), // container最大task数。一般设为2的幂次
)
for i := 0; i < 20; i++ {
executor.Add(i)
}
executor.Wait()
// time.Sleep(10 * time.Second)
// executor.Flush()
如果把executor.Flush()注释掉,只会打印出0到15,进程就退出了,剩下的任务就丢失了。因为periodicalexecutor.go里的 pe.waitGroup.Add(1) 是在子协程里调用的,所以调用了executor.Wait()主协程也等不到子协程到ticker的时间自动去flush就退出了,所以16到20的任务就丢失了。在主协程调用executor.Flush(),其实是把剩下的任务取出到了主协程来执行了,如果忘了的话就会丢任务。或者主协程执行的时间足够长,能让子协程达到ticker的时间自动去flush。

kevwan added a commit that referenced this issue Nov 6, 2020
@kevwan
Copy link
Contributor

kevwan commented Nov 6, 2020

fixed.

Flush first in Wait.

@kevwan kevwan closed this as completed Nov 6, 2020
kesonan pushed a commit to kesonan/go-zero that referenced this issue Nov 12, 2020
jeyrce pushed a commit to jeyrce/go-zero that referenced this issue Aug 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants