Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Django批量创建对象时,使用bulk_create进一步减少耗时 #7

Closed
dellkeji opened this issue Sep 6, 2022 · 4 comments · Fixed by #17
Closed

Django批量创建对象时,使用bulk_create进一步减少耗时 #7

dellkeji opened this issue Sep 6, 2022 · 4 comments · Fixed by #17
Labels
Django Django 框架的最佳实践 good first issue🌸 Good for newcomers

Comments

@dellkeji
Copy link

dellkeji commented Sep 6, 2022

# BAD
## 每次都执行commit,整体耗时较长(大约25s左右)
for num in range(10000):
    Record.objects.create(num=num)


# GOOD
## 统一提交数据库,耗时很短(1s以内)
inserted_list = []
for num in range(10000):
    inserted_list.append(Demo(num=num))

Record.objects.bulk_create(inserted_list)

注意: bulk_create 方法只执行一次数据库交互,这样相当于创建时间一样,并且自定字段不会在返回数据中

@IMBlues
Copy link
Collaborator

IMBlues commented Sep 6, 2022

之前遇到过一个坑:bulk_create 提交的东西过多,会导致 MySQL 异常,可能根据具体场景,适当对 inserted_list 做切片提交会更稳妥。

@IMBlues IMBlues added Django Django 框架的最佳实践 good first issue🌸 Good for newcomers labels Sep 6, 2022
@unique0lai
Copy link

之前遇到过一个坑:bulk_create 提交的东西过多,会导致 MySQL 异常,可能根据具体场景,适当对 inserted_list 做切片提交会更稳妥。

可以设置batch_size

@IMBlues
Copy link
Collaborator

IMBlues commented Sep 7, 2022

之前遇到过一个坑:bulk_create 提交的东西过多,会导致 MySQL 异常,可能根据具体场景,适当对 inserted_list 做切片提交会更稳妥。

可以设置batch_size

我之前居然还手写了切片 😵

@dellkeji
Copy link
Author

dellkeji commented Sep 7, 2022

当数量不可控时,一定要添加batch_size

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Django Django 框架的最佳实践 good first issue🌸 Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants