Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DjangoBookWriter does not support batch_size #24

Closed
readflybird opened this issue Dec 17, 2016 · 5 comments
Closed

DjangoBookWriter does not support batch_size #24

readflybird opened this issue Dec 17, 2016 · 5 comments

Comments

@readflybird
Copy link

In class DjangoBookWriter, the open_content() method, the parameters keywords are ignored and thus there is no way we can pass in the batch_size. Besides, the create_sheet() method does not take care of batch_size, neither. Therefore there is no way we can do a batch insert through DjangoModelWriter.

@chfw
Copy link
Member

chfw commented Dec 19, 2016

It is true that 'batch_size' was not passed on and 'batch_size' controls the number of records being inserted at once. If not mentioned, default batch_size 999 is used. However, it won't block batch insertion.

Please check for database error such as integrity error during bulk import. Once there is any exception, it will default to 'snail' alike insertion - one record at a time, skipping offending ones.

@chfw chfw closed this as completed in 438a87e Dec 20, 2016
@readflybird
Copy link
Author

Thanks for the quick fix. Is there any plan that to merge this change into a release branch?

@chfw
Copy link
Member

chfw commented Dec 21, 2016

by default, I would release it with v0.3.0, which should be released by the end of this year. Can you wait for it? Otherwise, I can do a quick release as v0.2.6.

chfw added a commit that referenced this issue Dec 21, 2016
@chfw
Copy link
Member

chfw commented Dec 21, 2016

released in v0.2.6 via pypi

@readflybird
Copy link
Author

Thanks a lot for the response! I have tried out the new version and it works great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants