Optimize Query insertMangaImage
& Typo Fix
#678
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello!
This PR is to change the
insertMangaImage
function which inserts images into the Manga table;I have also fixed a typo in the dropDatabase function.
insertMangaImage
function toinsertMangaImages
- this is to more closely align the name of the function with what it does with these changes.process_image
function; this is to improve speed and resource utilization while inserting manga into the DB, and to better form the data for theinsertMangaImages
function.dropDatabase
function had a typo in it, I changed the query fromDROP IF EXISTS TABLE
->DROP TABLE IF EXISTS
(Docs: https://www.sqlite.org/lang_droptable.html)Explanation & Reasoning:
The previous method invoked the
insertMangaImage
function for each page in a manga; this can create a lot of overhead, in terms of the number of times variables are passed around, the number of times extra functions are invoked, and as well as opening/committing/closing a connection to the SQLite DB for each page.The changes I have made reduce that to invoking the insertMangaImage(s) function once, opening and closing a single connection to the DB, and adding all needed rows to the DB in a batch operation-
This should result in a large increase in speed of inserting manga, but it largely depends on the number of pages in an image/artwork.
In my manual testing in the python console with dummy data, I've seen about a 3-5x increase in speed, but my testing may not be a direct parallel.
Please let me know if you think there are other changes needed here, or if you have feedback on these changes.
Info: