Use mmap to accelerate SQLite file accesses #4275
Conversation
The time it takes to issue "SELECT * FROM tvshowview" (the first time the
TV Shows library is opened) with 345 TV shows on a non-overclocked
Raspberry Pi changed as follows. Times are in seconds:
Before After
Mean StdDev Mean StdDev Confidence Change
Time 3.352 0.132 2.550 0.021 100.0% +31.5%
These times assume SQLite has already been upgraded to version 3080301
(see PR xbmc#4235).
The time taken to issue "SELECT * FROM tvshowview" in seconds improves
still further in this case.
Before After
Mean StdDev Mean StdDev Confidence Change
Time 2.550 0.021 2.375 0.043 100.0% +7.4%
Note, this requires the standard SQLite amalgamation tarball to be patched
prior to building. The change may not be desirable further upstream due to
the fact that it might cause database access times to increase in usage
scenarios that differ from those employed by XBMC.
|
According to sqllite docs: Can't we do this runtime instead so it takes effect on system libs too? Also that seem like a somewhat large value for the define? I'm not sure what happen if sqllite runs out of memory. |
|
I believe settings the MMAP size large is safe. Eben investigated this and commented: |
|
@bavison: can this be done at runtime (the first commit)? |
|
From the notes I made at the time, my first attempts did indeed use sqlite3_config(), but they weren't successful - they didn't have any statistically significant effect on timings. Part of the problem appears to have been that according to sqlite3.h, this call needed to be made before sqlite3_initialize(). sqlite3_initialize() is not explicitly called from anywhere in XBMC, but it is called from within sqlite3_open_v2(), which is called from dbiplus::SqliteDatabase::connect(). But that's called many times during XBMC startup, so there's no obvious place to put one-time initialisation of the SQLite engine as a whole, and a global "have we configured SQLite yet" flag seemed like a worse solution than changing a predefine. |
|
@bavison @popcornmix what's the story on this PR? |
|
I believe this PR gives a performance benefit and has been included in OpenELEC for over a year without issue: I think it should be merged. @bavison can you rebase? |
|
@bavison, mind rebasing? |
|
I picked this work in #7926 |
Here are a couple of changes that between them make SQLite searches about 41% faster in my tests on a Raspberry Pi.
They have only been tested with PR #4235 in place, and the timings were also done using SQLite version 3080301, but they may still work with the older version 3071000 (though you'll probably get at least some warnings about line offsets in the patch operation).