Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Move to SQLite 3080600 (aka 3.8.6) #4235

Merged
merged 1 commit into from

6 participants

@bavison

This new version of SQLite offers significant speed improvements in my testing. For example, the time it takes to issue

SELECT * FROM tvshowview  GROUP BY tvshowview.idShow

(the first time the TV Shows library is opened) with 344 TV shows on a non-overclocked Raspberry Pi changed as follows. Times are in seconds:

      Before          After
      Mean   StdDev   Mean   StdDev  Confidence  Change
Time  2935.6 6.2      2774.1 38.3    100.0%      +5.8%
@MartijnKaijser

If bumped this needs to be done for windows as well else the lib versions will diverge. And obviously this is after Gotham material.

@MartijnKaijser MartijnKaijser added this to the Pending for inclusion milestone
@bavison

Are you sure there really is a separate build system for Windows versions of SQLite? I'm not particularly familiar with the XBMC Windows build, but unlike with several of the other sub-projects in tools/depends, I can't see anything in any .vcproj or .vcxproj files that looks like it's being used to build SQLite.

@bavison bavison referenced this pull request from a commit in bavison/xbmc
@bavison bavison Enable SQLite to use mmap for file access.
The time it takes to issue "SELECT * FROM tvshowview" (the first time the
TV Shows library is opened) with 345 TV shows on a non-overclocked
Raspberry Pi changed as follows. Times are in seconds:

      Before          After
      Mean   StdDev   Mean   StdDev  Confidence  Change
Time  3.352  0.132    2.550  0.021   100.0%      +31.5%

These times assume SQLite has already been upgraded to version 3080301
(see PR #4235).
7696d5e
@wsoltys
Collaborator
@bavison

OK, part of the confusion was from the way the version numbers are represented differently: the makefile fetched the sources from http://www.sqlite.org/2014/sqlite-autoconf-3071000.tar.gz (before I changed it) and 0_package.list refers to sqlite-3.7.16.2-win32.7z. Also, even assuming the first one is in hex and the other in decimal, there's a difference in the fourth-level version number, but presumably that doesn't matter much.

Is there anyone in a position to assist in producing the precompiled lib of sqlite 3.8.3.1 for Windows (and presumably uploading it to wherever it's supposed to go)?

@wsoltys
Collaborator
@zag2me
Collaborator

20% speed improvement! nice!

@theimmortal68 theimmortal68 referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@miappa miappa referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@miappa miappa referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@miappa miappa referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@miappa miappa referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@bavison bavison referenced this pull request from a commit in bavison/xbmc
@bavison bavison Enable SQLite to use mmap for file access.
The time it takes to issue "SELECT * FROM tvshowview" (the first time the
TV Shows library is opened) with 345 TV shows on a non-overclocked
Raspberry Pi changed as follows. Times are in seconds:

      Before          After
      Mean   StdDev   Mean   StdDev  Confidence  Change
Time  3.352  0.132    2.550  0.021   100.0%      +31.5%

These times assume SQLite has already been upgraded to version 3080301
(see PR #4235).
5e6ba5b
@MartijnKaijser

should get this rolling again for Helix?
@Karlson2k You re-packaged it last time for VS2013 right?

@Karlson2k
Collaborator

@MartijnKaijser Don't remember, I'll check.

@Karlson2k
Collaborator

@MartijnKaijser I did it for last version in 3.7 branch. It's sqlite-3.7.16.2-win32-vc120.7z
Could update it for 3.8 branch.

@MartijnKaijser

Something for next merge window I guess. If @bavison could update his PR as well and we bump to the same versions. Perhaps retest the gain

@bavison

I assume you mean we'll update both with whatever is the latest version at the time? Just let me know when you're ready. For the record, the latest version has reached 3080500 at the time of writing.

@MartijnKaijser

Correct. We should of course do some runtime testing for possible regression before merging

@popcornmix
Collaborator

OpenELEC switched to this version (3080301) back in February, and is now at 3080403.

I've just updated to 3080500 on Pi. It built and seems to be running fine.

@MartijnKaijser

@bavison care to update? @Karlson2k do you want to also bump sql (perhaps through his branch)?

@bavison

OK, I've re-run the timings with the latest version. Unrelated changes appear to have reduced the percentage speedup since then, but it's still worth having.

@MartijnKaijser

@Karlson2k yes. can you compile the windows one and put it on the mirror? As well as the source for the other platforms?
@bavison can you remove the BASEURL and update the PR description?

@Karlson2k
Collaborator

@MartijnKaijser Will update win32 one.

@bavison

Yes, 3.8.6 appears to be the latest at the time of writing. I've updated the description (though I suspect we're now stuck with the branch name sqlite3080301, sorry).

I've forgotten the logic behind the BASE_URL line now. I was sceptical myself this morning, so I tried omitting it, the first time I tried reapplying the patch on top of master, and it failed to rebuild. Is there a better place to specify this?

@bavison bavison changed the title from Move to SQLite 3080301 to Move to SQLite 3080600
@popcornmix
Collaborator

@bavison
I think once Karlson2k has uploaded the new sqlite packages to xbmc's repo, then it will build with BASE_URL removed.

@MartijnKaijser

you can edit the PR name with the edit button behind the name
let's get this in on 1 september

@bavison bavison This new version of SQLite offers significant speed improvements in my
testing. For example, the time it takes to issue

    SELECT * FROM tvshowview  GROUP BY tvshowview.idShow

(the first time the TV Shows library is opened) with 344 TV shows on a
non-overclocked Raspberry Pi changed as follows. Times are in seconds:

          Before          After
          Mean   StdDev   Mean   StdDev  Confidence  Change
    Time  2935.6 6.2      2774.1 38.3    100.0%      +5.8%
f386a58
@bavison bavison changed the title from Move to SQLite 3080600 to Move to SQLite 3080600 (aka 3.8.6)
@bavison

OK, makes sense about the BASE_URL, so there's a caveat that this version isn't expected to build until the package is updated.

@Karlson2k
Collaborator

Uploaded sqlite-autoconf-3080600.tar.gz and sqlite-3.8.6-win32-vc120.7z.
project/BuildDependencies/scripts/0_package.list need to be updated to use new .dll on win32.

@MartijnKaijser MartijnKaijser merged commit 9d69669 into xbmc:master
@stefansaraev stefansaraev referenced this pull request from a commit in stefansaraev/xbmc
@bavison bavison Enable SQLite to use mmap for file access.
The time it takes to issue "SELECT * FROM tvshowview" (the first time the
TV Shows library is opened) with 345 TV shows on a non-overclocked
Raspberry Pi changed as follows. Times are in seconds:

      Before          After
      Mean   StdDev   Mean   StdDev  Confidence  Change
Time  3.352  0.132    2.550  0.021   100.0%      +31.5%

These times assume SQLite has already been upgraded to version 3080301
(see PR #4235).
715fc2a
@Leatherface75 Leatherface75 referenced this pull request from a commit in Leatherface75/xbmc
@bavison bavison Enable SQLite to use mmap for file access.
The time it takes to issue "SELECT * FROM tvshowview" (the first time the
TV Shows library is opened) with 345 TV shows on a non-overclocked
Raspberry Pi changed as follows. Times are in seconds:

      Before          After
      Mean   StdDev   Mean   StdDev  Confidence  Change
Time  3.352  0.132    2.550  0.021   100.0%      +31.5%

These times assume SQLite has already been upgraded to version 3080301
(see PR #4235).
1172630
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Aug 19, 2014
  1. @bavison

    This new version of SQLite offers significant speed improvements in my

    bavison authored
    testing. For example, the time it takes to issue
    
        SELECT * FROM tvshowview  GROUP BY tvshowview.idShow
    
    (the first time the TV Shows library is opened) with 344 TV shows on a
    non-overclocked Raspberry Pi changed as follows. Times are in seconds:
    
              Before          After
              Mean   StdDev   Mean   StdDev  Confidence  Change
        Time  2935.6 6.2      2774.1 38.3    100.0%      +5.8%
This page is out of date. Refresh to see the latest.
Showing with 1 addition and 1 deletion.
  1. +1 −1  tools/depends/target/sqlite3/Makefile
View
2  tools/depends/target/sqlite3/Makefile
@@ -3,7 +3,7 @@ DEPS= ../../Makefile.include Makefile
# lib name, version
LIBNAME=sqlite
-VERSION=3071000
+VERSION=3080600
SOURCE=$(LIBNAME)-autoconf-$(VERSION)
ARCHIVE=$(SOURCE).tar.gz
Something went wrong with that request. Please try again.