Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds SQLite3 support #980

Merged
merged 42 commits into from Jul 22, 2015

Conversation

Projects
None yet
5 participants
@mapkyca
Copy link
Member

mapkyca commented Jul 22, 2015

This patch does a bunch (below) but primarily it introduces support for SQLite file based sql storage.

I did this because SQL/Mongo has quite a lot of overhead in terms of both complexity and resources compared to straight files, and often seemed overkill for folk running Known on low powered devices (embedded / single use sites / simple hosting etc).

It provides a nice fallback, a useful tool for quickly building test builds, as well as reduces some of the admin overhead for folk.

I've got plans for this for some IoT type stuff, and was meaning to do it for a while, but it was prompted by some guy on IRC who's SD card bought the farm, and I figured a SQLite db might be more appropriate.

Anyway, this patch does:

  • Introduces SQLite dataconcierge
  • Adds SQLite to travis builds
  • Introduces a dataconciege test to build which tests mysql, sqlite and mongo builds
  • Fixes a few bugs exposed by these unit tests
  • Exposes some inconsistencies between mongo and mysql (#978)

Anyway, it seems to be fairly stable, but should probably be considered "experimental" until the path gets trodden a little more.

@benwerd

This comment has been minimized.

Copy link
Member

benwerd commented Jul 22, 2015

Waiting for Travis and then I'll merge this in. Your commit messages are ... interesting ;)

@kylewm

This comment has been minimized.

Copy link
Collaborator

kylewm commented Jul 22, 2015

oh hell yeah! this will also be useful for people who want to run on Sandstorm.io (since everything is containerized, they recommend SQLite if possible)

@mapkyca

This comment has been minimized.

Copy link
Member Author

mapkyca commented Jul 22, 2015

Frustration: got bit on the bum by mysql's weird FTS restraints

benwerd added a commit that referenced this pull request Jul 22, 2015

Merge pull request #980 from mapkyca/sqlite3
Adds SQLite3 support

@benwerd benwerd merged commit c352f0f into idno:master Jul 22, 2015

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@paulproteus

This comment has been minimized.

Copy link

paulproteus commented Jul 22, 2015

My main question here is -- what happens if two simultaneous HTTP requests to Known want to writes? Will these be properly sequenced? Will one of them fail with Error 500?

If things will be honky-dory, then awesome. But I just wanted to make sure to check what the answer to this is.

@mapkyca

This comment has been minimized.

Copy link
Member Author

mapkyca commented Jul 22, 2015

SQLite uses a file on disk, so it should be (almost) no different than accessing any other static file (for reads). Concurrent writes you'll start to lower performance since you enter the standard lock & queue IO gubbins... how bad that is depends on your file system.

@paulproteus

This comment has been minimized.

Copy link

paulproteus commented Jul 22, 2015

@mapkyca

This comment has been minimized.

Copy link
Member Author

mapkyca commented Jul 22, 2015

It's implemented using PHP's PDO interface, and sqlite purports to be ACID, so I'm going to kick that one down the road. ;)

@mapkyca mapkyca deleted the mapkyca:sqlite3 branch Jul 22, 2015

@Phyks

This comment has been minimized.

Copy link
Contributor

Phyks commented Jul 26, 2015

Do you plan on adding an easy way to migrate from a MySQL / MongoDB database to an SQLite one for old users?

@benwerd

This comment has been minimized.

Copy link
Member

benwerd commented Jul 26, 2015

There's going to be better site to site migration, and I suppose you could
use that for this.

On Sunday, July 26, 2015, Lucas Verney notifications@github.com wrote:

Do you plan on adding an easy way to migrate from a MySQL / MongoDB
database to an SQLite one for old users?


Reply to this email directly or view it on GitHub
#980 (comment).

Ben Werdmuller
http://goog_1933028737
benwerd.com | werd.io

+1 (312) 488-9373

@mapkyca

This comment has been minimized.

Copy link
Member Author

mapkyca commented Jul 26, 2015

Also, SQLite is an "as well as" rather than "instead of" wrt MySQL/mongo... Hopefully useful, but not a replacement :)

@Phyks

This comment has been minimized.

Copy link
Contributor

Phyks commented Jul 27, 2015

Ok :) I was trying to get rid of MySQL on my server as very few apps need it :)

@kylewm

This comment has been minimized.

Copy link
Collaborator

kylewm commented Jul 27, 2015

@mapkyca could you elaborate on why it wouldn't be useful as a replacement?

@mapkyca

This comment has been minimized.

Copy link
Member Author

mapkyca commented Jul 27, 2015

It certainly would be a useful replacement for some people, I'm just saying that currently mongo is the most mature and MySQL is the current "blessed" engine, so both of those are going to be much more well trodden paths, and I don't thing @benwerd has plans on changing the defaults any time soon :)

FWIW: Sqlite is the engine I'm using in my dev environment, as it lets me swap configs and scenarios around very quickly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.