-
Notifications
You must be signed in to change notification settings - Fork 704
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Don't use temp file when restoring to in-memory SQLite #739
Conversation
This saves a read and write to disk, the size of the SQLite database.
When I actually run a node, have it do a few snapshots, and then restart it -- the CLI is getting "not a database". |
Don't merge this, not until I understand the errors (and if they are real, why the test suite isn't catching them). |
Seems fine, can't reproduce the issue via the CLI. I've also added end-to-end testing to ensure restoring to both in-mem and on-disk databases works fine. |
This change also adds better logging, so the restoration process can be better observed.
I got the system into this state again.
Not good. |
The test case seems to be:
|
Very strange that none of the current testing catches this, only the command line testing. |
I think I see the issue. This problem doesn't occur 100% of the time with the following script, but pretty close: #!/bin/bash
DATA_DIR=`mktemp -d`
$RQLITED -raft-snap 13 -raft-snap-int=1s $DATA_DIR &
sleep 3
$RQBENCH -b 10 -m 1 -n 500 -o 'CREATE TABLE foo (id INTEGER NOT NULL PRIMARY KEY, name TEXT)' 'INSERT INTO foo(name) VALUES("fiona")'
killall rqlited
sleep 3
$RQLITED -raft-snap 50000 -raft-snap-int=500s $DATA_DIR SQLite is using the memory passed in, and making changes to it -- that memory represents the database after all. But that memory is in the Go memory space. And I'm sure this is allowed, since Go is a garbage collected language. |
Saves a read and write, the size of the SQLite database, when restoring an in-memory database.