added backup function to database object#883
Closed
schraf wants to merge 3 commits intoTryGhost:masterfrom
Closed
Conversation
|
Changes Unknown when pulling 2a01921 on schraf:master into ** on mapbox:master**. |
|
I got an |
|
Actually the problem is the path not existed,we need to create the path first. |
kewde
requested changes
Jan 13, 2019
| Local<Function> cb = Nan::New(baton->callback); | ||
|
|
||
| if (baton->status != SQLITE_OK) { | ||
| EXCEPTION(Nan::New(baton->message.c_str()).ToLocalChecked(), baton->status, exception); |
Collaborator
There was a problem hiding this comment.
EXCEPTION(baton->message, baton->status, exception);
Collaborator
There was a problem hiding this comment.
To make it comply with V8 7.1
| }); | ||
|
|
||
| it('backup database', function(done) { | ||
| db.backup('test/support/backup.db', done); |
Collaborator
There was a problem hiding this comment.
Seems a bit of a thin test case.
I don't mind improving it if anyone has ideas of what could get wrong.
paulfitz
added a commit
to paulfitz/node-sqlite3
that referenced
this pull request
Jan 26, 2019
This is based on TryGhost#883 It creates a backup object that can be kept around so that the backup can be performed in leisurely steps, with plenty of opportunity for interruption.
paulfitz
added a commit
to paulfitz/node-sqlite3
that referenced
this pull request
Jan 28, 2019
This is based on TryGhost#883 It creates a backup object that can be kept around so that the backup can be performed in leisurely steps, with plenty of opportunity for interruption.
paulfitz
added a commit
to paulfitz/node-sqlite3
that referenced
this pull request
Jan 29, 2019
This is based on TryGhost#883 It creates a backup object that can be kept around so that the backup can be performed in leisurely steps, with plenty of opportunity for interruption.
paulfitz
added a commit
to paulfitz/node-sqlite3
that referenced
this pull request
Jan 29, 2019
This exposes the sqlite3 backup api as described at https://sqlite.org/backup.html. This implementation draws on TryGhost#883, extending it to create a backup object that can be used in the background, without leaving the database locked for an extended period of time. This is crucial for making backups of large live databases in a non-disruptive manner. Example usage: ``` var db = new sqlite3.Database('live.db'); var backup = db.backup('backup.db'); ... // in event loop, move backup forward when we have time. if (backup.idle) { backup.step(NPAGES); } if (backup.completed) { /* success! backup made */ } if (backup.failed) { /* sadness! backup broke */ } // do other work in event loop - fine to modify live.db ... ``` Here is how sqlite's backup api is exposed: * `sqlite3_backup_init`: This is implemented as `db.backup(filename, [callback])` or `db.backup(filename, destDbName, sourceDbName, filenameIsDest, [callback])`. * `sqlite3_backup_step`: This is implemented as `backup.step(pages, [callback])`. * `sqlite3_backup_finish`: This is implemented as `backup.finish([callback])`. * `sqlite3_backup_remaining`: This is implemented as a `backup.remaining` getter. * `sqlite3_backup_pagecount`: This is implemented as a `backup.pageCount` getter. Some conveniences are added in the node api. There are the following read-only properties: * `backup.completed` is set to `true` when the backup succeeeds. * `backup.failed` is set to `true` when the backup has a fatal error. * `backup.idle` is set to `true` when no operation is currently in progress or queued for the backup. * `backup.remaining` is an integer with the remaining number of pages after the last call to `backup.step` (-1 if `step` not yet called). * `backup.pageCount` is an integer with the total number of pages measured during the last call to `backup.step` (-1 if `step` not yet called). There is the following writable property: * `backup.retryErrors`: an array of sqlite3 error codes that are treated as non-fatal - meaning, if they occur, backup.failed is not set, and the backup may continue. By default, this is `[sqlite3.BUSY, sqlite3.LOCKED]`. The `db.backup(filename, [callback])` shorthand is sufficient for making a backup of a database opened by node-sqlite3. If using attached or temporary databases, or moving data in the opposite direction, the more complete (but daunting) `db.backup(filename, destDbName, sourceDbName, filenameIsDest, [callback])` signature is provided. A backup will finish automatically when it succeeds or a fatal error occurs, meaning it is not necessary to call `db.finish()`. By default, SQLITE_LOCKED and SQLITE_BUSY errors are not treated as failures, and the backup will continue if they occur. The set of errors that are tolerated can be controlled by setting `backup.retryErrors`. To disable automatic finishing and stick strictly to sqlite's raw api, set `backup.retryErrors` to `[]`. In that case, it is necessary to call `backup.finish()`. In the same way as node-sqlite3 databases and statements, backup methods can be called safely without callbacks, due to an internal call queue. So for example this naive code will correctly back up a db, if there are no errors: ``` var backup = db.backup('backup.db'); backup.step(-1); backup.finish(); ```
paulfitz
added a commit
to paulfitz/node-sqlite3
that referenced
this pull request
Jan 29, 2019
This exposes the sqlite3 backup api as described at https://sqlite.org/backup.html. This implementation draws on TryGhost#883, extending it to create a backup object that can be used in the background, without leaving the database locked for an extended period of time. This is crucial for making backups of large live databases in a non-disruptive manner. Example usage: ``` var db = new sqlite3.Database('live.db'); var backup = db.backup('backup.db'); ... // in event loop, move backup forward when we have time. if (backup.idle) { backup.step(NPAGES); } if (backup.completed) { /* success! backup made */ } if (backup.failed) { /* sadness! backup broke */ } // do other work in event loop - fine to modify live.db ... ``` Here is how sqlite's backup api is exposed: * `sqlite3_backup_init`: This is implemented as `db.backup(filename, [callback])` or `db.backup(filename, destDbName, sourceDbName, filenameIsDest, [callback])`. * `sqlite3_backup_step`: This is implemented as `backup.step(pages, [callback])`. * `sqlite3_backup_finish`: This is implemented as `backup.finish([callback])`. * `sqlite3_backup_remaining`: This is implemented as a `backup.remaining` getter. * `sqlite3_backup_pagecount`: This is implemented as a `backup.pageCount` getter. Some conveniences are added in the node api. There are the following read-only properties: * `backup.completed` is set to `true` when the backup succeeeds. * `backup.failed` is set to `true` when the backup has a fatal error. * `backup.idle` is set to `true` when no operation is currently in progress or queued for the backup. * `backup.remaining` is an integer with the remaining number of pages after the last call to `backup.step` (-1 if `step` not yet called). * `backup.pageCount` is an integer with the total number of pages measured during the last call to `backup.step` (-1 if `step` not yet called). There is the following writable property: * `backup.retryErrors`: an array of sqlite3 error codes that are treated as non-fatal - meaning, if they occur, backup.failed is not set, and the backup may continue. By default, this is `[sqlite3.BUSY, sqlite3.LOCKED]`. The `db.backup(filename, [callback])` shorthand is sufficient for making a backup of a database opened by node-sqlite3. If using attached or temporary databases, or moving data in the opposite direction, the more complete (but daunting) `db.backup(filename, destDbName, sourceDbName, filenameIsDest, [callback])` signature is provided. A backup will finish automatically when it succeeds or a fatal error occurs, meaning it is not necessary to call `db.finish()`. By default, SQLITE_LOCKED and SQLITE_BUSY errors are not treated as failures, and the backup will continue if they occur. The set of errors that are tolerated can be controlled by setting `backup.retryErrors`. To disable automatic finishing and stick strictly to sqlite's raw api, set `backup.retryErrors` to `[]`. In that case, it is necessary to call `backup.finish()`. In the same way as node-sqlite3 databases and statements, backup methods can be called safely without callbacks, due to an internal call queue. So for example this naive code will correctly back up a db, if there are no errors: ``` var backup = db.backup('backup.db'); backup.step(-1); backup.finish(); ```
Contributor
kewde
pushed a commit
that referenced
this pull request
Feb 21, 2019
This exposes the sqlite3 backup api as described at https://sqlite.org/backup.html. This implementation draws on #883, extending it to create a backup object that can be used in the background, without leaving the database locked for an extended period of time. This is crucial for making backups of large live databases in a non-disruptive manner. Example usage: ``` var db = new sqlite3.Database('live.db'); var backup = db.backup('backup.db'); ... // in event loop, move backup forward when we have time. if (backup.idle) { backup.step(NPAGES); } if (backup.completed) { /* success! backup made */ } if (backup.failed) { /* sadness! backup broke */ } // do other work in event loop - fine to modify live.db ... ``` Here is how sqlite's backup api is exposed: * `sqlite3_backup_init`: This is implemented as `db.backup(filename, [callback])` or `db.backup(filename, destDbName, sourceDbName, filenameIsDest, [callback])`. * `sqlite3_backup_step`: This is implemented as `backup.step(pages, [callback])`. * `sqlite3_backup_finish`: This is implemented as `backup.finish([callback])`. * `sqlite3_backup_remaining`: This is implemented as a `backup.remaining` getter. * `sqlite3_backup_pagecount`: This is implemented as a `backup.pageCount` getter. Some conveniences are added in the node api. There are the following read-only properties: * `backup.completed` is set to `true` when the backup succeeeds. * `backup.failed` is set to `true` when the backup has a fatal error. * `backup.idle` is set to `true` when no operation is currently in progress or queued for the backup. * `backup.remaining` is an integer with the remaining number of pages after the last call to `backup.step` (-1 if `step` not yet called). * `backup.pageCount` is an integer with the total number of pages measured during the last call to `backup.step` (-1 if `step` not yet called). There is the following writable property: * `backup.retryErrors`: an array of sqlite3 error codes that are treated as non-fatal - meaning, if they occur, backup.failed is not set, and the backup may continue. By default, this is `[sqlite3.BUSY, sqlite3.LOCKED]`. The `db.backup(filename, [callback])` shorthand is sufficient for making a backup of a database opened by node-sqlite3. If using attached or temporary databases, or moving data in the opposite direction, the more complete (but daunting) `db.backup(filename, destDbName, sourceDbName, filenameIsDest, [callback])` signature is provided. A backup will finish automatically when it succeeds or a fatal error occurs, meaning it is not necessary to call `db.finish()`. By default, SQLITE_LOCKED and SQLITE_BUSY errors are not treated as failures, and the backup will continue if they occur. The set of errors that are tolerated can be controlled by setting `backup.retryErrors`. To disable automatic finishing and stick strictly to sqlite's raw api, set `backup.retryErrors` to `[]`. In that case, it is necessary to call `backup.finish()`. In the same way as node-sqlite3 databases and statements, backup methods can be called safely without callbacks, due to an internal call queue. So for example this naive code will correctly back up a db, if there are no errors: ``` var backup = db.backup('backup.db'); backup.step(-1); backup.finish(); ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I wanted to have the backup support from sqlite3 exposed in node-sqlite3 for a project I am working on. Here is a link to the documentation for this API: Using the SQLite Online Backup API. I have included a unit test that just makes a backup of an existing database in the test/support directory. I only tested this on my ArchLinux server.