Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Live channel database backup #951

Merged
merged 17 commits into from Apr 19, 2019

Conversation

Projects
None yet
2 participants
@sstone
Copy link
Member

commented Apr 17, 2019

Backups are handled by a custom actor that listens to persistence events sent by channels. They are done in 2 steps:

  • create a sqlite backup in a temporary file
  • rename the temporary file, which should be atomic (though it actually depends on the OS and file system).

Users would typically rename/move the backup file then make a safe copy of it.

Backups are sequential. If backup requests pile up while a backup is still in progress, only the last one will be kept.

sstone added some commits Apr 16, 2019

Backup running channel database when needed
Everytime our channel database needs to be persisted, we create a backup which is always
safe to copy even when the system is busy.
@pm47
Copy link
Member

left a comment

A few bikeshed comments, overall looks good to me. The PR is missing assumptions that we are making on the number of channels.

Also this should include an update to the documentation.

Show resolved Hide resolved eclair-core/src/main/resources/reference.conf Outdated
Show resolved Hide resolved eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala Outdated
Show resolved Hide resolved eclair-core/src/main/scala/fr/acinq/eclair/db/BackupHandler.scala Outdated
Show resolved Hide resolved eclair-core/src/main/scala/fr/acinq/eclair/db/BackupHandler.scala Outdated
Show resolved Hide resolved eclair-core/src/main/scala/fr/acinq/eclair/db/BackupHandler.scala Outdated
context become idle

// we use the pipe pattern so errors are wrapped in akka.Status.Failure
case Status.Failure(cause) if again.isDefined =>

This comment has been minimized.

Copy link
@pm47

pm47 Apr 17, 2019

Member

Have you considered using a dedicated thread for this actor instead?

Show resolved Hide resolved eclair-core/src/main/scala/fr/acinq/eclair/db/BackupHandler.scala Outdated
Show resolved Hide resolved eclair-core/src/main/scala/fr/acinq/eclair/db/BackupHandler.scala Outdated
Show resolved Hide resolved eclair-core/src/main/scala/fr/acinq/eclair/db/Databases.scala
Backup handler: use a custom mailbox
This is an alternate design for handling multiple backup events: we use
a custom mailbox with a max capacity of 1 message (ignore messages go directly to dead-letters)
and use synchronous backups in the actual event handler.
@pm47
Copy link
Member

left a comment

More bikeshedding (time to use the new batch suggestion feature!)

pm47 and others added some commits Apr 18, 2019

Renamed backup handler parameters
Co-Authored-By: sstone <sstone@users.noreply.github.com>
BackupHandler: configure a specific mailbox
In our configuration we specify a custom bounded mailbox for our backup handler, which is
assigned by matching on its name.
Backup: make sure that our custom mailbox is always use
BackupHandler is now private, users have to call BackupHandler.props() which always
specifies our custom bounded maibox.
BackupHandler: minor changes
Remove tmp file paramater, we just add ".tmp" to the backup file

@sstone sstone marked this pull request as ready for review Apr 18, 2019

@pm47
Copy link
Member

left a comment

I think eclair.sqlite.snapshot would be a better name for the resulting file.

Show resolved Hide resolved README.md Outdated
Show resolved Hide resolved README.md Outdated

pm47 added some commits Apr 19, 2019

Show resolved Hide resolved README.md Outdated

pm47 and others added some commits Apr 19, 2019

Add backup notification script
Once a new backup has been created, call an optional user defined exe/script.
Run backup notification script in the backup handler's thread
We wait until it terminates and log its exit code.
@pm47

pm47 approved these changes Apr 19, 2019

@sstone sstone merged commit bed47de into master Apr 19, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@sstone sstone deleted the sqlite-backup branch Apr 19, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.