The shutdown logic needs to be updated to trigger a dump at the shutdown unless the SHUTDOWN NOSAVE command has been issued #316
Projects
Comments
danielealbano
added a commit
that referenced
this issue
Apr 9, 2023
This PR implements the necessary support to generate the RDB snapshots
in the background.
A new fiber has been introduced which takes care of generating a
snapshot, depending on the settings. A new set of parameters have been
introduced, example below with explanation in line.
```
snapshots:
# The path where the snapshot file will be stored, if the rotation is enabled, the path will be used as prefix and
# the timestamp of the start of the snapshot will be appended to the file name.
path: /var/lib/cachegrand/dump.rdb
# The interval between the snapshots, the allowed units are s, m, h, if not specified the default is seconds.
interval: 5m
# The number of keys that must be changed before a snapshot is taken, 0 means no limit
min_keys_changed: 1000
# The amount of data that must be changed before a snapshot is taken, the allowed units are b, k, m, g, if not
min_data_changed: 100mb
# Rotation settings, optional, if missing the snapshots rotation will be disabled
rotation:
# The max number of snapshots files to keep, minimum 2
max_files: 10
```
The example is from cachegrand.yaml.skel
The new mechanism takes care of reporting every 3 seconds a status
update.
Closes #293
Notes:
- Currently the redis commands SAVE and BGSAVE are not implemented, so
it's not possible to trigger a backup on demand (#314)
- The SHUTDOWN command needs to be updated to support SAVE and NOSAVE
(#315)
- The shutdown logic needs to be updated to trigger a dump at the
shutdown unless the SHUTDOWN NOSAVE command has been issued (#316)
- The current implementation also doesn't compress strings with the LZF
algorithm as liblzf is causing segfault and the issue has to be
investigated (#312)
- Tests for the high level snapshotting process (implemented in
storage_db_snapshot.c mostly) are not included in this PR (#317)
danielealbano
added a commit
that referenced
this issue
Apr 22, 2023
This PR introduces the ability to trigger a snapshot when the workers start to shutdown. To achieve this quite some changes have been done to the workers events loops: meanwhile before the initialization and shutdown of the internal database and the networking could have performed before entering the loop and at its end, with the need to be able to perform I/O the loop has still to run at the shutdown but it's necessary to terminate all the connections and pending operations. The PR introduces the required changes and therefore it's now possible perform operations both at the startup and at the shutdown, within the loop! The PR also introduces a new field in the snapshot structure to keep track of the number of iteration performed, very handy to check if a snapshot has been taken, used now internally. A number of additional fixes have been introduced, e.g. to better sync the shutdown of the signal handler thread or of the epoch gc workers., better tests for the SAVE and BGSAVE commands, improve tests for SHUTDOWN, etc. Closes #316
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The shutdown logic needs to be updated to trigger a dump at the shutdown unless the SHUTDOWN NOSAVE command has been issued
The text was updated successfully, but these errors were encountered: