Restoring a database state require to start from a known data state and apply every missing insert/delete/update operation. A checkpoint is exactly such known state.
HerdDB can be tuned to produce checkpoints as fast as possible or as clean as possible. A fast checkpoint will block write operations less time, conversely a clean checkpoint optimize memory usage (less memory pages contains less dirty data) and speedups searches (less page io is needed). For good performance a balance must be found between fast and clean.
server.checkpoint.period: time in milliseconds between scheduled checkpoints, it defaults to 15 minutes.
server.checkpoint.duration: whole checkpoint duration target time in milliseconds. HerdDB will try to cope with this limit when checkpointing tables. Note that is a soft limit, checkpoint will last more if needed. Defaults to no duration limit.
server.checkpoint.compaction: checkpoint page compaction target time in milliseconds. When checkpointing HerdDB will try to compact too small pages, this configuration limits the time dedicated to page compaction. Defaults to 1 second.
server.checkpoint.page.dirty.max.threshold: maximum dirty bytes percentage at which a pages will be considered for rebuild during a checkpoint. This value must be between 0 and 1.0, defaults to 0.25.
server.checkpoint.page.fill.min.threshold: minimum byte fill percentage at which a pages will be considered for rebuild during a checkpoint. This value must be between 0 and 1.0, defaults to 0.75.
Limiting checkpoint duration
The best way to limit checkpoint duration is tune the parameter 'server.checkpoint.page.dirty.max.threshold': lower values will produce longer but cleaner checkpoints.
Checkpoint duration can be limited by the 'server.checkpoint.duration' parameter too but be aware that configuring this parameter to small values could impact performances on the long run increasing pages pollution with dirty not reclaimed records: in many cases is safer to configure a wider dirty page threshold.
The property 'server.checkpoint.compaction' will terminate early page compaction (it couples with 'server.checkpoint.page.fill.min.threshold' parameter), during normal operations very few pages will be small but can be used to tune long checkpoints time after a data page size reconfiguration.
Checkpoints and dumps
When a data dump is requested such as by effect of a backup, a fully clean checkpoint is needed. To generate such checkpoint any dirty threshold or duration limit will disregarded.