Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve snapshot scheduling interface #1590

Closed
romange opened this issue Jul 25, 2023 · 8 comments
Closed

improve snapshot scheduling interface #1590

romange opened this issue Jul 25, 2023 · 8 comments
Assignees
Labels
enhancement New feature or request good first issue Good for newcomers minor nice to have enhancement

Comments

@romange
Copy link
Collaborator

romange commented Jul 25, 2023

save_schedule flag is a bit confusing and not flexible enough.

Instead, we should introduce a more standard way of scheduling snapshots like cronjob expression.

this library can help with parsing and matching the events.
https://github.com/mariusbancila/croncpp

we should introduce a new flag (e.g. snapshot_cron?) and leave the old one to allow an easy transition before we deprecate it.

@romange romange added enhancement New feature or request good first issue Good for newcomers minor nice to have enhancement labels Jul 25, 2023
@eliskovets
Copy link

Hi @romange ,
Would it be possible to schedule saves only in case changes were made to the data?
In our case, we have a huge cluster that's mostly read-only, and we would like "save" to be triggered only if changes were made to the data.
The only possibility I see right now is to trigger bgsave when we finish writing data from the client code.

Thank you!

@kostasrim
Copy link
Contributor

Hi @eliskovets,

Would it be possible to schedule saves only in case changes were made to the data?

Currently you can either save a snapshot explicitly via bgsave or periodically via the save_schedule flag.

Probably the best way would be to trigger it explicitly each time a client writes the data to the server.

Moreover, is there a specific reason why you would like snapshotting on write trigger?

@romange
Copy link
Collaborator Author

romange commented Jul 28, 2023

I agree with @kostasrim here. It's a very narrow use case and it requires from us some additional tracking and wiring to implement. Seems that calling BGSAVE/SAVE is an adequate solution for this problem.

dor132 added a commit to dor132/dragonfly that referenced this issue Jul 30, 2023
Requested by dragonflydb#1590.
Introducing a new flag `--snapshot_cron`, which enables users
to use cron expressions to time snapshot saves.

Signed-off-by: Dor Avrahami <da19965@gmail.com>
dor132 added a commit to dor132/dragonfly that referenced this issue Jul 30, 2023
Requested by dragonflydb#1590.
Introducing a new flag `--snapshot_cron`, which enables users
to use cron expressions to time snapshot saves.

Signed-off-by: Dor Avrahami <da19965@gmail.com>
dor132 added a commit to dor132/dragonfly that referenced this issue Jul 30, 2023
Requested by dragonflydb#1590.
Introducing a new flag `--snapshot_cron`, which enables users
to use cron expressions to time snapshot saves.

Signed-off-by: Dor Avrahami <da19965@gmail.com>
@dor132
Copy link
Contributor

dor132 commented Jul 31, 2023

@romange This can be marked as closed now.

@romange
Copy link
Collaborator Author

romange commented Aug 1, 2023

That was quick 😀. Thank you!

@romange romange closed this as completed Aug 1, 2023
@romange romange reopened this Aug 1, 2023
@romange
Copy link
Collaborator Author

romange commented Aug 1, 2023

I would like to apply some changes.

@dor132
Copy link
Contributor

dor132 commented Aug 1, 2023

What are the changes?

@romange
Copy link
Collaborator Author

romange commented Aug 1, 2023

@dor132 I see you already saw - I think your proposal is good.

dor132 added a commit to dor132/dragonfly that referenced this issue Aug 2, 2023
Introducing a new flag --snapshot_cron, enabling users to use cronjob expressions to time snapshot saves.
Cronjob expressions are parsed using a third party library croncpp.
This PR continues dragonflydb#1599, updating cron expressions to crontab style,
up to minutes resolution instead of seconds.

Signed-off-by: Dor Avrahami <da19965@gmail.com>
chakaz pushed a commit that referenced this issue Aug 2, 2023
Requested by #1590.
Introducing a new flag --snapshot_cron, enabling users to use cronjob expressions to time snapshot saves.
Cronjob expressions are parsed using a third party library croncpp.
This PR continues #1599, updating cron expressions to crontab style,
up to minutes resolution instead of seconds.

Signed-off-by: Dor Avrahami <da19965@gmail.com>
@romange romange closed this as completed Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers minor nice to have enhancement
Projects
None yet
Development

No branches or pull requests

4 participants