Create backups of persistency

[landauermax]

There should be a parameter for the command line that backups the persistency in regular intervals. Also, there should be a command for the remote control that saves the persistency when executed.

The persistency should be copied into a directory /var/lib/aminer/backup/yyyy-mm-dd-hh-mm-ss/...

There should also be the possibility to restore configs, by remote control, config settings, etc.

[ernstleierzopf]

I don't think it is smart or possible to reload the complete configs from an file as this happens in the aminer.py class. The program would not be stable if too many changes were made. The way to go is to restart the aminer with the old config file. Single config settings already can be changed. Restoring them is also possible by just reversing the values of the initial remote control command.

[ernstleierzopf]

Is it ok to create a seperate folder for the backups ([persistence_dir]_backup/yyyy-mm-dd-hh-mm-ss, for exampe /var/lib/aminer_backup/yyyy-mm-dd-hh-mm-ss?
Copying a whole folder into a subfolder is problematic with most of the existing python functions.

By creating an seperate folder it is possible to use shutil.copytree().

[ernstleierzopf]

what is the expected timespan for the backups? Currently once per day is implemented.

[landauermax]

I don't think it is smart or possible to reload the complete configs from an file as this happens in the aminer.py class. The program would not be stable if too many changes were made. The way to go is to restart the aminer with the old config file. Single config settings already can be changed. Restoring them is also possible by just reversing the values of the initial remote control command.

I am not sure if I understand the comment correctly, since this issue is not about the config, but only the persisted data.

[landauermax]

Is it ok to create a seperate folder for the backups ([persistence_dir]_backup/yyyy-mm-dd-hh-mm-ss, for exampe /var/lib/aminer_backup/yyyy-mm-dd-hh-mm-ss?
Copying a whole folder into a subfolder is problematic with most of the existing python functions.

By creating an seperate folder it is possible to use shutil.copytree().

I think this is what I suggested in the issue description. A separate folder that contains the persistencies, e.g., /var/lib/aminer/backup/yyyy-mm-dd-hh-mm-ss/NewMatchPathValueDetector/Default

[landauermax]

what is the expected timespan for the backups? Currently once per day is implemented.

Probably depends on the application case, but yes, i can imagine that in live systems once a day is sufficient.

[ernstleierzopf]

I don't think it is smart or possible to reload the complete configs from an file as this happens in the aminer.py class. The program would not be stable if too many changes were made. The way to go is to restart the aminer with the old config file. Single config settings already can be changed. Restoring them is also possible by just reversing the values of the initial remote control command.

I am not sure if I understand the comment correctly, since this issue is not about the config, but only the persisted data.

I understood the first comment as restoring the whole config. If that was not the case, everything is clear.

[ernstleierzopf]

Is it ok to create a seperate folder for the backups ([persistence_dir]_backup/yyyy-mm-dd-hh-mm-ss, for exampe /var/lib/aminer_backup/yyyy-mm-dd-hh-mm-ss?
Copying a whole folder into a subfolder is problematic with most of the existing python functions.
By creating an seperate folder it is possible to use shutil.copytree().

I think this is what I suggested in the issue description. A separate folder that contains the persistencies, e.g., /var/lib/aminer/backup/yyyy-mm-dd-hh-mm-ss/NewMatchPathValueDetector/Default

as /var/lib/aminer is the default persistence directory /var/lib/aminer/backup is an subfolder of that. Better is /var/lib/aminer_backup for example.

[landauermax]

No, this issue is not about restoring the config, only the persistency.

Please create a subfolder backup in the /var/lib/aminer/ directory. The reason for this is that creating a new directory /var/lib/aminer_backup may cause issues with permissions. /var/lib/aminer is the directory of the user aminer, which we assume for our process.

[ernstleierzopf]

Okay. I have changed that. shutil offers ignore_patterns which are helpful in this case.

[ernstleierzopf]

Please move this issue to Done.