Create and document backup & restore & vacuum process

[novoj]

We need to implement the vacuuming process, the backup & restore procedure. We want all of them to be executable while the system is running and writing to the original files without interruption. This requires special integration tests and manual testing. We should also partially rewrite the current storage logic where we use "nice" names for entity collections and catalogs. We (probably) need to avoid situations that require renaming existing files and use monotonically increasing indexes for the files and switch the reference to the currently used file only in storage data structures. Finally, if the system detects that there is no living pointer to the old file, it will be permanently deleted.

Vacuuming processes:

• small: when trash ratio exceeds limit ⇨ write memory snapshot to a new file with incremented index
• big: regularly delete old files when:
  • there is no active transaction working with the file (otherwise log error)
  • their last modification timestamp is lesser than the required history to be kept
  • compact catalog header file to track only those headers that relate to existing files

Backup process:

• small: creates ZIP file with:
  1. last record catalog header only
  2. current catalog content file
  3. current entity content files
• big: creates ZIP file with:
  1. entire catalog header file
  2. all catalog content files
  3. all entity content files

Ensures no vacuuming process is running and prevents it to be executed for particular catalog.

Restoring process:

• simple - just unzip entire catalog contents and load the contents from it
• chirurgical (can be used only on "big" backups) - it looks up into the ZIP file and finds catalog header that matches requested timestamp, if the catalog history is newer than the timestamp, error is printed with the information of the earliest timestamp, that can be used. If the particular header can be found, its contents are restored as a fresh snapshot without any waste; it accepts two arguments:
  • backup file name (mandatory)
  • timestamp the catalog should be restored in (optional - if not set, current date and time is used)

[novoj]

The API of the backup should be in form:

1. api call "backup start" -> backup_id
2. api call "backup status " -> queued, still running, finished on path: xxx
3. backup
   1. api call "backup download " -> backup archive (developers)
   2. direct backup from volume path (on the same host cluster) that can be derived from known location and
4. api call "backup release " -> deleted (or autoremoval after certain period - configurable)

[novoj]

During implementation the naming convention for data files has changed. We'd need full-reindex with this version.

[novoj]

Finally, implemented. Documentation will be added when the functionality is available through REST / GraphQL / Lab interfaces. This will be done in follow-up issue #627