New user, new ideas

[Tassleh0ff]

Hi @nicotsx, fist of all thank you for Zerobyte.

I use restic almost since day one, and not only on my personal hosts but also at work, I literally filled datacenters with restic and its backups proven to be the most reliable, fast and flexible compared to also enterprise backup services I worked with (Commvault, Tivoli Storage Manager, Cohesity, EMC2 Data Domain, etc etc...) in almost 30 years of experience as professional sysadmin.

I tried Zerobyte and man, it's a really neat service that fills a lot... I mean A LOT of gaps that restic left open, and I don't mean only a nice UI but specially good notifications and a centralized management interface.

In the test I'm running on my home lab (a very simple RPi5) I'm using SFTP for volume mount and Backblaze B2 as backup repository.
I have a few suggestions, some minor but a big one that involves a radical change in Zerobyte infrastructure; I would like to focus on Zerobyte as centralized management platform for restic backups on several hosts on several regions or locations.

Recently I found another project that has many things in common with Zerobyte, it's called Portabase and its objective is to do backups for rdbms with a centralized architecture.

Sudo for sftp volume mount
One of the first problem I got is that most of the times restic has to run with superuser provileges to be able to backup critical data.
For example on my hosts I always schedule a backup of every filesystem starting from /, because this makes possible to rebuild a host from scratch in case of a complete disaster (just like the good old "full system tar backup" that saved generations of sysadmins).

In most of the case an ssh connection using directly root is not possible for many reasons (security, policies, etc etc..), that's why this sftp fuse approach has many problems imho.

Dashboard
I think the main screen once logged on Zerobyte should be a dashboard that shows the backup results and the status of the volumes, repositories, backups and notification services. Take a look to Portabase for an example.

Restic mount
It would be awesome to mount a backup repository (just like the "restic mount" command) on a specific path defined as bind volume on the docker host.

For example
In the docker compose manifest define a bind volume to use a base path to mount restic repositories

    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data:/var/lib/zerobyte
      - ./restic-mount:/mnt/restic

Mount a restic repo inside the Zerobyte container on the /mnt/restic/repo-mount (restic -r /mnt/restic/repo-mount)
Browse the mounted restic repo on the docker host in /restic-mount/repo-mount

The region problem
I work a lot on AWS EC2 instances, and I use several AWS regions.
With restic I usually use S3 buckets as restic repositories, and usually I choose buckets on the same regions as the EC2 instances, simply because the network traffic between EC2 instances and S3 buckets on the same region is free, otherwise I'll pay a lot of money only for network transfers.

As you can imagine in a similar scenario using Zerobyte makes me move everything between regions.
Hosts on the same region with the Zerobyte container (and using an S3 bucket on the same region as well) will cost nothing in terms of network traffic.
Doing a backup of hosts on a different region means paying AWS a lot of money for network transfers between regions (and obviously in that case I will use an S3 bucket in the same region with the Zerobyte host).

This could be solved with the next point, the big one.

The big one
Have you ever thought to use a different architecture? Something like the Portabase architecture.

• A centralized Zerobyte dashboard where users will log on to define backups, schedule them, configure notifications and define repositories.
• An agent for each host you want to backup, securely connected to the Zerobyte dashboard, running as daemon with superuser privileges (no more fuse, no more root sftp connections, no more "sudo for sftp volume mount" problem)
• Restic backups will be triggered by the centralized dashboard but run on each host independently, basically running restic directly on each host to backup by the Zerobyte agent
• Each agent will run the backup and transfer the backup data to the proper restic repository defined in the dashboard; no more backup transfers done by the Zerobyte Dashboard (unless you want to use the Dashboard local storage as backup repository).
  For example:

I perfectly understand that the last point is a complete different architecture compared to the actual project (as far as I understood, maybe I'm wrong), but it will solve almost any problem and will make Zerobyte jump from a nice local solution to a real centralized backup solution ready for enterprise environments.

Again thanks for you work, I hope these ideas could be useful for the future evolution of your project.

[nicotsx]

Hello @Tassleh0ff thank you very much for all your ideas! I'm aligned on many things you have suggested. And I'm happy to tell you that your last point, the big one, is my primary focus since a few weeks. I even have a working version locally that still needs a lot of polishing but this is in the making and will be out as soon as my free time allows

[Tassleh0ff]

Hi @nicotsx, fantastic! I cant' way to test it!