Can be used for local backup?

[quevon24]

Thanks for the development of this feature, but I wonder if there is a way to make the backup locally? like /futurice/docker-volume-backup but keeping the file rotation.

[m90]

If you want local backups you could also run a local MinIO instance (which is very lightweight when idling most of the time) that does nothing but serve as the target for your backup and you'd get all current features (including rotation) out of the box, plus your backups will also be stored locally. You can check the integration tests in this repository for an example of how this works.

That being said, I am pretty sure the MinIO client could also rotate away files that are stored locally (as in no server) instead of remotely. Being honest, I am not super keen on adding this feature myself as I feel local backups defeat a very important reason for having backups, which is infrastructure failing, but I also see how it could be useful in some scenarios. If anyone wants to implement this I am happy to review and merge a PR that adds this feature.

[m90]

This is now possible as of v1.7.0 (latest as of now is v1.8.1 already). Please read the docs about how this could affect other files in the target dir before using.

[quevon24]

Thanks for this, I tested it and works fine, it created the local backup correctly.

I manually place an older file to test pruning, deletes file correcty but container log shows an error when deleting older files.

This is my docker-compose config:

backup:
    image: offen/docker-volume-backup:latest
    restart: always
    environment:
      #      BACKUP_ARCHIVE: "/archive"
      #      BACKUP_CRON_EXPRESSION: "* * * * *"
      BACKUP_RETENTION_DAYS: "7"
    volumes:
      # Mounting the Docker socket allows the script to stop and restart
      # the container during backup. You can omit this if you don't want
      # to stop the container
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - media_volume:/backup/media_volume:ro
      # If you mount a local directory or volume to `/archive` a local
      # copy of the backup will be stored there. You can override the
      # location inside of the container by setting `BACKUP_ARCHIVE`
      - /var/opt/mediabackups:/archive

volumes:
  media_volume:

Tested on windows with wsl2 and throw this:

[INFO] Creating backup
tar: removing leading '/' from member names
backup/
backup/media_volume/
backup/media_volume/README.md
backup/media_volume/uploads/
backup/media_volume/uploads/2021/
backup/media_volume/uploads/2021/07/
backup/media_volume/uploads/2021/07/20/
backup/media_volume/uploads/2021/07/20/test/
backup/media_volume/uploads/2021/07/20/test/bc3fe64b-96c2-4e95-9f23-faac9b5334ed.pdf
backup/media_volume/.gitignore
[INFO] Starting containers/services back up
Restarting e1520661de6d
[INFO] Copying backup to local archive
Will copy to "/archive".
`backup-2021-08-19T21-04-10.tar.gz` -> `/archive/backup-2021-08-19T21-04-10.tar.gz`
Total: 0 B, Transferred: 78.48 KiB, Speed: 69.18 MiB/s
Finished copying.
[INFO] Cleaning up
removed 'backup-2021-08-19T21-04-10.tar.gz'
[INFO] Backup finished
Will wait for next scheduled backup.
[INFO] Pruning old backups
Sleeping 10m before checking eligibility.
[INFO] Pruning old backups from local archive
Removing `/archive/mb_driver_542_w10_10.048.0315.2021.zip`.
mc: <ERROR> Failed to remove `/archive` recursively. remove /archive: device or resource busy
Successfully pruned 1 backups older than 1 days.
Restarting b42218719a05

Tested with ubuntu server 20.04 with file owner different from root, delete files correctly but same message:

[INFO] Pruning old backups from local archive
Removing `/archive/test-data.tar`.
mc: <ERROR> Failed to remove `/archive` recursively. remove /archive: device or resource busy
Successfully pruned 1 backups older than 1 days.

Tested setting filename to backup name, make it owned by root and same error. Deletes the file correctly but shows the same error.

Beside that message, works perfectly.

[m90]

Hmm, it seems there are subtle differences between how mc behaves on a local filesystem as compared to a remote bucket. I need to look into this.

[m90]

I implemented a different way of rotating local backups (using find instead of mc in #17) which is now released in v1.8.2. Maybe you could try updating your setup to that version and let me know if the error message is gone.

[quevon24]

Hi, i changed this: find $target* -delete -type f -mtime $BACKUP_RETENTION_DAYS to this: find $target* -type f -mtime $BACKUP_RETENTION_DAYS -exec rm -rf '{}' ';'

Because the other way deletes everything and throws /archive is busy

I submit a pull request with the change or if you can apply the change manually.

[lonix1]

I am not using kubernetes which MinIO seems to push. Can I still use it in the way you proposed above, together with this app?

[m90]

@lonix1 this issue is rather old and refers to an old version of the image. You can find extensive documentation on how.to do local backups using the image in the repository README.

[lonix1]

Wow you've been busy! So many new backup targets. Well done :)

Any plans for services like dropbox / box ?

[m90]

Any plans for services like dropbox / box

Not from my end but I wouldn't object to merging a PR that adds it. If you feel like you want to add it yourself, I am happy to assist in getting it merged, else feel free to create a new issue requesting the feature and maybe someone else wants to pick it up.