Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Add par2 parity files and auto repair to backends #3879

Closed
wants to merge 37 commits into from

Conversation

BlueBlock
Copy link
Contributor

@BlueBlock BlueBlock commented Aug 31, 2019

(I'm putting this out to get early feedback. This top-post will be updated with all progress.)

Still is a work-in-progress

Summary:

This PR adds support for adding parity files to all files uploaded to a backend.

One option has been added to control the use of parity files.

  • parity-file-redundancy: represents the percentage of par2 redundancy (value 0 through 100, a default of 0, a value of 0 disables use of parity files)

Additional discussion on the topic can be found here.

To be done:

  • need solution for adding par2cmdline functionality to linux (manual install by user?)

Features:

  • all backends are supported
  • (Windows) par2cmdline added to provide par2 functionality
  • a backend file has an individual parity file, a 1:1 relationship
  • a parity file is named after the source backend filename + ".par2.zip"
  • a parity file are generated and accompanies the upload of the backend file
  • a parity file removed when the source backend file is removed
  • an automatic repair attempt will be made when source backend file is damaged
  • a repaired source backend file will be re-uploaded and replace the corrupted source backend file
  • parity files can be enabled on existing backups and parity files uploaded to backend (TBD)

@BlueBlock BlueBlock changed the title WIP Add par2 parity files and checking to backends WIP Add par2 parity files and repair to backends Sep 1, 2019
@BlueBlock BlueBlock changed the title WIP Add par2 parity files and repair to backends WIP Add par2 parity files and auto repair to backends Sep 2, 2019
@warwickmm warwickmm added the DO NOT MERGE don't merge label Nov 23, 2019
@duplicatibot
Copy link

This pull request has been mentioned on Duplicati. There might be relevant details there:

https://forum.duplicati.com/t/failed-invalid-header-marker/2871/44

@warwickmm
Copy link
Member

Closing this due to lack of activity. We can revisit this later if necessary.

@warwickmm warwickmm closed this Jul 8, 2020
@duplicatibot
Copy link

This pull request has been mentioned on Duplicati. There might be relevant details there:

https://forum.duplicati.com/t/reliability-of-duplicati-2-0/10435/6

@duplicatibot
Copy link

This pull request has been mentioned on Duplicati. There might be relevant details there:

https://forum.duplicati.com/t/btrfs-as-a-filesystem-for-the-backup-disc/12614/2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DO NOT MERGE don't merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants