Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: Use reed-solomon encoding to reduce data size #17

Open
Hiddendoom45 opened this issue Aug 20, 2020 · 1 comment
Open

Suggestion: Use reed-solomon encoding to reduce data size #17

Hiddendoom45 opened this issue Aug 20, 2020 · 1 comment

Comments

@Hiddendoom45
Copy link

The current implementation of the program appears to copy the encrypted file into each horcrux, which becomes pretty storage intensive with a larger file and more horcruxes.

n is the number of horcruxes, t the number needed to ressurect, s is size of the file, t ≤ n
Reed-solomon encoding can split the original file into n pieces, each piece has a size of s/t. Any combination of t pieces can be used to recreate the original file.

So the total space used by the horcruxes would be n*(s/t) instead of n*s.

This should also ensure the integrity of the data as the reconstruction of the file should fail if a piece is modified.

@wdhwg001
Copy link

A possible solution is to use par2: https://github.com/Parchive/par2cmdline

With the current master branch version, you may create redundancy > 100%.

Example: par2 create -r167 -u -n5 diary.txt.par2 diary.txt, or any -r larger than 100/t*n but smaller than 100/t*(n+1).

It works as a rough replacement of horcrux -n 5 -t 3 split diary.txt but you will get smaller files you want.

Then simply delete the extra diary.txt.par2 file, keeping only diary.txt.volXXXXX.par2 files.

As a result, you may delete any 2 files, the remaining 3 files can still restore your diary.txt safely.

However, please DO NOTE THAT REED-SOLOMON IS NOT SECURE AND YOU NEED TO ENCRYPT YOUR DIARY BEFORE SPLITTING WITH A PASSWORD.

@stokebreakup stokebreakup mentioned this issue May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants