New backup stategies

[andrii29]

According to this lines https://github.com/AlexAkulov/clickhouse-backup/blob/master/main.go#L402-L416 there are two backup strategies:
- tree
- archive
It would be nice to add compressed_archive (tar.gz), because not always we have enogh space to "doube" the clickhouse data.
Also it would be nice to add option tmp_folder - path to templorary folder, because /tmp usually exists on disk with 5-10 Gb space.

[AlexAkulov]

Hello!
The most economical way it is the 'tree' strategy. Files that were created on the 'tree' strategy is hard links and they don't spend your space in fact. You can create a new backup every day and although tools like 'du' show that sizes of every backups equals to a full size of the database but in fact their size is a size of the difference between freezes.
I'm thinking of removing the 'archive' strategy in future. You can create a 'tree' backup and archive with any tool afterwards.

[AlexAkulov]

I rewritten the 'archive' strategy and it satisfied all of your requests:

• now tar packaging is being executing during uploading and extracting during downloading in memory and doesn't create temporary files on the disk
• you can choose one of the 5 compressing formats: 'lz4', 'bzip2', 'gzip', 'sz', 'xz' and compression level in config file. By default 'lz4' with level 1 is used. You can see benchmarks and comparison here https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO

[andrii29]

Thank you!

[andrii29]

@AlexAkulov Hello,
it looks like new archive strategy is using only one cpu core. It works very slow and can't be used for terabyte range backups (gzip, level 7)

[AlexAkulov]

Hello! Yes, it is. Please, create Issue. ср, 3 апр. 2019 г. в 11:11, andrew.k <notifications@github.com>:

…

@AlexAkulov <https://github.com/AlexAkulov> Hello, it looks like new archive strategy is using only one cpu core. It works very slow and can't be used for terabyte range backups [image: image] <https://user-images.githubusercontent.com/31315876/55456690-68c89a80-55f0-11e9-86c3-c40b2d562ad1.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGznF14OGmG_KPc-AiIzmxN0cDgAZfl1ks5vdEYugaJpZM4cKvhw> .