Consider an adaptive algorithm by default

[debuglevel]

I really like loomchild/volume-backup because it just gets the job done. The only thing I usually stumble accross is the bzip2 default. In my cases, it just eats lots of CPU time without any real benefit. If I do not backup to a file, but copy a volume to another host (on the same 1GBit network), I usually look at htop asking myself why I did not pass -c none.

I wondered if there is any adaptive compression algorithm around which automagically knows whether it should compress --fast or --best in order to provide maximum throughput. And it seems that Zstandard actually implemented this: facebook/zstd#75

As @loomchild mentioned in other issues, it's unlikely something other than bzip2 will be the default due to backwards compatibility. If you ever consider to change it, Zstandard with --adapt might actually be a good choice.

For the time being, it might be a nice feature to pass the --adapt argument to zstd, if possible. (Notice: I didn't do any tests; I just assume --adapt works as nice as described).

[loomchild]

Hi.

It's an interesting suggestion. I tried to implement it, but I encounter issues with escaping parameters (--adapt option is passed to tar instead of zstd). I switched to bash shell and used arrays to resolve the issue, but I am worried that such big change will cause regressions. Therefore, I need more time for testing it.

If you have an idea for a quick solution that doesn't require rewriting large part of the script, please share.

[loomchild]

I finally managed to work on it. zstd now runs with adapt options. Please let me know what do you think.

[debuglevel]

Awesome! Will try it out the next time :)

[loomchild]

I have reverted this change since it decreased performance of zstd compression (see above issue).

[debuglevel]

That's a bummer. Thanks for investigating!