Compared to others

Conserve aims to fill a good niche of being simpler and safer than Duplicity or Obnam, but more friendly to cloud storage than rsync.

The great thing about backup programs is, to some extent, you don't have to pick just one: for safety, make multiple backups with different tools and approaches. So this document is not to knock the other tools or to persuade you not to use them, but rather to explain why it's worth writing Conserve too.

Duplicity

Duplicity addresses a similar case of backup to cloud or dumb servers, and uses librsync delta compression. But it fails several Conserve manifesto items, some of them by design:

• Verification and restoring a single file requires reading the whole archive.

  • Somewhat inherent in having a layer at which the whole backup is represented as one tar file, which makes it hard to seek into the middle. Perhaps not impossible to fix, if it is possible to look at the middle of a tar file and seek forward until finding the next header? Could add an index of blocks to help with this: perhaps say which file and which position within that file.
  • rdiff also done at whole-tar level

• I have experienced Duplicity bugs that make the archive unreadable and, although it is based on simpler underlying formats (rdiff, rdiffdir, tar, etc) it is still hard in practice to recover.

• Performance is unpredictable and sometimes bad.

• for decent performance, requires sometimes doing a full backup; but in fact you may often have some files that never change and copying them all the way up seems redundant

• until the first full backup completes, you can't restore anything?

• can resume backups but this seems particularly buggy?

Obnam

obnam format too complex? Otherwise looks pretty good.

rsync

In some ways a very good option:

• fairly quickly makes incremental progress
• recovers cleanly and seamlessly after interruption - but not perfectly, since it does always start scanning from the start
• has this "piecewise-robust"

However, has some problems with writing to dumb cloud storage:

• requires a smart server
• stores files unpacked so can't encrypt and can only represent things the destination filesystem can represent (usernames, permissions, etc)
• no builtin incremental backup capabilities (though you can make hard-linked trees using some external script)

And not perfect at incremental operation:

• recent versions don't require sending a whole file list before sending files but it does always start scanning the whole filesystem from scratch, so a short run may not make progress