[ENH] Document in which ways 2.x is incompatible with 1.x

[t9t]

Is your feature request related to a problem? Please describe.

I have been using rdiff-backup 1.x for many years. Yesterday when I installed rdiff-backup on a newly installed Ubuntu 20.04 it installed 2.x. So I decided to also update my other systems to 2.x.

However, I quickly found out about some incompatibilities between the two.

In one instance, I was hit by #357 which unfortunately grew that particular backup from a few dozen gigabytes to over a hundred gigabytes and had to nuke it because it proved too difficult to restore the state of the backup in a usable way.

In another instance I use --include-globbing-filelist include.txt --exclude "**" "D:/" ... which previously would backup everything mentioned in the include file, but now seems to have cleared the entire backup destination (and moved all the previously present files into the rdiff-backup-data directory. Again, this also seems too difficult to fix so I'm nuking that one too and starting it from scratch.

There might be other - major or minor - incompatibilities that I'm unaware of.

Describe the solution you'd like

I would like a big warning, perhaps on the home page or on the Github releases page, that extreme caution should be taken when upgrading an existing situation from 1.x to 2.x

Furthermore, it seems prudent that when rdiff-backup version 2 detects it's going to operate on a version 1 backup destination, it will refuse to start by default unless a specific command line argument is provided. This will trigger the user to be extra cautious, examine the situation, perhaps do a few isolated tests, instead of accidentally ruining their backup destination.

Describe alternatives you've considered

Honestly, after my two primary backup locations being compromised, the alternatives I'm considering are... other backup programs. :) Especially since I now basically have to "start over".

While rdiff-backup 1 was very old and has a few bugs as well, at least it was stable to enough for me to use for many years without major issues (though I'm not sure how much that can be attributed to luck and how much it was "actual" stability). My first feelings after using rdiff-backup 2 are that it's immature and maybe should have been released in beta/experimental to start with. That way, I would have acted more cautiously as well and done more "off-line" tests before using it on my actual backups (though perhaps my assumption that it would "just work" and be fully backwards compatible was misguided).

Additional context

Rdiff-backup is a very complex tool due to the great number of features it has. At some point I start wondering, does it perhaps have too many features that can make it confusing to use (to users, but also perhaps developers) and make it more prone to bugs. Just as an example, there are so many different ways to specify what to include and exclude that a decade ago when I started using it, I had to invest some serious time to understand them all and set up a proper command line for my backups. Now some of these seems to have changed (perhaps on purpose, perhaps accidentally) and I have to invest time again to see how it all actually works...

[ericzolf]

I'm sorry to hear that you've got problems upgrading, but indeed upgrading requires tests. We've released 3 beta versions and one release candidate, and called for our users to test before we released. We have fixed all regression tests we've been made aware of, and a few more already present in 1.2. We can only be as good as our users make us, we have a pipeline and we improve it as we go, but we have no army of testers, and we don't get paid for our work on rdiff-backup.

Doing a roll-back is possible and was discussed on the mailing list, I've now even added an FAQ for it in #435

Since 2.0 we've already created two more bug-fix releases, doing our best to come on-par to what 1.2 did (in other words, undocumented changes are accidental).
And if you'd create a specific bug issue for --include-globbing-filelist include.txt --exclude "**" "D:/" with slightly more details, I'd look into it (a quick test didn't show any anomaly, probably the exact content of the include file is important).

Last but not least, you're not wrong writing that the code might be too complex, and this is what we're addressing as our next step: simplifying, improving, modernizing the code, but we couldn't do it without having first a solid basis, working on Python3, with a proper testing pipeline, and doing the same as version 1.2 (bugs still being possible of course).

[t9t]

Thanks for getting back to me so quickly.

I understand that it's a difficult position to be in. While rdiff-backup probably has a lot of users, probably not a lot of them are involved with its development or testing of newer versions unfortunately. This can make it difficult to identify issues before making a release.

But I would also like to ask you to view this from my perspective. I use dozens pieces of software, some which bump major versions multiples times a year (eg. web browsers) while others maybe only increase a major version once a decade or even less often. It is difficult to keep track of the development of all that software, so typically I rely on the fact that if my software update offers an upgrade, it is probably compatible (especially when upgrading from one Ubuntu LTS to another LTS). I did have a glance at the website when I noticed my server upgraded to 2.x, to see if I could safely start using it in place of 1.x and didn't really notice an indication that I should be extra careful or do some additional tests beforehand.

All in all, rdiff-backup is probably used by a lot of people, and it's a tool that deals with sensitive data. So that is where my suggestion to fail when the backup destination is an older version comes from too: to prompt the users to be careful when upgrading, making a conscious decision to do so, and perhaps even as a way to get them involved in testing of the new version.

As a software developer myself, I know how delicate complex software with a lot of logic can be. And in such software, making big improvements can often be extremely risky, with a big chance of breaking some (or a lot) of that logic. Especially if the old code base perhaps is not so nicely written a lot of unintentional bugs can creep in. As I mentioned before, in particular when a new developer joins a project, unaware of all the history of the code or maybe ignorant of some of the finer implementation details, the chance of introducing a bug here or there increases again.

Having said all that, I do appreciate all the effort you put in. I can imagine it's not an easy task to pick up such a project with a long history, learn everything that is inside, and try to improve it while also trying to add additional tests and making sure nothing breaks.

Thanks for providing a link with information on how to roll back a backup destination! I will have a look and see if I can restore my botched backups using that.

I will also create an issue for the case where my backup destination got cleared. One final thought I want to share about that: if rdiff-backup is going to be overhauled and a lot of improvements are going to be made, perhaps a closer look could be taken at the user friendliness of the file selection. Just this morning I realised, why is there even a distinction between "globbing" and not-globbing... if a user enters a * then most likely they would like to have it match a wildcard, right? How many people actually have file names with asterisks in them? And simple escaping (\*) could then be used to disable wildcards for certain rows, just as in most other tools.

Since I use rdiff-backup a lot, also from some Windows computers to a Linux server, I will try to be more active reporting issues when I see them from now on too.

[ericzolf]

It's an interesting discussion with a lot of threads, so let me try to answer quickly the most important ones:

1. we released version 2.0 with the confidence that we had done our best to avoid incompatibilities (at the repository and command line level, not at the client/server level), so, yeah, at some point in time, we need to release.
2. we got only reasonable feedback once 2.0 was released, so it's a bit an act of balance. Perhaps we can do better next time, but the right balance isn't easy to get: either we don't release officially, and it isn't broadly used aka tested, or we release broadly and "shit happens".
3. regarding the selection bug, please use the latest version 2.0.5 to be sure it hasn't been fixed.
4. regarding the globbing/selection issue, I fully agree with you and want to improve, but on the other hand, we need to keep the compatibility with older versions, especially in the client/server context, so that people can upgrade one side then the other without urgency (that was also a clear feedback from our users). Just saying that it won't be done tomorrow.
5. we could definitely use some more developers/testers on the Windows side; as Linux aficionado, I struggle to put the same passion in Windows 🙄 It would especially be cool if someone could make the test suite and pipeline work under Windows; I'd feel a lot better about it to be honest. I've started in an outdated feature branch and I'll do it if nobody else does it, but it's obviously time lost for other things.

[t9t]

Thanks for sharing your insights. As an outsider I would personally don't mind if the file selection options change in the command line tool at some point, if it meant it's easier to understand and rules would be simpler. But then again there are thousands of users who might not feel the same way.

I did some more testing to pinpoint the issue I faced that which wiped my backup destination, but unfortunately it seems like it was a case of user error/PEBCAK. My backup script had --include-globbing-filelist include.txt --exclude '**' which printed some errors about the contents of include.txt (which I will still track down and create an issue for if it's still a problem, also in the latest version). After I realised the globbing patterns in include.txt were unnecessary, I changed them to just directory paths and changed the command line to --include-filelist include.txt --exclude '**'. This I should definitely have tested elsewhere first before running on my actual backup! This wiped my destination, and after testing it seems this is consistent with the version 1.2.8 that I was using before.

So since both of the issues I described at the start of the issue are not actually issues any more at all, perhaps it's better to close this issue.

As for Windows, perhaps I can see if I can help out. But it would greatly help me get started if #347 was updated with a few notes on what has to be done, what's already done, and what is yet pending. The scope is a bit unclear to me right now (duplicate existing tests for Windows, fix tests that are already present but failing, etc).