Upgrade scylla major version

[dahankzter]

Currently we employ a check that the scylla version of the target state is the same as the previous.
This prohibits an upgrade and we need to device a way to allow the user to upgrade.

[dahankzter]

What do you think @tzach should we put this on 1.0? Right now there is coded logic preventing an upgrade. Removing it requires some care and validation to allow for a consistent upgrade.

[tzach]

I do not think this is very urgent.
We can push it, unless its trivial to add.

[dahankzter]

I think perhaps middle ground would be safe and fairly easy. That would be to allow upgrade within a major version. We can perhaps handle major upgrades later although that is also a matter of verification that the user knows what they are doing.

[dahankzter]

This has potentially a sizeable amount of work what with snapshots being needed between major versions and some procedure for restoring to an older version. Let's postpone it as we discussed.

[dahankzter]

Perhaps a scheme suggested by @penberg can work.

We allow a user to upgrade to the latest minor version and only then do we allow them to upgrade
to a new major version. Essentially this gives them the possibility to upgrade to any minor version but only increase to new major version in case they are on the latest minor version.

How do we check this? We use docker hub REST API https://docs.docker.com/registry/spec/api/
from which we can deduce the appropriate rules.

Did I get that right @penberg? What do you think @tzach ?

[gnumoreno]

I know that upgrades sound simple but they are not. I would avoid major upgrades on the operator until this is handled by scylla-manager itself. There are a lot of requirements.

[dahankzter]

After my discussion with @penberg the other day it got much clearer. Is the above strategy not correct? More or less. Waiting for Manager to do upgrades is a long way to go and also an orthogonal feature that I am not sure we want in Manager at all. At least not initially.

[espindola]

I am not sure I understand why requiring an upgrade to the last minor makes things simpler for the operator. I can understand why it might be a good thing for scylla overall (less combinations for QA), but as long as we support upgrading the major version, knowing that we have the latest minor doesn't make things simpler as far as I can tell.

We also only support backups with the scylla manager, no? How would we handle upgrades without the scylla manager? Do we implement backups directly first?

Is this really a noob task?