[JUJU-3969] Allow upgrades without specifying version #15727

SimonRichardson · 2023-06-09T15:40:10Z

The following allows migrating to a new version of juju without explicitly stating the newer major version (3.x -> 4.x). Juju shouldn't have to know about new major versions, so to fix this requires removing the code that was recently added to pass the target controller version to the source pre-checks. In addition to this, I've also refactored the migration precheck context types to not allow target pre-check to explicitly know about model checking. This should prevent any target and source issues.

This is required as 3.1 was earmarked as the last LTS before releasing 4.0. The target controller will then validate where the model is coming from, and ensure it's within its own range before allowing it to migrate.

Checklist

Code style: imports ordered, good names, simple structure, etc
Comments saying why design decisions were made
Go unit tests, with comments saying what you're testing

QA steps

We require this PR one to land before we can fully test the workflow.

To ensure we didn't break any backward compatibility:

$ sudo snap install juju --channel 3.0/stable
$ juju bootstrap lxd test30
$ juju add-model default
$ juju deploy ubuntu
$ sudo snap remove juju --purge
$ make go-install
$ juju bootstrap lxd test31
$ juju switch controller
$ juju wait-for application controller
$ juju switch test30
$ juju wait-for application ubuntu
$ sleep 10
$ juju migrate default test31
$ juju switch test31
$ sleep 10
$ juju switch default
$ juju wait-for application ubuntu

SimonRichardson · 2023-06-09T15:40:37Z

upgrades/upgradevalidation/version_test.go

@@ -24,7 +24,6 @@ type versionCheckTC struct {
 	allowed bool
 	minVers string
 	err     string
-	patch   bool


Patch was removed, because it was set to true for every test.

upgrades/upgradevalidation/upgrade.go

hmlanigan · 2023-06-30T18:47:09Z

@SimonRichardson what has changed since we last talked? If this will be done in pieces, what are they?

upgrades/upgradevalidation/upgrade.go

The following allow migrating to a new version of juju without explicitly stating the newer major version (3.x -> 4.x). Juju shouldn't have to know about new major versions, so to fix this, we check that it's not in the version map, before checking if we're moving to a larger major version. This is required as 3.1 has been ear marked as the last LTS before releasing 4.0. The target controller will then validate where the model is coming from, and ensure it's with it's own range before allowing it to migrate.

So that others can do the review and approve while I"m on vacation.

The prior code had logic from the 2.9 branch. We can remove the branching logic and just rely on the fact that the current version can never downgrade.

upgrades/upgradevalidation/migrate.go

This removes the hack to upgrade version, instead just remove all source checks for source requests.

The following removes the target version from source prechecking. The source check should __never__ know about the version it's moving to. The source doesn't care. The target cares. Removing this logic is the first step of cleaning all of this up. Inreality we should send a big set of information for the target controller to validate. Nothing should be checked on the source.

SimonRichardson · 2023-07-03T14:22:23Z

upgrades/upgradevalidation/migrate.go

 ) []Validator {
-	validators := []Validator{
-		getCheckTargetVersionForModel(targetVersion, MigrateToAllowed),


This line is the real meat of it. We no longer check the source for the target version. This is totally up to the target to check for.

Can MigrateToAllowed be deleted?

SimonRichardson · 2023-07-03T14:22:58Z

upgrades/upgradevalidation/migrate.go

-	if targetVersion.Major == 3 {
-		validators = append(validators,
-			checkNoWinMachinesForModel,
-			checkForDeprecatedUbuntuSeriesForModel,
-			getCheckForLXDVersion(cloudspec),
-		)
-		if targetVersion.Minor >= 1 {
-			validators = append(validators, checkForCharmStoreCharms)
-		}
-	}
-	return validators


We're already at 3.1 so these branches can just be removed.

wallyworld

Is a separate PR doing the version validation on the target controller? We would need to do that up front before any model import is done. We'd lose the advantage of not having to bother to quiesce the source model, unless we added a PreCheck() method on the target controller or something like that.

wallyworld · 2023-07-04T08:20:25Z

migration/types.go

+// Copyright 2023 Canonical Ltd.
+// Licensed under the AGPLv3, see LICENCE file for details.
+
+package migration


Other packages call this file interface.go
Maybe consistency is best?

wallyworld · 2023-07-04T08:21:06Z

upgrades/upgradevalidation/migrate.go

 	environscloudspec "github.com/juju/juju/environs/cloudspec"
 )

-// ValidatorsForModelMigrationSource returns a list of validators to run source controller for model migration,
+// ValidatorsForModelMigrationSource returns a list of validators to run source


... to run on the ...

wallyworld · 2023-07-04T08:24:38Z

upgrades/upgradevalidation/migrate.go

 ) []Validator {
-	validators := []Validator{
-		getCheckTargetVersionForModel(targetVersion, MigrateToAllowed),


Can MigrateToAllowed be deleted?

SimonRichardson · 2023-07-04T08:35:52Z

@wallyworld 👇

Is a separate PR doing the version validation on the target controller

There is always a check on the target controller. https://github.com/juju/juju/blob/3.2/migration/precheck.go#L258

Can MigrateToAllowed be deleted?

No, because we need to check this in the target precheck for the source-to-target validation.

This just removes the code that the target was already checking.

hmlanigan

Problem in QA

hmlanigan · 2023-07-05T20:54:27Z

upgrades/upgradevalidation/upgrade.go

-			validators = append(validators, checkForCharmStoreCharms)
-		}
+	if targetVersion.Major == 3 && targetVersion.Minor >= 1 {
+		validators = append(validators, checkForCharmStoreCharms)


I'm not hitting this check. Migration doesn't fail until here:
machine-0: 20:44:19 ERROR juju.worker.migrationmaster.a98b55 model data transfer failed, failed to migrate binaries: cannot upload charms: cannot upload charm: Post https://10.186.60.233:17070/migrate/charms?arch=&name=ubuntu&revision=24&schema=cs&series=&user=: cannot upload charm: unsupported schema "cs"

~~Yeah because you're not allowed cs charms in 3.1. So this is correct.~~

So I did some investigation. This is entirely correct for what we have now. If you try and migrate from 2.9 to 3.1 there is no check at all, on the source side.

juju/upgrades/upgradevalidation/migrate.go

Lines 13 to 28 in 02d4986

func ValidatorsForModelMigrationSource(

targetVersion version.Number, cloudspec environscloudspec.CloudSpec,

) []Validator {

validators := []Validator{

getCheckTargetVersionForModel(targetVersion, MigrateToAllowed),

getCheckUpgradeSeriesLockForModel(false),

}

if targetVersion.Major == 3 {

validators = append(validators,

checkNoWinMachinesForModel,

checkForDeprecatedUbuntuSeriesForModel,

getCheckForLXDVersion(cloudspec, minLXDVersion),

)

}

return validators

}

The target side will only ever validate from 3.1 onwards

juju/upgrades/upgradevalidation/upgrade.go

Lines 21 to 31 in d3c8991

if targetVersion.Major == 3 {

validators = append(validators,

checkMongoVersionForControllerModel,

checkNoWinMachinesForModel,

checkForDeprecatedUbuntuSeriesForModel,

getCheckForLXDVersion(cloudspec),

)

if targetVersion.Minor >= 1 {

validators = append(validators, checkForCharmStoreCharms)

}

}

So this PR doesn't actually break any existing flow from 2.9 to 3.1. It's currently been this way. There is no way to get a cs charm into 3.1, so we could actually remove the check on the source side as well.

Here are my steps to reproduce exactly the same thing as this comment:

$ snap install juju --classic --channel=2.9/stable $ /snap/bin/juju bootstrap lxd src29 $ /snap/bin/juju deploy cs:~jameinel/ubuntu-lite

$ git checkout 3.1 $ juju bootstrap lxd target31 --build-agent

$ /snap/bin/juju migrate src29:default target31

@hmlanigan we can fix that issue in a subsequent PR, but not this one, then I'm fine with that.

The following code is just not used anymore because we're no longer checking the source.

hmlanigan

We can fix the QA fail in a new pr

SimonRichardson · 2023-07-07T14:53:30Z

/merge

#15886 Merges the following patches: - #15884 - #15883 - #15727 - #15876 ### Conflicts - migration/precheck_test.go - upgrades/upgradevalidation/migrate_test.go

#15894 Merges the following patches: - #15893 - #15888 - #15877 - #15887 - #15886 - #15879 - #15880 - #15884 - #15883 - #15727 - #15876 ### Conflicts None.

#15896 Forward ports: - #15845 - #15725 - #15852 - #15854 - #15849 - #15855 - #15857 - #15861 - #15862 - #15860 - #15864 - #15825 - #15866 - #15863 - #15870 - #15871 - #15872 - #15873 - #15874 - #15876 - #15875 - #15881 - #15727 - #15883 - #15884 - #15880 - #15879 - #15886 - #15887 - #15877 - #15888 - #15893 - #15894 Conflicts: - cmd/juju/ssh/debugcode_test.go - cmd/juju/ssh/debughooks_test.go - cmd/juju/ssh/scp_unix_test.go - cmd/juju/ssh/ssh_machine.go - cmd/juju/ssh/ssh_machine_test.go - cmd/juju/ssh/ssh_unix_test.go - worker/dbaccessor/worker.go - caas/kubernetes/provider/k8s_test.go - caas/kubernetes/provider/operator_test.go - caas/kubernetes/provider/package_test.go - caas/kubernetes/provider/rbac.go - cmd/juju/application/deployer/bundlehandler.go - cmd/juju/application/deployer/bundlehandler_test.go - cmd/jujud/agent/model/manifolds.go - core/bundle/changes/changes.go - worker/caasapplicationprovisioner/application.go - worker/caasapplicationprovisioner/application_test.go - worker/caasapplicationprovisioner/mock_test.go

SimonRichardson requested review from wallyworld, ycliuhw and hmlanigan June 9, 2023 15:40

SimonRichardson commented Jun 9, 2023

View reviewed changes

SimonRichardson mentioned this pull request Jun 9, 2023

[JUJU-3969] Precheck migration checks #15715

Closed

4 tasks

SimonRichardson added do not merge Even if a PR has been approved, do not merge the PR! 3.1 labels Jun 12, 2023

hmlanigan previously requested changes Jun 12, 2023

View reviewed changes

upgrades/upgradevalidation/upgrade.go Show resolved Hide resolved

SimonRichardson force-pushed the allow-upgrades-to-new-version branch from ceb3581 to 07df7b5 Compare June 28, 2023 08:54

SimonRichardson removed the do not merge Even if a PR has been approved, do not merge the PR! label Jun 28, 2023

hmlanigan self-requested a review June 28, 2023 14:14

SimonRichardson commented Jun 30, 2023

View reviewed changes

upgrades/upgradevalidation/upgrade.go Show resolved Hide resolved

SimonRichardson force-pushed the allow-upgrades-to-new-version branch from 07df7b5 to f318d70 Compare June 30, 2023 19:43

Only allow updates from 3.1 onwards

abfbef7

The prior code had logic from the 2.9 branch. We can remove the branching logic and just rely on the fact that the current version can never downgrade.

SimonRichardson force-pushed the allow-upgrades-to-new-version branch from f318d70 to abfbef7 Compare June 30, 2023 19:45

SimonRichardson commented Jun 30, 2023

View reviewed changes

upgrades/upgradevalidation/migrate.go Outdated Show resolved Hide resolved

Remove the hack to upgrade version

d6daec0

This removes the hack to upgrade version, instead just remove all source checks for source requests.

SimonRichardson force-pushed the allow-upgrades-to-new-version branch from c06f669 to 30ac8d4 Compare July 3, 2023 14:02

SimonRichardson force-pushed the allow-upgrades-to-new-version branch from 30ac8d4 to ede7ea7 Compare July 3, 2023 14:07

SimonRichardson commented Jul 3, 2023

View reviewed changes

wallyworld reviewed Jul 4, 2023

View reviewed changes

wallyworld approved these changes Jul 4, 2023

View reviewed changes

hmlanigan requested changes Jul 5, 2023

View reviewed changes

Remove deadcode

4ad7f57

The following code is just not used anymore because we're no longer checking the source.

SimonRichardson force-pushed the allow-upgrades-to-new-version branch from 8460b58 to 4ad7f57 Compare July 7, 2023 08:13

hmlanigan approved these changes Jul 7, 2023

View reviewed changes

jujubot merged commit 4e7eb62 into juju:3.1 Jul 7, 2023
20 checks passed

SimonRichardson deleted the allow-upgrades-to-new-version branch July 7, 2023 15:05

barrettj12 mentioned this pull request Jul 10, 2023

Merge 3.1 into 3.2 #15886

Merged

barrettj12 mentioned this pull request Jul 10, 2023

Merge 3.2 into 3.3 #15894

Merged

hpidcock mentioned this pull request Jul 11, 2023

Merge 3.3 to main #15896

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JUJU-3969] Allow upgrades without specifying version #15727

[JUJU-3969] Allow upgrades without specifying version #15727

SimonRichardson commented Jun 9, 2023 •

edited

Loading

SimonRichardson Jun 9, 2023

hmlanigan commented Jun 30, 2023

SimonRichardson Jul 3, 2023

wallyworld Jul 4, 2023

SimonRichardson Jul 3, 2023

wallyworld left a comment

wallyworld Jul 4, 2023

SimonRichardson Jul 4, 2023

wallyworld Jul 4, 2023

wallyworld Jul 4, 2023

SimonRichardson commented Jul 4, 2023

hmlanigan left a comment

hmlanigan Jul 5, 2023

SimonRichardson Jul 6, 2023 •

edited

Loading

SimonRichardson Jul 7, 2023

hmlanigan left a comment

SimonRichardson commented Jul 7, 2023

	func ValidatorsForModelMigrationSource(
	targetVersion version.Number, cloudspec environscloudspec.CloudSpec,
	) []Validator {
	validators := []Validator{
	getCheckTargetVersionForModel(targetVersion, MigrateToAllowed),
	getCheckUpgradeSeriesLockForModel(false),
	}
	if targetVersion.Major == 3 {
	validators = append(validators,
	checkNoWinMachinesForModel,
	checkForDeprecatedUbuntuSeriesForModel,
	getCheckForLXDVersion(cloudspec, minLXDVersion),
	)
	}
	return validators
	}

	if targetVersion.Major == 3 {
	validators = append(validators,
	checkMongoVersionForControllerModel,
	checkNoWinMachinesForModel,
	checkForDeprecatedUbuntuSeriesForModel,
	getCheckForLXDVersion(cloudspec),
	)
	if targetVersion.Minor >= 1 {
	validators = append(validators, checkForCharmStoreCharms)
	}
	}

[JUJU-3969] Allow upgrades without specifying version #15727

[JUJU-3969] Allow upgrades without specifying version #15727

Conversation

SimonRichardson commented Jun 9, 2023 • edited Loading

Checklist

QA steps

Choose a reason for hiding this comment

hmlanigan commented Jun 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wallyworld left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimonRichardson commented Jul 4, 2023

hmlanigan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimonRichardson Jul 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hmlanigan left a comment

Choose a reason for hiding this comment

SimonRichardson commented Jul 7, 2023

SimonRichardson commented Jun 9, 2023 •

edited

Loading

SimonRichardson Jul 6, 2023 •

edited

Loading