Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

2.9 into develop #14394

Merged
merged 40 commits into from
Jul 29, 2022
Merged

2.9 into develop #14394

merged 40 commits into from
Jul 29, 2022

Conversation

ycliuhw
Copy link
Member

@ycliuhw ycliuhw commented Jul 29, 2022 •
edited 

Conflicts:
    agent/agentbootstrap/bootstrap_test.go
    apiserver/facades/client/charms/services/mocks/interface_mocks.go
    apiserver/facades/client/charms/services/repofactory_test.go
    apiserver/facades/controller/charmdownloader/mocks/mocks.go
    cmd/juju/application/deployer/bundlehandler_test.go
    cmd/juju/commands/bootstrap_test.go
    cmd/juju/commands/synctools.go
    cmd/juju/commands/synctools_test.go
    cmd/juju/commands/upgrademodel.go
    cmd/juju/commands/upgrademodel_test.go
    cmd/juju/commands/upgradecontroller_test.go
    cmd/jujud/agent/agenttest/agent.go
    cmd/jujud/agent/bootstrap_test.go
    cmd/jujud/agent/mongo_test.go
    core/charm/downloader/mocks/downloader_mocks.go
    environs/tools/build.go
    featuretests/bakerystorage_test.go
    featuretests/dblog_test.go
    featuretests/initiate_replset_test.go
    go.mod
    go.sum
    juju/api_test.go
    juju/testing/conn.go
    mongo/admin_test.go
    mongo/oplog_test.go
    provider/azure/environ_test.go
    provider/dummy/environs.go
    provider/dummy/environs_test.go
    state/backups/restore_test.go
    state/binarystorage/binarystorage_test.go
    state/cloudimagemetadata/image_test.go
    state/controller_test.go
    state/globalclock/reader_test.go
    state/globalclock/updater_test.go
    state/imagestorage/image_test.go
    state/initialize_test.go
    state/internal_test.go
    state/logs_internal_test.go
    state/model_test.go
    state/raftlease/target_test.go
    state/state_test.go
    state/storage/storage_test.go
    state/testing/conn.go
    state/testing/suite.go
    state/testing/suite_wallclock.go
    state/watcher/txnwatcher_test.go
    testing/cert.go
    testing/mgo.go
    worker/apiserver/worker_state_test.go
    worker/controllerport/manifold_test.go
    worker/globalclockupdater/manifold_test.go
    worker/httpserver/manifold_test.go
    worker/httpserver/state_test.go
    worker/httpserver/worker_test.go
    worker/raft/raftforwarder/manifold_test.go
    worker/state/manifold_test.go
    worker/upgrader/upgrader.go

hpidcock and others added 30 commits July 23, 2022 00:26
@hpidcock
Drop use of mgo testing from juju/testing instead use mgo/v2/testing.
f8cdba6
@ycliuhw
Refactor sync tool command;
349422a
@ycliuhw
Move most of the upgrade model logic from CLI to ModelUpgrader facade;
f73a930
@ycliuhw
Refactor the upgrade model command to use the new ModelUpgrader facade;
6648be1
@ycliuhw
Re-generate mocks for upgrade model command code;
93f0d44
@ycliuhw
Add/fix tests for upgrade model and upgrade controller command;
b2c934d
@ycliuhw
Fix errors.Is check failed for errors.Wrap(ed) error;
9ba00ce
@ycliuhw
Remove debug logs and tidy go.mod;
e1f1c1b
@ycliuhw
Bump modelupgrader facade to v2 and remove v1;
840e7f6
@ycliuhw
Run make rebuild-schema;
4e845b0
@ycliuhw
Clean up api/client/modelupgrader/upgrader.go;
69d4b30
@ycliuhw
Fix test for upgradeNewSuite;
1276159
@ycliuhw
Fix test for environ package;
e282f5e
@ycliuhw
Move upload tool check logic to command side;
1ba3e6b
@ycliuhw
Fix sync-agent-binary command;
7d84fe9
@manadart
Ensures that we do not instantiate a LXD server until the container
aac52c7 
manager is first used.

Although we only invoke `lxd init` upon the creation of the first
container, creating the container manager was activating LXD.
@hmlanigan
Add addLocalCharm method.
1194901 
addCharm is too long, make more readable by pulling out a different code
path.
@hmlanigan
Add helper function: seriesSelectorRequirements.
421ef66
@hmlanigan
Use new helper seriesSelectorRequirements
ad693c3
@benhoyt
Clean up remaining doc.go issues with "generated GoMock package"
eb3f63c 
This removes the "Package X is a generated GoMock package." from the
package comments. For the most part I've done this by moving the mocks
to their own mocks/ directory. For the testing/ packages where the
mocks need to be in the package I've added -write_package_comment=false
to the mockgen command line.
@hpidcock
Fix more uses of testing when they should use mgo/v2/testing
28dbbf1
@hpidcock
Fix remaining testing to mgo/v2/testing changes.
11469e0
@hpidcock
Upgrade golint to 1.4.6
0ed532a
@ycliuhw
Rephrase some log message for upgrade command;
b82c4e9
@jujubot
Merge pull request juju#14384 from benhoyt/fix-doc.go-generated
fd611ec 
juju#14384

This removes the "Package X is a generated GoMock package." from the
package comments. For the most part I've done this by moving the mocks
to their own mocks/ directory. For the testing/ packages where the
mocks need to be in the package I've added -write_package_comment=false
to the mockgen command line.

Also updates the doc.go static check script I just added, removing the need
to skip "generated GoMock package" comments.

## Checklist

- [x] Code style: imports ordered, good names, simple structure, etc
- [ ] ~Comments saying why design decisions were made~
- [ ] ~Go unit tests, with comments saying what you're testing~
- [ ] ~[Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing~
- [x] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages

## QA steps

```sh
$ cd tests
$ ./main.sh static_analysis test_doc_go
```
@jujubot
Merge pull request juju#14365 from hpidcock/mgov2
e61e715 
juju#14365

Moves away from the mgo testing infrastructure in juju/testing to instead use the versioned testing
in juju/mgo/v2/testing

## QA steps

Unit tests

## Documentation changes

N/A

## Bug reference

N/A
@jujubot
Merge pull request juju#14381 from manadart/2.9-lazy-lxd-init
13fffc6 
juju#14381

Although we only invoke `lxd init` upon the first creation of a container, we create a LXD server to pass to the container manager when we first set up the machine agent.

This means that LXD services are always activated, even if we never provision any LXD containers.

Here we ensure that the server itself is created lazily, by passing a factory function instead of an already-instantiated server. We ensure this has been called before executing any logic that depends on the server.

## QA steps

- Add a machine.
- SSH to the machine and check that `systemctl is-active snap.lxd.daemon.service` reports the service as inactive.
- Check that we can provision and remove container-in-machine for LXD.

## Documentation changes

None.

## Bug reference

https://bugs.launchpad.net/juju/+bug/1934176
@jujubot
Merge pull request juju#14344 from ycliuhw/move-find-tools2controller
416c894 
juju#14344

This PR mainly focuses on moving find tools logic from the command side to the controller and also bump ModelUpgrader facade to v2 and removed v1;
It also includes some drive-by:
* refactored the sync tool command;

## Checklist

 - [ ] ~Requires a [pylibjuju](https://github.com/juju/python-libjuju) change~
 - [ ] ~Added [integration tests](https://github.com/juju/juju/tree/develop/tests) for the PR~
 - [ ] ~Added or updated [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) related to packages changed~
 - [x] Comments answer the question of why design decisions were made

## QA steps

```console
$ juju version --all
version: 2.9.31-ubuntu-amd64
git-commit: 315d242
git-tree-state: dirty
compiler: gc

$ juju bootstrap lxd k1 --config test-mode=true --build-agent

$ juju models -c k1 --format json | jq '.models[] | {name,"agent-version"}'
{
 "name": "admin/controller",
 "agent-version": "2.9.31.1"
}
{
 "name": "admin/default",
 "agent-version": "2.9.31.1"
}

# upgrade to the latest stable -> 2.9.32
$ juju upgrade-controller
best version:
 2.9.32
started upgrade to 2.9.32

$ juju models -c k1 --format json | jq '.models[] | {name,"agent-version"}'
{
 "name": "admin/controller",
 "agent-version": "2.9.32"
}
{
 "name": "admin/default",
 "agent-version": "2.9.31.1"
}

# ensure the new controler has the new modelUpgrader
$ juju scp --debug -m k1:controller $GOPATH/bin/jujud 0:~/
$ juju exec -m controller --machine 0 sudo mv /home/ubuntu/jujud /var/lib/juju/tools/2.9.32-ubuntu-amd64/jujud
$ juju exec -m controller --machine 0 sudo systemctl restart jujud-machine-0.service
$ juju exec -m controller --machine 0 'sudo /var/lib/juju/tools/2.9.32-ubuntu-amd64/jujud version --all'
version: 2.9.32-ubuntu-amd64
git-commit: 315d242
git-tree-state: dirty
compiler: gc

$ juju upgrade-model -m default
best version:
 2.9.32
started upgrade to 2.9.32

$ juju models -c k1 --format json | jq '.models[] | {name,"agent-version"}'
{
 "name": "admin/controller",
 "agent-version": "2.9.32"
}
{
 "name": "admin/default",
 "agent-version": "2.9.32"
}

$ juju upgrade-controller --build-agent
no prepackaged agent binaries available, using local agent binary 2.9.32.1 (built from source)
best version:
 2.9.32.1
started upgrade to 2.9.32.1

```

```console
$ juju version --all
version: 2.9.31-ubuntu-amd64
git-commit: 315d242
git-tree-state: dirty
compiler: gc

$ juju bootstrap lxd k1 --config test-mode=true --build-agent

$ juju models -c k1 --format json | jq '.models[] | {name,"agent-version"}'
{
 "name": "admin/controller",
 "agent-version": "2.9.31.1"
}
{
 "name": "admin/default",
 "agent-version": "2.9.31.1"
}

$ juju upgrade-controller --agent-version 2.9.55
ERROR no matching agent versions available

$ juju upgrade-controller --agent-version 2.9.32
best version:
 2.9.32
started upgrade to 2.9.32

$ juju models -c k1 --format json | jq '.models[] | {name,"agent-version"}'
{
 "name": "admin/controller",
 "agent-version": "2.9.32"
}
{
 "name": "admin/default",
 "agent-version": "2.9.31.1"
}

```

```console
$ sudo snap install juju_2.9.31_amd64.snap --dangerous --classic
juju 2.9.31 installed

$ juju version --all
version: 2.9.31-ubuntu-amd64
compiler: gc

$ juju bootstrap lxd k1 --config test-mode=true

$ juju models -c k1 --format json | jq '.models[] | {name,"agent-version"}'
{
 "name": "admin/controller",
 "agent-version": "2.9.31"
}
{
 "name": "admin/default",
 "agent-version": "2.9.31"
}

# upgrade to the latest stable -> 2.9.32
$ juju upgrade-controller
best version:
 2.9.32
started upgrade to 2.9.32

$ juju models -c k1 --format json | jq '.models[] | {name,"agent-version"}'
{
 "name": "admin/controller",
 "agent-version": "2.9.32"
}
{
 "name": "admin/default",
 "agent-version": "2.9.31"
}

$ juju upgrade-controller
no upgrades available

$ sudo snap install juju_2.9.33_amd64.snap --dangerous --classic
juju 2.9.33 installed

$ juju upgrade-controller
no prepackaged agent binaries available, using local agent binary 2.9.33.1
best version:
 2.9.33.1
started upgrade to 2.9.33.1

$ juju upgrade-controller
no upgrades available

$ juju upgrade-controller --agent-version 2.9.35
WARNING try again with --agent-version=2.9.33 if you want to upload the local binary
no upgrades available

$ juju upgrade-controller --agent-version 2.9.33
no prepackaged agent binaries available, using local agent binary 2.9.33.2
best version:
 2.9.33.2
started upgrade to 2.9.33.2

```
@hmlanigan
Select a series for a charm, if the Resolver doesn't suggest one
d5e0333
@jujubot
Merge pull request juju#14386 from hpidcock/backport-14152
829af42 
juju#14386

Backport juju#14152
jujubot and others added 6 commits July 29, 2022 00:47
@jujubot
Merge pull request juju#14392 from hmlanigan/bundle-series-new
4e08de5 
juju#14392

Before calling AddCharm, ensure we have a series selected after calling ResolveCharm.

A little refactoring around requirements for using the series selector and pulling out deploy a local charm to its own method.

## QA steps

Deploy the simple charm in the bug: https://bugs.launchpad.net/juju/+bug/1982921/comments/3 to a microk8s model

With a machine controller, deploy the following bundle
```
series: focal
applications:
 oneubuntu:
 charm: ubuntu
 twoubuntu:
 charm: ubuntu
 threeubuntu:
 charm: ubuntu
 revision: 19
 channel: stable
 fourubuntu:
 charm: ubuntu
 fiveubuntu:
 charm: ubuntu
 revision: 18
 channel: stable
```

## Bug reference

https://bugs.launchpad.net/juju/+bug/1982921
@barrettj12
[dummy-sink] check token on start event
ade8452
@benhoyt
Update Pebble to latest version
87a9cb4 
This picks up three recent changes:

* Explicitly set HOME and USER env vars if service user+group set:
  canonical/pebble#129:
* Make write/push and make-dir chown intermediate dirs when setting
  user: canonical/pebble#130
* Make exec checks consistently not inherit parent environment
  variables: canonical/pebble#131
@barrettj12
Create upgrade series bash test
5dda6ea
@barrettj12
Remove old upgrade_series python test
6eb6d7f
@jujubot
Merge pull request juju#14362 from barrettj12/nw-upgradeseries
5239f16 
juju#14362

The `nw-upgradeseries-amd64-lxd` is constantly failing. Hence, now's a good time to migrate it into the bash test suites.

## QA steps

```console
cd tests
./main.sh -v upgrade_series
```

## TODO
- [x] rebase to target 2.9 instead, `nw-upgradeseries-amd64-lxd` is failing there as well
- [x] `dummy-sink` gets stuck in maintenance status after upgrade - fix this
- [x] test on 2.9
- [x] test on 3.0
- [ ] make some corresponding changes to `juju-qa-jenkins`
@ycliuhw ycliuhw added do not merge Even if a PR has been approved, do not merge the PR! 3.0 labels Jul 29, 2022
jujubot and others added 3 commits July 29, 2022 08:48
@jujubot
Merge pull request juju#14393 from benhoyt/update-pebble-4
c1a2e2c 
juju#14393

This picks up three recent bug fixes (the first two were reported by users):

* canonical/pebble#129:
* canonical/pebble#130
* canonical/pebble#131

## Checklist

- [ ] ~Code style: imports ordered, good names, simple structure, etc~
- [ ] ~Comments saying why design decisions were made~
- [ ] ~Go unit tests, with comments saying what you're testing~
- [ ] ~[Integration tests](https://github.com/juju/juju/tree/develop/tests), with comments saying what you're testing~
- [ ] ~[doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages~

## QA steps

```sh
$ juju bootstrap microk8s
$ juju deploy snappass-test
$ juju status
# browse to http://<ip address>:5000 to ensure it's working
```
@ycliuhw
Merge branch '2.9' into 2.9-into-develop
b0ebcfa
@ycliuhw
Remove clientAPI from upgrade-controller;
7dfa6f9
@ycliuhw ycliuhw force-pushed the 2.9-into-develop branch 4 times, most recently from 4477315 to 2c92654 Compare July 29, 2022 09:25
@ycliuhw ycliuhw removed the do not merge Even if a PR has been approved, do not merge the PR! label Jul 29, 2022
hpidcock
hpidcock approved these changes Jul 29, 2022
View reviewed changes
Copy link
Member

@hpidcock hpidcock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a typo in UpgradeModelParams that needs fixing

rpc/params/model.go
ModelTag string `json:"model-tag"`
ToVersion version.Number `json:"to-version"`
TargetVersion version.Number `json:"targert-version"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

targert?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ops, thanks for fixing this!

@wallyworld
Copy link
Member

/merge

@jujubot jujubot merged commit ff74688 into juju:develop Jul 29, 2022
@manadart manadart mentioned this pull request Aug 15, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hpidcock hpidcock hpidcock approved these changes

@wallyworld wallyworld wallyworld approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
8 participants
@ycliuhw @wallyworld @hpidcock @jujubot @manadart @hmlanigan @benhoyt @barrettj12