Merge pull request #13607 from wallyworld/merge-2.9-20220112

#13607

Merge 2.9. 
2.9 was updated to use juju-db 4.4 - here we change that to 5.0

#13587 [JUJU-381] Fix test race in RaftLeaseRemoteSuite.TestSetAddress
#13588 [JUJU-349] Remove stub sentence from add-machine helper
#13570 [JUJU-256] MVP verify app health after controller/model upgrade
#13566 [JUJU-338] Drop miscellaneous NewProviderAddress constructors in favour of a functional approach
#13589 Update to latest version of Pebble
#13591 [JUJU-388] Ensure 'hostname -f' returns juju-assigned hostname on equinix metal
#13592 Remove the format2 test charm
#13593 The interactive version command doesn't need a controller
#13590 [JUJU-403] Remove txn watcher wrench
#13594 [JUJU-335] Expand functionality of ProviderAddress
#13601 [JUJU-402] Unit machine test fixes for fire-walled env (s390x)
#13603 [JUJU-412] Fix racy tests for the CAAS firewaller worker
#13604 [JUJU-413] Test Raft queue immediate dispatch instead of 1 batch per operation
#13602 [JUJU-106] Add support for mgo scram-sha256 auth; default to mongo 4.4 on bootstrap
#13599 [JUJU-396] juju info/find/download run without a controller
#13606 [JUJU-418] Fix some intermittent unit test failures
#13600 [JUJU-380] Shutdown application worker properly

Conflicts were in snap version change, imports, and removed code.
```
# Conflicts:
# agent/agent_test.go
# api/charmhub/client_test.go
# api/charmhub/data.go
# caas/kubernetes/provider/bootstrap_test.go
# cmd/juju/charmhub/data.go
# cmd/juju/charmhub/download.go
# cmd/juju/charmhub/download_test.go
# cmd/juju/charmhub/find.go
# cmd/juju/charmhub/find_test.go
# cmd/juju/charmhub/info.go
# cmd/juju/charmhub/info_test.go
# cmd/juju/charmhub/mocks/api_mock.go
# mongo/mongodfinder_test.go
# provider/maas/interfaces.go
# provider/maas/interfaces_test.go
# service/snap/snap_test.go
```
## QA steps

See PRs

[JUJU-381]: https://warthogs.atlassian.net/browse/JUJU-381?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-349]: https://warthogs.atlassian.net/browse/JUJU-349?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-256]: https://warthogs.atlassian.net/browse/JUJU-256?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-338]: https://warthogs.atlassian.net/browse/JUJU-338?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-403]: https://warthogs.atlassian.net/browse/JUJU-403?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-335]: https://warthogs.atlassian.net/browse/JUJU-335?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-402]: https://warthogs.atlassian.net/browse/JUJU-402?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-412]: https://warthogs.atlassian.net/browse/JUJU-412?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-413]: https://warthogs.atlassian.net/browse/JUJU-413?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-106]: https://warthogs.atlassian.net/browse/JUJU-106?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-396]: https://warthogs.atlassian.net/browse/JUJU-396?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

[JUJU-388]: https://warthogs.atlassian.net/browse/JUJU-388?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

.github/verify-apache2.sh

@@ -0,0 +1,7 @@
+#!/usr/bin/bash
+
+set -euxo pipefail
+
+ip=$(juju status --format json | jq -r '.applications.apache2.units[]."public-address"')
+
+curl --silent --output /dev/null --max-time 3 "$ip"

.github/workflows/client-tests.yml

@@ -48,14 +48,6 @@ jobs:
     - name: "Install Mongo Dependencies: ubuntu-latest"
       if: (matrix.os == 'ubuntu-latest')
       run: |
-        # Remove the default mongo
-        for version in "4.2" "4.4"; do
-          sudo rm "/etc/apt/sources.list.d/mongodb-org-${version}.list" || true
-        done
-        sudo DEBIAN_FRONTEND=noninteractive apt-get purge -y mongodb-org
-        sudo DEBIAN_FRONTEND=noninteractive apt autoremove
-        sudo rm -rf /usr/bin/mongo* || true
-
         make install-mongo-dependencies
 
     - name: "Remove Mongo Dependencies: windows-latest"
@@ -68,18 +60,18 @@ jobs:
       if: (matrix.os == 'windows-latest')
       uses: crazy-max/ghaction-chocolatey@v1
       with:
-        args: install mongodb.install --version=4.0.21 --allow-downgrade
+        args: install mongodb.install --version=4.4.11 --allow-downgrade
 
     # GitHub runners already have preinstalled version of mongodb, but
-    # we specifically need 4.0.21, otherwise our tests will not pass
+    # we specifically need 4.4.11, otherwise our tests will not pass
     - name: "Install Mongo Dependencies: macOS-latest"
       if: (matrix.os == 'macOS-latest')
       run: |
-        curl -o mongodb-4.0.21.tgz https://fastdl.mongodb.org/osx/mongodb-osx-ssl-x86_64-4.0.21.tgz
-        tar xzvf mongodb-4.0.21.tgz
+        curl -o mongodb-4.4.11.tgz https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-4.4.11.tgz
+        tar xzvf mongodb-4.4.11.tgz
         sudo rm -rf /usr/local/mongodb
         sudo mkdir -p /usr/local/mongodb
-        sudo mv mongodb-osx-x86_64-4.0.21/bin/* /usr/local/mongodb
+        sudo mv mongodb-macos-x86_64-4.4.11/bin/* /usr/local/mongodb
         sudo mkdir -p /usr/local/bin
         sudo rm /usr/local/bin/mongod
         sudo ln -s /usr/local/mongodb/mongod /usr/local/bin/mongod

.github/workflows/smoke.yml

@@ -77,7 +77,7 @@ jobs:
         sudo chmod a+wr /var/snap/lxd/common/lxd/unix.socket
         echo "/snap/bin" >> $GITHUB_PATH
 
-    - name: Bootstrap Juju latest/stable
+    - name: Bootstrap Juju
       if: env.RUN_TEST == 'RUN'
       shell: bash
       run: |
@@ -92,6 +92,26 @@ jobs:
       if: env.RUN_TEST == 'RUN'
       uses: actions/checkout@v2
 
+    - name: Deploy some applications
+      if: env.RUN_TEST == 'RUN'
+      shell: bash
+      run: |
+        set -euxo pipefail
+
+        juju deploy apache2 --series focal
+
+        juju wait-for application apache2
+
+        attempts=0
+        while ! ./.github/verify-apache2.sh; do
+          sleep 10
+          attempts=$((attempts+1))
+          if [ "$attempts" -eq 5 ]; then
+            echo "Apache health check timed out"
+            exit 0
+          fi
+        done
+
     - name: Build snap
       if: env.RUN_TEST == 'RUN'
       shell: bash
@@ -104,7 +124,7 @@ jobs:
       shell: bash
       run: |
         set -euxo pipefail
-        sudo snap install *.snap --dangerous --classic
+        sudo snap install juju*.snap --dangerous --classic
 
     - name: Preflight
       if: env.RUN_TEST == 'RUN'
@@ -144,6 +164,8 @@ jobs:
             exit 1
         fi
 
+        ./.github/verify-apache2.sh
+
     - name: Test upgrade model
       if: env.RUN_TEST == 'RUN'
       shell: bash
@@ -194,3 +216,5 @@ jobs:
             juju debug-log --replay --no-tail
             exit 1
         fi
+
+        ./.github/verify-apache2.sh

Makefile

@@ -181,10 +181,11 @@ endif
 
 WAIT_FOR_DPKG=sh -c '. "${PROJECT_DIR}/make_functions.sh"; wait_for_dpkg "$$@"' wait_for_dpkg
 
+JUJU_DB_CHANNEL=4.4/stable
 install-mongo-dependencies:
 ## install-mongo-dependencies: Install Mongo and its dependencies
-	@echo Installing 4.4 juju-db snap for mongodb
-	@sudo snap install juju-db --channel=4.4/stable
+	@echo Installing ${JUJU_DB_CHANNEL} juju-db snap for mongodb
+	@sudo snap refresh juju-db --channel=${JUJU_DB_CHANNEL} 2> /dev/null; sudo snap install juju-db --channel=${JUJU_DB_CHANNEL} 2> /dev/null
 	@$(WAIT_FOR_DPKG)
 	@sudo apt-get --yes install  $(strip $(DEPENDENCIES))
 

README.md

@@ -113,8 +113,8 @@ centralised enterprise grade management and operations systems.
 Our community hangs out at the [Charmhub discourse](https://discourse.juju.is/)
 which serves as a combination mailing list and web forum. Keep up with the news
 and get a feel for operator engineering and usage there. Get  the Juju CLI on
-Windows, macOS or Linux with the 
-[install instructions](https://juju.is/docs/installing) and 
+Windows, macOS or Linux with the
+[install instructions](https://juju.is/docs/installing) and
 [try the tutorials](https://juju.is/docs/tutorials). All you need is a small
 K8s cluster, or an Ubuntu machine or VM to run MicroK8s.
 
@@ -126,5 +126,5 @@ of commands and usage.
 Follow our [code and contribution guidelines](CONTRIBUTING.md) to learn how
 to make code changes. File bugs in
 [Launchpad](https://bugs.launchpad.net/juju/+filebug) or ask questions on
-our [Freenode IRC channel](https://webchat.freenode.net/#juju), and 
+our [Freenode IRC channel](https://webchat.freenode.net/#juju), and
 [Mattermost](chat.charmhub.io/).

agent/agentbootstrap/bootstrap_test.go

@@ -126,7 +126,7 @@ LXC_BRIDGE="ignored"`[1:])
 	expectBootstrapConstraints := constraints.MustParse("mem=1024M")
 	expectModelConstraints := constraints.MustParse("mem=512M")
 	expectHW := instance.MustParseHardware("mem=2048M")
-	initialAddrs := corenetwork.NewProviderAddresses(
+	initialAddrs := corenetwork.NewMachineAddresses([]string{
 		"zeroonetwothree",
 		"0.1.2.3",
 		"10.0.3.1", // lxc bridge address filtered.
@@ -135,7 +135,7 @@ LXC_BRIDGE="ignored"`[1:])
 		"10.0.4.1", // lxd bridge address filtered.
 		"10.0.4.4", // lxd bridge address filtered.
 		"10.0.4.5", // not an lxd bridge address
-	)
+	}).AsProviderAddresses()
 	filteredAddrs := corenetwork.NewSpaceAddresses(
 		"zeroonetwothree",
 		"0.1.2.3",