Add support for Ubuntu 20.04 LTS #37

mounaiban · 2021-03-08T05:48:46Z

Hi, here's my hack to enable support for Ubuntu 20.x (see #35). This change only affects the main deployment script deploy-mgmt-hub.sh; Ubuntu 20 support has not been enabled for the SDO test script test-sdo.sh as I could not get it to finish on any of my test systems due to problems with importing the device vouchers.

The main script ``deploy-mgmt-hub.sh`` now runs on Ubuntu 20.x, but the SDO test still only runs on Ubuntu 18 Signed-off-by: mounaiban <29518031+mounaiban@user.noreply.github.com>

bmpotter · 2021-03-10T20:30:05Z

mgmt-hub/README.md

+  - If you are setting up a fresh Ubuntu installation to try Horizon, please do *not* select Docker during the installation, to avoid potential compatibility problems. This script will install the latest version of Docker using apt, via Docker's own repositories.
+  - If you have chosen to install Docker during the Ubuntu installation, you can undo this after installing Ubuntu. Log into the system, wait for about half a minute if necessary, then run ``sudo snap remove docker``.


Do you know for a fact that the version of docker that comes with ubuntu 20.x is a problem?

Ran into problems with filesystem mounts when I tried deploying to a new installation of Ubuntu Server 20.04.2. I remembered selecting Docker from the Featured Server Snaps screen during installation.

The problems disappeared when I repeated the installation exactly as @joewxboy did on the Installing Ubuntu Server video (he did not select any Featured Snaps).

I will try to repeat the problem, and post screenshots with more details when I get the chance.

In the meantime, you can also try it for yourself:

Install Ubuntu 20.04.2, leave everything as default but select the Docker snap in the last menu of the installation.

Download the script with curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh > deploy.sh and make it executable chmod +x deploy.sh

Edit the first line of isUbuntu18() in the script to

if [[ "$DISTRO" == 'Ubuntu 20.'* ]]; then

sudo -i, then run the script (accounting for the change in the working directory 🤓)

Wait until the part where the services are brought online, the process should fail with mount errors.

Please note that I used QEMU instead of VirtualBox to run the VM.

Edit: add extra steps necessary to run deploy-mgmt-hub.sh on Ubuntu 20

I finally found a chance to perform more Ubuntu 20 tests. At time of writing:

The Docker version that comes with Ubuntu 20 via Snap was 19.03.13, and no updates were available

The version that was pulled via apt by deploy-mgmt-hub.sh was 20.10.5

This happens when I try to use deploy-mgmt-hub-sh with the Snap-bundled Docker:

root@ub20-issue35:~# docker --version Docker version 19.03.13, build cd8016b6bc root@ub20-issue35:~# /home/cuddlebunny/deploy.sh -v ----------- Verifying input and the host OS... Management hub services will listen on 127.0.0.1 Updating apt package index... Hit:1 http://au.archive.ubuntu.com/ubuntu focal InRelease Get:2 http://au.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] Get:3 http://au.archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] Get:4 http://au.archive.ubuntu.com/ubuntu focal-security InRelease [109 kB] Get:5 http://au.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [844 kB] Get:6 http://au.archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [749 kB] Fetched 1,917 kB in 2s (882 kB/s) Reading package lists... Installing prerequisites, this could take a minute... Reading package lists... Building dependency tree... Reading state information... gettext-base is already the newest version (0.19.8.1-10build1). gettext-base set to manually installed. The following additional packages will be installed: libjq1 libonig5 Suggested packages: make-doc The following NEW packages will be installed: jq libjq1 libonig5 make 0 upgraded, 4 newly installed, 0 to remove and 23 not upgraded. Need to get 475 kB of archives. After this operation, 1,455 kB of additional disk space will be used. Get:1 http://au.archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] Get:2 http://au.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] Get:3 http://au.archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] Get:4 http://au.archive.ubuntu.com/ubuntu focal/main amd64 make amd64 4.2.1-1.2 [162 kB] Fetched 475 kB in 2s (248 kB/s) Selecting previously unselected package libonig5:amd64. (Reading database ... 71214 files and directories currently installed.) Preparing to unpack .../libonig5_6.9.4-1_amd64.deb ... Unpacking libonig5:amd64 (6.9.4-1) ... Selecting previously unselected package libjq1:amd64. Preparing to unpack .../libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... Selecting previously unselected package jq. Preparing to unpack .../jq_1.6-1ubuntu0.20.04.1_amd64.deb ... Unpacking jq (1.6-1ubuntu0.20.04.1) ... Selecting previously unselected package make. Preparing to unpack .../make_4.2.1-1.2_amd64.deb ... Unpacking make (4.2.1-1.2) ... Setting up make (4.2.1-1.2) ... Setting up libonig5:amd64 (6.9.4-1) ... Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... Setting up jq (1.6-1ubuntu0.20.04.1) ... Processing triggers for man-db (2.9.1-1) ... Processing triggers for libc-bin (2.31-0ubuntu9.2) ... ----------- Downloading template files... verbose: Downloading https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/docker-compose.yml ... verbose: Downloading https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/exchange-tmpl.json ... verbose: Downloading https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/agbot-tmpl.json ... verbose: Downloading https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/css-tmpl.conf ... verbose: Downloading https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh ... verbose: Downloading https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/test-sdo.sh ... Substituting environment variables into template files... ----------- Downloading/starting Horizon management hub services... Downloading management hub docker images... Pulling openhorizon/amd64_agbot:latest... latest: Pulling from openhorizon/amd64_agbot 57de4da701b5: Pull complete cf0f3ebe9f53: Pull complete f96bbf06a0df: Pull complete c12308477d36: Pull complete 328edf248011: Pull complete c8b156b6bd2b: Pull complete e81a6d84aa9d: Pull complete 6a647b6d76d6: Pull complete 215d966616f4: Pull complete 1f8f73ec33d2: Pull complete f739147468c5: Pull complete c64f3c7d32b8: Pull complete f88c80618cbb: Pull complete 7821c7d90659: Pull complete a9d702e9b765: Pull complete 983ccef3d782: Pull complete Digest: sha256:73b50e55e6ce0dc6fdae7a14217fe5ba167a30041feab13d0a4644e5902e712c Status: Downloaded newer image for openhorizon/amd64_agbot:latest docker.io/openhorizon/amd64_agbot:latest Pulling openhorizon/amd64_exchange-api:latest... latest: Pulling from openhorizon/amd64_exchange-api a6b97b4963f5: Pull complete 13948a011eec: Pull complete 94ce93aa525a: Pull complete cf2499f2f0a7: Pull complete 04c26ae46d4e: Pull complete 13f0c3a16fa6: Pull complete 112d1ec85238: Pull complete 003d63433bdd: Pull complete b2b7f63a7bbf: Pull complete 608492d5117f: Pull complete 293d2b7da889: Pull complete Digest: sha256:8805e4d7d8740339c6dcb6e1cc50cfacfd5d14320f13503d5a0ad0d1cb009be3 Status: Downloaded newer image for openhorizon/amd64_exchange-api:latest docker.io/openhorizon/amd64_exchange-api:latest Pulling openhorizon/amd64_cloud-sync-service:latest... latest: Pulling from openhorizon/amd64_cloud-sync-service 57de4da701b5: Already exists cf0f3ebe9f53: Already exists 9b3ad5223059: Pull complete 43584a9e5ebb: Pull complete 44ef2f0b5ccd: Pull complete 0a82f3bd433e: Pull complete 204ee7cb23b1: Pull complete 1e14b696f8a5: Pull complete 397cf930becc: Pull complete bb7d586fda3c: Pull complete 34be8d4b4523: Pull complete Digest: sha256:34953d2a4b45f1182c7200ba7634bdf4f5f6e5d4d59222409d949b72d6b631c8 Status: Downloaded newer image for openhorizon/amd64_cloud-sync-service:latest docker.io/openhorizon/amd64_cloud-sync-service:latest Pulling postgres:latest... latest: Pulling from library/postgres 45b42c59be33: Pull complete 40adec129f1a: Pull complete b4c431d00c78: Pull complete 2696974e2815: Pull complete 564b77596399: Pull complete 5044045cf6f2: Pull complete d736e67e6ac3: Pull complete 390c1c9a5ae4: Pull complete c0e62f172284: Pull complete ebcdc659c5bf: Pull complete 29be22cb3acc: Pull complete f63c47038e66: Pull complete 77a0c198cde5: Pull complete c8752d5b785c: Pull complete Digest: sha256:5cfcfb0516ddd317109e2746a462b1dc398d152936640a2b69284629f48dc288 Status: Downloaded newer image for postgres:latest docker.io/library/postgres:latest Pulling mongo:latest... latest: Pulling from library/mongo 92dc2a97ff99: Pull complete be13a9d27eb8: Pull complete c8299583700a: Pull complete f61ed17142e4: Pull complete bed7676d225b: Pull complete ba553bcfc69c: Pull complete e5046b6c236f: Pull complete 80191acfded2: Pull complete d41d63fc76cc: Pull complete 5605b8c2e9f7: Pull complete e8b16825b485: Pull complete 3d40ccce1309: Pull complete Digest: sha256:845fd775d7a58f564fae7c17a4eec7ff038e1c4004ebe05cc1cb5fc3767cf6cc Status: Downloaded newer image for mongo:latest docker.io/library/mongo:latest Pulling openhorizon/sdo-owner-services:latest... latest: Pulling from openhorizon/sdo-owner-services 57de4da701b5: Already exists cf0f3ebe9f53: Already exists 561cf9ec4c98: Pull complete c72d68aeac2b: Pull complete 545313414f47: Pull complete bc31c80f1d72: Pull complete fbb77f1cd0ff: Pull complete 248a71fe54ee: Pull complete 1d5339da588b: Pull complete a938491d293a: Pull complete c2617796a2e0: Pull complete ed5917c2f733: Pull complete 31346de31bde: Pull complete 465ca486626f: Pull complete Digest: sha256:ac45742c0a5b67482691d0c6a2bd34777741aeafa9d6787dce6223e1842625c5 Status: Downloaded newer image for openhorizon/sdo-owner-services:latest docker.io/openhorizon/sdo-owner-services:latest Starting management hub containers... Creating network "hzn_horizonnet" with driver "bridge" Creating volume "hzn_postgresvol" with default driver Creating volume "hzn_mongovol" with default driver Creating volume "hzn_agbotmsgkeyvol" with default driver Creating volume "hzn_ocsdb" with default driver Creating mongo ... done Creating postgres ... done Creating exchange-api ... error ERROR: for exchange-api Cannot start service exchange-api: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/horizon/exchange.json\\\" to rootfs \\\"/var/snap/docker/common/var-lib-docker/overlay2/ee0127f69d19fbedc477d482e067fc04f802ef59f1bfea383e74320dfaa151c9/merged\\\" at \\\"/var/snap/docker/common/var-lib-docker/overlay2/ee0127f69d19fbedc477d482e067fc04f802ef59f1bfea383e74320dfaa151c9/merged/etc/horizon/exchange/exchange-api.tmpl\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type ERROR: for exchange-api Cannot start service exchange-api: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/horizon/exchange.json\\\" to rootfs \\\"/var/snap/docker/common/var-lib-docker/overlay2/ee0127f69d19fbedc477d482e067fc04f802ef59f1bfea383e74320dfaa151c9/merged\\\" at \\\"/var/snap/docker/common/var-lib-docker/overlay2/ee0127f69d19fbedc477d482e067fc04f802ef59f1bfea383e74320dfaa151c9/merged/etc/horizon/exchange/exchange-api.tmpl\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type ERROR: Encountered errors while bringing up the project. Error: exit code 1 from: starting docker-compose services

This surprises/confuses me, because i've run deploy-mgmt-hub.sh successfully on ubuntu 18 with docker 19.03. I'm guessing the errors above can be overcome with a different mount option. Please investigate.

I am seeing the same error with Ubuntu 20 and docker installed via snap. I removed docker from snap and reinstalled same docker version 19.03.8 via apt and it worked afterwards. So it does not look like to be docker version specific but snap installed docker.

bmpotter · 2021-03-15T14:15:53Z

mgmt-hub/README.md

+  - If you are setting up a fresh Ubuntu installation to try Horizon, please do *not* select Docker during the installation, to avoid potential compatibility problems. This script will install the latest version of Docker using apt, via Docker's own repositories.
+  - If you have chosen to install Docker during the Ubuntu installation, you can undo this after installing Ubuntu. Log into the system, wait for about half a minute if necessary, then run ``sudo snap remove docker``.


This surprises/confuses me, because i've run deploy-mgmt-hub.sh successfully on ubuntu 18 with docker 19.03. I'm guessing the errors above can be overcome with a different mount option. Please investigate.

joewxboy · 2021-03-19T13:14:59Z

Ubuntu 20.04 LTS

mounaiban · 2021-03-21T01:41:10Z

Just completed test installations with Ubuntu Server 18.04, and the results were the same as what @losdevnull got, except on a different Ubuntu version.

Snap Docker on Ubuntu 18.04 doesn't work, while "normal" Docker installed with apt works perfectly. The Docker version was 19.03.13 in both cases.

At this point I think the problem is in the implementation of Snap Docker, which is more virtualised/sandboxed than the average Docker.

I might rename #35 to something more accurate, like support Snap Docker in Ubuntu 18+.

Signed-off-by: Mounaiban <29518031+mounaiban@users.norelpy.github.com>

Add support for Ubuntu 20.04 LTS

2b97a7a

The main script ``deploy-mgmt-hub.sh`` now runs on Ubuntu 20.x, but the SDO test still only runs on Ubuntu 18 Signed-off-by: mounaiban <29518031+mounaiban@user.noreply.github.com>

bmpotter self-requested a review March 10, 2021 20:23

bmpotter requested changes Mar 10, 2021

View reviewed changes

bmpotter requested changes Mar 15, 2021

View reviewed changes

Clarify compatibility with Ubuntu and Docker snaps

860b0e8

Signed-off-by: Mounaiban <29518031+mounaiban@users.norelpy.github.com>

mounaiban force-pushed the master branch from 19b568f to 860b0e8 Compare March 21, 2021 15:30

mounaiban requested a review from bmpotter March 21, 2021 19:14

bmpotter approved these changes Mar 22, 2021

View reviewed changes

bmpotter merged commit 170c053 into open-horizon:master Mar 22, 2021

mounaiban mentioned this pull request Mar 23, 2021

Support Ubuntu 20.04.2 with deploy-mgmt-hub.sh #35

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Ubuntu 20.04 LTS #37

Add support for Ubuntu 20.04 LTS #37

mounaiban commented Mar 8, 2021 •

edited

Loading

bmpotter Mar 10, 2021

mounaiban Mar 11, 2021 •

edited

Loading

mounaiban Mar 11, 2021 •

edited

Loading

bmpotter Mar 15, 2021

losdevnull Mar 19, 2021

bmpotter Mar 15, 2021

joewxboy commented Mar 19, 2021

mounaiban commented Mar 21, 2021 •

edited

Loading

		- If you are setting up a fresh Ubuntu installation to try Horizon, please do not select Docker during the installation, to avoid potential compatibility problems. This script will install the latest version of Docker using apt, via Docker's own repositories.
		- If you have chosen to install Docker during the Ubuntu installation, you can undo this after installing Ubuntu. Log into the system, wait for about half a minute if necessary, then run ``sudo snap remove docker``.

Add support for Ubuntu 20.04 LTS #37

Add support for Ubuntu 20.04 LTS #37

Conversation

mounaiban commented Mar 8, 2021 • edited Loading

bmpotter Mar 10, 2021

Choose a reason for hiding this comment

mounaiban Mar 11, 2021 • edited Loading

Choose a reason for hiding this comment

mounaiban Mar 11, 2021 • edited Loading

Choose a reason for hiding this comment

bmpotter Mar 15, 2021

Choose a reason for hiding this comment

losdevnull Mar 19, 2021

Choose a reason for hiding this comment

bmpotter Mar 15, 2021

Choose a reason for hiding this comment

joewxboy commented Mar 19, 2021

mounaiban commented Mar 21, 2021 • edited Loading

mounaiban commented Mar 8, 2021 •

edited

Loading

mounaiban Mar 11, 2021 •

edited

Loading

mounaiban Mar 11, 2021 •

edited

Loading

mounaiban commented Mar 21, 2021 •

edited

Loading