[docker] Upgrade docker engine to docker-ce 18.09.0~3-0 #2409

yxieca · 2019-01-02T18:12:48Z

This change requires sonic-net/sonic-utilities#423 in place, otherwise, reboot takes about 10 minutes and that will surely fail nightly tests.

- What I did

Initially install docker 1.11.1 for docker image loading. 1.11.1
doesn't require docker service running in target filesystem root
to load images. The latest version supports so is 1.12.2-0.
After all docker images have been loaded, upgrade docker engine to
docker ce 18.09.0~3-0. Also due to the complicity of chroot
installation. Removing docker-engine.prerm is needed for successful
uninstall of version 1.11.1.
New version deprecated docker daemon sub-command, using dockerd
instead. (Deprecated 1.13.0 and removed in 17.12).
dockerd -H fd:// doesn't work. Change to -H unix://.

- How to verify it

Rolling warm-reboot between 2 images for 392 iterations:
================ iteration 392 ===================
18:02:54 up 8 min, 0 users, load average: 1.61, 1.41, 0.77
Candidates: 1230.01 1230.02
Removing image SONiC-OS-wb-20181230.02
Removing image root filesystem...
Image removed
Image 1230.01 is installed
Image 1230.02 is not installed, installing...
Wed Jan 2 18:04:01 UTC 2019 Pausing orchagent ...
RESTARTCHECK succeeded
Wed Jan 2 18:04:01 UTC 2019 Stopping bgp ...
Wed Jan 2 18:04:01 UTC 2019 Stopped bgp ...
swss
Wed Jan 2 18:04:04 UTC 2019 Initialize pre-shutdown ...
0
Wed Jan 2 18:04:05 UTC 2019 Requesting pre-shutdown ...
requested PRE-SHUTDOWN shutdown
Wed Jan 2 18:04:05 UTC 2019 Waiting for pre-shutdown ...
Wed Jan 2 18:04:13 UTC 2019 Pre-shutdown succeeded ...
Wed Jan 2 18:04:13 UTC 2019 Backing up database ...

OK
Wed Jan 2 18:04:14 UTC 2019 Stopping teamd ...
Wed Jan 2 18:04:14 UTC 2019 Stopped teamd ...
Wed Jan 2 18:04:14 UTC 2019 Stopping syncd ...
Wed Jan 2 18:04:25 UTC 2019 Stopped syncd ...
Wed Jan 2 18:04:27 UTC 2019 Rebooting with /sbin/reboot to SONiC-OS-wb-20181230.02 ...

- Initially install docker 1.11.1 for docker image loading. 1.11.1 doesn't require docker service running in target filesystem root to load images. The latest version supports so is 1.12.2-0. - After all docker images have been loaded, upgrade docker engine to docker ce 18.09.0~3-0. Also due to the complicity of chroot installation. Removing docker-engine.prerm is needed for successful uninstall of version 1.11.1. - New version deprecated docker daemon sub-command, using dockerd instead. (Deprecated 1.13.0 and removed in 17.12). - dockerd -H fd:// doesn't work. Change to -H unix://. Signed-off-by: Ying Xie <ying.xie@microsoft.com>

zhenggen-xu · 2019-01-02T23:39:37Z

Starting with an old docker version then upgrade to a new one seems to be complex and may even break the future upgrades.

The docker service used during build process was for tagging the images, If the new docker version is not compatible with chroot, I would suggest we do the tagging outside of "chroot" and then copy over the images, in this case, we just have one clean docker version.

yxieca · 2019-01-03T00:42:31Z

@zhenggen-xu tagging wasn't the issue failed first. "docker load" was the first thing failed. I agree that this procedure is not ideal. I am actively engaging docker community to get a better approach. But at the moment. This unfortunately is the only combination I found working so far.

yxieca · 2019-01-03T00:44:38Z

I noticed binary size hiking with docker upgrading:

The original sizes:
-rwxr-xr-x 1 localadmin localadmin 439974575 Dec 22 01:44 sonic-aboot-broadcom.swi
-rwxr-xr-x 1 localadmin localadmin 446577942 Dec 22 01:44 sonic-broadcom.bin

After upgrading docker:
-rwxr-xr-x 1 localadmin localadmin 482742498 Dec 31 00:47 sonic-aboot-broadcom.swi
-rwxr-xr-x 1 localadmin localadmin 489350422 Dec 31 00:47 sonic-broadcom.bin

zhenggen-xu · 2019-01-03T01:48:13Z

@yxieca "docker load" and "docker tag" were meant for tagging the images, when I say tagging, it included both. What I suggested was that we make changes in the build script (sonic_debian_extension.j2), and we do tagging process (docker load/tag etc) outside the chroot, once the process is done, we can copy the images to $FILESYSTEM_ROOT?

lguohan · 2019-01-03T07:19:08Z

build_debian.sh

+
+function prepare_docker_upgrade()
+{
+    ## This upgrade preparation is very verion speific. Because we are chroot'ing


Thanks fixed now.

lguohan · 2019-01-03T07:22:14Z

ThirdPartyLicenses.txt

@@ -977,7 +977,7 @@ Microsoft is offering you a license to use the following components, to the exte
 * <http://www.gnu.org/philosophy/why-not-lgpl.html>.
 */

-4. apt-clean, apt-gzip-indexes, apt-no-languages imported from docker v1.11.1


THIS was originally imported, think we can keep as it is.

Done. Reverted.

lguohan · 2019-01-03T07:39:31Z

@zhenggen-xu , docker load is for preparing the final sonic image, it is different from docker tag.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

yxieca · 2019-01-04T18:22:32Z

Guohan found the right solution for the problem. That also solves the reboot delay issue.

``` 99425a8 (HEAD -> 202205, origin/202205) [actions] Support Semgrep by Github Actions (sonic-net#2417) f41e4d1 Fix for show vxlan tunnel command display issue sonic-net#11902 (sonic-net#2391) e1d827e [VxLAN]Fix Vxlan delete command to throw error when there are references (sonic-net#2404) d77acf8 [doc] add documentation on automatic techsupport based on memory (sonic-net#2411) 2cfc75a [doc] update "config feature" section with "--block" option (sonic-net#2409) 9dc8471 [Vxlanmgrd] [CPA] Update the vxlan_tunnel name len to be under IFNAMIZ to overcome netdev creation failure (sonic-net#2398) 342589e Added cisco config platform commands (sonic-net#2242) (sonic-net#2418) be7da6b [sonic-installer] use host docker startup arguments when running dockerd in chroot (sonic-net#2179) (sonic-net#2407) d112f7c [202205][auto-ts] add memory check (sonic-net#2116) (sonic-net#2413) ``` Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>

linkmgrd: * a5ac7f6 2022-10-05 | [Active-Active] Post link prober stats to state db (sonic-net#140) (HEAD -> 202205, github/202205) [Jing Zhang] * f4b0e53 2022-10-05 | [Active-Active] Retry config mux mode standby (sonic-net#139) [Jing Zhang] utilities: * a255838 2022-10-04 | [minigraph] new workflow for golden path (sonic-net#2396) (HEAD -> 202205, github/202205) [jingwenxie] * 99425a8 2022-10-03 | [actions] Support Semgrep by Github Actions (sonic-net#2417) [Mai Bui] * f41e4d1 2022-09-30 | Fix for show vxlan tunnel command display issue sonic-net#11902 (sonic-net#2391) [Senthil Bhava] * e1d827e 2022-09-29 | [VxLAN]Fix Vxlan delete command to throw error when there are references (sonic-net#2404) [Sudharsan Dhamal Gopalarathnam] * d77acf8 2022-09-28 | [doc] add documentation on automatic techsupport based on memory (sonic-net#2411) [Stepan Blyshchak] * 2cfc75a 2022-09-28 | [doc] update "config feature" section with "--block" option (sonic-net#2409) [Stepan Blyshchak] * 9dc8471 2022-09-28 | [Vxlanmgrd] [CPA] Update the vxlan_tunnel name len to be under IFNAMIZ to overcome netdev creation failure (sonic-net#2398) [Vivek] * 342589e 2022-10-03 | Added cisco config platform commands (sonic-net#2242) (sonic-net#2418) [yucgu] swss: * 9d9f395 2022-10-04 | [intfmgr]: Enable `accept_untracked_na` kernel param (sonic-net#2436) (HEAD -> 202205, github/202205) [Lawrence Lee] * 6b6d25d 2022-10-04 | [orchdaemon]: Fixed sairedis record file rotation (sonic-net#2480) [Bryan Crossland] Signed-off-by: Ying Xie <ying.xie@microsoft.com>

linkmgrd: * a5ac7f6 2022-10-05 | [Active-Active] Post link prober stats to state db (#140) (HEAD -> 202205, github/202205) [Jing Zhang] * f4b0e53 2022-10-05 | [Active-Active] Retry config mux mode standby (#139) [Jing Zhang] utilities: * a255838 2022-10-04 | [minigraph] new workflow for golden path (#2396) (HEAD -> 202205, github/202205) [jingwenxie] * 99425a8 2022-10-03 | [actions] Support Semgrep by Github Actions (#2417) [Mai Bui] * f41e4d1 2022-09-30 | Fix for show vxlan tunnel command display issue #11902 (#2391) [Senthil Bhava] * e1d827e 2022-09-29 | [VxLAN]Fix Vxlan delete command to throw error when there are references (#2404) [Sudharsan Dhamal Gopalarathnam] * d77acf8 2022-09-28 | [doc] add documentation on automatic techsupport based on memory (#2411) [Stepan Blyshchak] * 2cfc75a 2022-09-28 | [doc] update "config feature" section with "--block" option (#2409) [Stepan Blyshchak] * 9dc8471 2022-09-28 | [Vxlanmgrd] [CPA] Update the vxlan_tunnel name len to be under IFNAMIZ to overcome netdev creation failure (#2398) [Vivek] * 342589e 2022-10-03 | Added cisco config platform commands (#2242) (#2418) [yucgu] swss: * 9d9f395 2022-10-04 | [intfmgr]: Enable `accept_untracked_na` kernel param (#2436) (HEAD -> 202205, github/202205) [Lawrence Lee] * 6b6d25d 2022-10-04 | [orchdaemon]: Fixed sairedis record file rotation (#2480) [Bryan Crossland] Signed-off-by: Ying Xie <ying.xie@microsoft.com> Signed-off-by: Ying Xie <ying.xie@microsoft.com>

ac71d745d [VxLAN]Fix Vxlan delete command to throw error when there are references (#2404) 7419c6731 Added cisco config platform commands (#2242) 8760bbe80 Add UT to check sonic installer does not depend on database (#2401) 6bef65260 [doc] add documentation on automatic techsupport based on memory (#2411) 4a783745f [doc] update "config feature" section with "--block" option (#2409) dd6210fcc [Vxlanmgrd] [CPA] Update the vxlan_tunnel name len to be under IFNAMIZ to overcome netdev creation failure (#2398) bdc4a8a60 Fix broken pipeline build URL (#2363) b31681b43 Fix display disorder problem of show vrf (#2392) 123504a85 YANG validation for ConfigDB Updates: portchannel add/remove, loopback interface, VLAN 28f6820c6 [link-local]Modify RIF check to include link-local enabled interfaces (#2394)

423779410 [muxcable][config] add CLI support for mux mode detach (sonic-net#2425) a817896b1 YANG validation for ConfigDB Updates: MGMT_INTERFACE, PORTCHANNEL_MEMBER use cases (sonic-net#2420) 81e2aecca [minigraph] new workflow for golden path (sonic-net#2396) c1206aac3 ConfigDB Updates with YANG Validation: Include potential for YANG validation even when adhoc validation is used (sonic-net#2412) 57c509a9d [show] vnet endpoint [ip/ipv6] command (sonic-net#2342) 4b2b766ac [actions] Support Semgrep by Github Actions (sonic-net#2417) 156257e2a check for vxlan mapping before removing vlan (sonic-net#2388) cb0edd310 Fix for show vxlan tunnel command display issue sonic-net#11902 (sonic-net#2391) ac71d745d [VxLAN]Fix Vxlan delete command to throw error when there are references (sonic-net#2404) 7419c6731 Added cisco config platform commands (sonic-net#2242) 8760bbe80 Add UT to check sonic installer does not depend on database (sonic-net#2401) 6bef65260 [doc] add documentation on automatic techsupport based on memory (sonic-net#2411) 4a783745f [doc] update "config feature" section with "--block" option (sonic-net#2409) dd6210fcc [Vxlanmgrd] [CPA] Update the vxlan_tunnel name len to be under IFNAMIZ to overcome netdev creation failure (sonic-net#2398) bdc4a8a60 Fix broken pipeline build URL (sonic-net#2363) b31681b43 Fix display disorder problem of show vrf (sonic-net#2392) 123504a85 YANG validation for ConfigDB Updates: portchannel add/remove, loopback interface, VLAN 28f6820c6 [link-local]Modify RIF check to include link-local enabled interfaces (sonic-net#2394)

stcheng added the Submodule Update ⏫ label Jan 2, 2019

lguohan reviewed Jan 3, 2019

View reviewed changes

address review comment

d2ea110

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

yxieca closed this Jan 4, 2019

yxieca mentioned this pull request Oct 6, 2022

[202205][linkmgrd][utilities][swss] advance submodule head #12292

Merged

wen587 mentioned this pull request Oct 8, 2022

[master] update sonic-utilities #12307

Merged

7 tasks

wen587 mentioned this pull request Oct 10, 2022

[master] sonic-utilities: Fix previous commit info (no code change) #12328

Closed

7 tasks

wen587 mentioned this pull request Oct 10, 2022

[master] update sonic-utilities (#12307) #12329

Closed

7 tasks

aravindmani-1 mentioned this pull request Nov 18, 2022

[Submodule] Update sonic-utilities pointer #12760

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[docker] Upgrade docker engine to docker-ce 18.09.0~3-0 #2409

[docker] Upgrade docker engine to docker-ce 18.09.0~3-0 #2409

yxieca commented Jan 2, 2019 •

edited

Loading

zhenggen-xu commented Jan 2, 2019 •

edited

Loading

yxieca commented Jan 3, 2019

yxieca commented Jan 3, 2019

zhenggen-xu commented Jan 3, 2019

lguohan Jan 3, 2019

yxieca Jan 4, 2019

lguohan Jan 3, 2019

yxieca Jan 4, 2019

lguohan commented Jan 3, 2019

yxieca commented Jan 4, 2019

[docker] Upgrade docker engine to docker-ce 18.09.0~3-0 #2409

[docker] Upgrade docker engine to docker-ce 18.09.0~3-0 #2409

Conversation

yxieca commented Jan 2, 2019 • edited Loading

zhenggen-xu commented Jan 2, 2019 • edited Loading

yxieca commented Jan 3, 2019

yxieca commented Jan 3, 2019

zhenggen-xu commented Jan 3, 2019

lguohan Jan 3, 2019

Choose a reason for hiding this comment

yxieca Jan 4, 2019

Choose a reason for hiding this comment

lguohan Jan 3, 2019

Choose a reason for hiding this comment

yxieca Jan 4, 2019

Choose a reason for hiding this comment

lguohan commented Jan 3, 2019

yxieca commented Jan 4, 2019

yxieca commented Jan 2, 2019 •

edited

Loading

zhenggen-xu commented Jan 2, 2019 •

edited

Loading