Switch to VMware Harbor registry for Antrea Docker images #1617

antoninbas · 2020-12-04T21:55:17Z

All "user-facing" images (antrea/antrea-ubuntu, antrea/antrea-windows,
antrea/octant-antrea-ubuntu) are now pulled from the VMware Harbor
distribution registry (projects.registry.vmware.com) to avoid Docker
pull rate limiting.

The YAML manifests (top-of-tree and releases) now refer to the new
registry.

This is not a complete transition from Dockerhub to Harbor. Images used
for build purposes (e.g. antrea/openvswitch) are still pulled from
Dockerhub by default. Rate limiting is not as much of an issue for these
(not user-facing, Github workflows are not subject to rate limiting, we
have workarounds for the Jenkins CI jobs). One thing to keep is mind is
that we cannot push to the VMware Harbor registry from outside of the
the VMware corporate network. This is why all images are pushed to
the Dockerhub registry, which is then mirrored by the distribution
Harbor registry.

See #1555

antrea-bot · 2020-12-04T21:55:31Z

Thanks for your PR.
Unit tests and code linters are run automatically every time the PR is updated.
E2e, conformance and network policy tests can only be triggered by a member of the vmware-tanzu organization. Regular contributors to the project should join the org.

The following commands are available:

/test-e2e: to trigger e2e tests.
/skip-e2e: to skip e2e tests.
/test-conformance: to trigger conformance tests.
/skip-conformance: to skip conformance tests.
/test-all-features-conformance: to trigger conformance tests with all alpha features enabled.
/skip-all-features-conformance: to skip conformance tests with all alpha features enabled.
/test-whole-conformance: to trigger all conformance tests on linux.
/skip-whole-conformance: to skip all conformance tests on linux.
/test-networkpolicy: to trigger networkpolicy tests.
/skip-networkpolicy: to skip networkpolicy tests.
/test-windows-conformance: to trigger windows conformance tests.
/skip-windows-conformance: to skip windows conformance tests.
/test-windows-networkpolicy: to trigger windows networkpolicy tests.
/skip-windows-networkpolicy: to skip windows networkpolicy tests.
/test-hw-offload: to trigger ovs hardware offload test.
/skip-hw-offload: to skip ovs hardware offload test.
/test-all: to trigger all tests (except whole conformance).
/skip-all: to skip all tests (except whole conformance).

codecov-io · 2020-12-04T21:59:28Z

Codecov Report

Merging #1617 (9a002c1) into master (9d3d10b) will increase coverage by 0.64%.
The diff coverage is 60.22%.

@@            Coverage Diff             @@
##           master    #1617      +/-   ##
==========================================
+ Coverage   63.31%   63.96%   +0.64%     
==========================================
  Files         170      181      +11     
  Lines       14250    15433    +1183     
==========================================
+ Hits         9023     9871     +848     
- Misses       4292     4527     +235     
- Partials      935     1035     +100

Flag	Coverage Δ
e2e-tests	`46.02% <39.23%> (?)`
kind-e2e-tests	`53.26% <53.25%> (-2.13%)`	⬇️
unit-tests	`40.53% <24.61%> (-0.75%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cmd/antrea-agent/agent.go	`0.00% <0.00%> (ø)`
.../agent/apiserver/handlers/networkpolicy/handler.go	`58.33% <ø> (ø)`
...gent/controller/noderoute/node_route_controller.go	`61.04% <ø> (+14.58%)`	⬆️
pkg/agent/stats/collector.go	`97.72% <ø> (ø)`
pkg/antctl/antctl.go	`100.00% <ø> (ø)`
pkg/antctl/transform/controllerinfo/transform.go	`0.00% <ø> (ø)`
pkg/antctl/transform/version/transform.go	`0.00% <ø> (ø)`
pkg/controller/networkpolicy/tier.go	`90.00% <ø> (ø)`
pkg/features/antrea_features.go	`16.66% <ø> (ø)`
pkg/ovs/openflow/ofctrl_builder.go	`59.85% <0.00%> (-2.32%)`	⬇️
... and 91 more

jianjuns · 2020-12-07T23:14:44Z

ci/kind/test-e2e-kind.sh

@@ -99,7 +99,7 @@ if $coverage; then
    manifest_args="$manifest_args --coverage"
    COMMON_IMAGES_LIST+=("antrea/antrea-ubuntu-coverage:latest")
 else
-    COMMON_IMAGES_LIST+=("antrea/antrea-ubuntu:latest")


Do you have any good way to define the registry URL once in a file and other code just read the URL?

Or maybe have an update-image-name-with-registry.sh script to update all images' names in the repo? Then we don't need to add logic in the code.

I guess specifically for test / CI, we can have something like this:

images.yml

antrea/antrea-ubuntu: projects.registry.vmware.com/antrea/antrea-ubuntu

get_img.sh

function get_img { while read -r line; do IFS=':' read -ra ADDR <<< "$line" name=$(echo ${ADDR[0]} | xargs) img=$(echo ${ADDR[1]} | xargs) if [[ $name == $1 ]]; then echo $img; fi done < "images.yml" } [[ $_ != $0 ]] || get_img $1

for example, CI scripts can source get_img.sh and use "$(get_img antrea/antrea-ubuntu)"

I am not sure it's much better though. Maybe we can wait a bit and see how things evolve in the short term? Right now we have to push and pull from different registries which is making things a bit tricky.

Good idea. Yes we can wait and see.

lzhecheng

LGTM

antoninbas · 2020-12-08T20:05:18Z

/test-all

lzhecheng

LGTM

antoninbas · 2020-12-09T03:59:26Z

/test-all

antoninbas · 2020-12-09T19:57:24Z

/test-e2e
/test-networkpolicy

antoninbas · 2020-12-10T00:54:49Z

/test-all

All "user-facing" images (antrea/antrea-ubuntu, antrea/antrea-windows, antrea/octant-antrea-ubuntu) are now pulled from the VMware Harbor distribution registry (projects.registry.vmware.com) to avoid Docker pull rate limiting. The YAML manifests (top-of-tree and releases) now refer to the new registry. This is not a complete transition from Dockerhub to Harbor. Images used for build purposes (e.g. antrea/openvswitch) are still pulled from Dockerhub by default. Rate limiting is not as much of an issue for these (not user-facing, Github workflows are not subject to rate limiting, we have workarounds for the Jenkins CI jobs). One thing to keep is mind is that we cannot push to the VMware Harbor registry from outside of the the VMware corporate network. This is why all images are pushed to the Dockerhub registry, which is then mirrored by the distribution Harbor registry. See antrea-io#1555

antoninbas · 2020-12-10T19:59:55Z

/test-all

antoninbas · 2020-12-11T02:20:35Z

@lzhecheng I need a new review & approval for this. I found a couple issues during testing that I had to address (see second and third commits)

lzhecheng

LGTM

Changes are the same as in antrea-io#1617 but for the antrea/flow-aggregator Docker image. There was also a typo in the flow-aggregator YAML: the image key didn't match the one used in generate-manifest-flow-aggregator.sh when invoking kustomize.

Changes are the same as in #1617 but for the antrea/flow-aggregator Docker image. There was also a typo in the flow-aggregator YAML: the image key didn't match the one used in generate-manifest-flow-aggregator.sh when invoking kustomize.

…o#1650) Changes are the same as in antrea-io#1617 but for the antrea/flow-aggregator Docker image. There was also a typo in the flow-aggregator YAML: the image key didn't match the one used in generate-manifest-flow-aggregator.sh when invoking kustomize.

* Switch to VMware Harbor registry for Antrea Docker images All "user-facing" images (antrea/antrea-ubuntu, antrea/antrea-windows, antrea/octant-antrea-ubuntu) are now pulled from the VMware Harbor distribution registry (projects.registry.vmware.com) to avoid Docker pull rate limiting. The YAML manifests (top-of-tree and releases) now refer to the new registry. This is not a complete transition from Dockerhub to Harbor. Images used for build purposes (e.g. antrea/openvswitch) are still pulled from Dockerhub by default. Rate limiting is not as much of an issue for these (not user-facing, Github workflows are not subject to rate limiting, we have workarounds for the Jenkins CI jobs). One thing to keep is mind is that we cannot push to the VMware Harbor registry from outside of the the VMware corporate network. This is why all images are pushed to the Dockerhub registry, which is then mirrored by the distribution Harbor registry. See #1555 * Changes for Jenkins scripts * Fix issue in generate-manifest.sh when enabling ipsec and coverage

Following antrea-io#1617, the default image name in the antrea.yml file changed to use the harbor image. Although the patch includes a change in the make file to change the image name correctly in case the image was built, the hw-offload CI uses `docker build` to build the image, and this made it so that the CI would not use the image it was building, but the upstream latest image. This patch fixes that by building the antrea manifests with the image name being the same as the one that was built.

Following antrea-io#1617, the default image name in the antrea.yml file changed to use the harbor image. Although the patch includes a change in the make file to change the image name correctly in case the image was built, the hw-offload CI uses `docker build` to build the image, and this made it so that the CI would not use the image it was building, but the upstream latest image. Also changed the way to enable the hw-offload to use the generate-manifests.sh instead of using `sed`. This patch fixes that by building the antrea manifests with the image name being the same as the one that was built.

vmwclabot added the cla-not-required label Dec 4, 2020

antoninbas requested review from jianjuns, lzhecheng, tnqn and Dyanngg December 4, 2020 21:55

antoninbas added this to the Antrea v0.12.0 release milestone Dec 4, 2020

srikartati mentioned this pull request Dec 7, 2020

Change L2 forwarding flows for traffic to gateway and tunnel #1594

Merged

jianjuns previously approved these changes Dec 7, 2020

View reviewed changes

lzhecheng previously approved these changes Dec 8, 2020

View reviewed changes

antoninbas dismissed stale reviews from lzhecheng and jianjuns via 9744fd4 December 8, 2020 22:19

antoninbas force-pushed the switch-to-vmware-harbor-distribution-registry-for-antrea-images branch from d537fcb to 9744fd4 Compare December 8, 2020 22:19

lzhecheng previously approved these changes Dec 9, 2020

View reviewed changes

antoninbas dismissed lzhecheng’s stale review via 89f8554 December 9, 2020 03:59

antoninbas force-pushed the switch-to-vmware-harbor-distribution-registry-for-antrea-images branch from 9744fd4 to 89f8554 Compare December 9, 2020 03:59

antoninbas force-pushed the switch-to-vmware-harbor-distribution-registry-for-antrea-images branch from 89f8554 to 63c2778 Compare December 9, 2020 23:43

antoninbas added 3 commits December 10, 2020 11:58

Changes for Jenkins scripts

86d659a

Fix issue in generate-manifest.sh when enabling ipsec and coverage

9a002c1

antoninbas force-pushed the switch-to-vmware-harbor-distribution-registry-for-antrea-images branch from 63c2778 to 9a002c1 Compare December 10, 2020 19:59

liu4480 mentioned this pull request Dec 11, 2020

add simulator for scale tests #1493

Merged

antoninbas requested a review from lzhecheng December 11, 2020 02:20

lzhecheng approved these changes Dec 11, 2020

View reviewed changes

antoninbas merged commit ff305a4 into antrea-io:master Dec 11, 2020

antoninbas deleted the switch-to-vmware-harbor-distribution-registry-for-antrea-images branch December 11, 2020 02:58

antoninbas mentioned this pull request Dec 11, 2020

Use VMware Harbor registry for flow-aggregator Docker image #1650

Merged

antoninbas mentioned this pull request Dec 11, 2020

Fix CI cloud scripts #1651

Merged

abdallahyas mentioned this pull request Apr 13, 2021

[CI] Fix image names in hw-offload CI #2077

Merged

tnqn mentioned this pull request Aug 24, 2022

kind: fix antrea-ubuntu image in build/yamls #4147

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to VMware Harbor registry for Antrea Docker images #1617

Switch to VMware Harbor registry for Antrea Docker images #1617

antoninbas commented Dec 4, 2020

antrea-bot commented Dec 4, 2020

codecov-io commented Dec 4, 2020 •

edited

jianjuns Dec 7, 2020

lzhecheng Dec 8, 2020

antoninbas Dec 8, 2020

lzhecheng Dec 8, 2020

lzhecheng left a comment

antoninbas commented Dec 8, 2020

lzhecheng left a comment

antoninbas commented Dec 9, 2020

antoninbas commented Dec 9, 2020

antoninbas commented Dec 10, 2020

antoninbas commented Dec 10, 2020

antoninbas commented Dec 11, 2020

lzhecheng left a comment

Switch to VMware Harbor registry for Antrea Docker images #1617

Switch to VMware Harbor registry for Antrea Docker images #1617

Conversation

antoninbas commented Dec 4, 2020

antrea-bot commented Dec 4, 2020

codecov-io commented Dec 4, 2020 • edited

Codecov Report

jianjuns Dec 7, 2020

Choose a reason for hiding this comment

lzhecheng Dec 8, 2020

Choose a reason for hiding this comment

antoninbas Dec 8, 2020

Choose a reason for hiding this comment

lzhecheng Dec 8, 2020

Choose a reason for hiding this comment

lzhecheng left a comment

Choose a reason for hiding this comment

antoninbas commented Dec 8, 2020

lzhecheng left a comment

Choose a reason for hiding this comment

antoninbas commented Dec 9, 2020

antoninbas commented Dec 9, 2020

antoninbas commented Dec 10, 2020

antoninbas commented Dec 10, 2020

antoninbas commented Dec 11, 2020

lzhecheng left a comment

Choose a reason for hiding this comment

codecov-io commented Dec 4, 2020 •

edited