Support antrea-agent UBI8 based image #3273

ksamoray · 2022-01-31T10:17:54Z

Add the required code to build an Antrea image based on Red Hat UBI
(Universal Base Image), which is in used by Red Hat platforms.

Signed-off-by: Kobi Samoray ksamoray@vmware.com

codecov-commenter · 2022-01-31T10:24:08Z

Codecov Report

Merging #3273 (94bd51e) into main (e23cf3b) will decrease coverage by 18.27%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main    #3273       +/-   ##
===========================================
- Coverage   60.12%   41.84%   -18.28%     
===========================================
  Files         331      199      -132     
  Lines       28444    24106     -4338     
===========================================
- Hits        17102    10088     -7014     
- Misses       9483    12981     +3498     
+ Partials     1859     1037      -822

Flag	Coverage Δ
kind-e2e-tests	`?`
unit-tests	`41.84% <ø> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/ovs/openflow/default.go	`0.00% <0.00%> (-100.00%)`	⬇️
pkg/util/runtime/runtime.go	`0.00% <0.00%> (-100.00%)`	⬇️
pkg/agent/nodeportlocal/k8s/annotations.go	`0.00% <0.00%> (-100.00%)`	⬇️
pkg/ovs/openflow/logs.go	`9.52% <0.00%> (-90.48%)`	⬇️
pkg/apis/controlplane/register.go	`0.00% <0.00%> (-90.00%)`	⬇️
pkg/agent/controller/networkpolicy/reject.go	`0.00% <0.00%> (-87.91%)`	⬇️
pkg/agent/client.go	`0.00% <0.00%> (-77.42%)`	⬇️
pkg/ovs/ovsconfig/ovs_client_linux.go	`0.00% <0.00%> (-76.93%)`	⬇️
pkg/flowaggregator/certificate.go	`0.00% <0.00%> (-76.58%)`	⬇️
...agent/controller/traceflow/traceflow_controller.go	`0.00% <0.00%> (-73.41%)`	⬇️
... and 247 more

antoninbas

Overall, LGTM

Can you check the Docker image size and make sure it is reasonable?

UBI8 image creation will be automated by the operator build and will not
be created automatically in the Antrea project scope.

Is there a reason for that? Given that all the build scripts are in this repo, IMO it would make sense to have a Github workflow here to build & push the Docker image.

antoninbas · 2022-01-31T18:47:24Z

build/images/Dockerfile.build.ubi

+USER root
+
+COPY build/images/scripts/* /usr/local/bin/
+COPY --from=antrea-build /antrea/bin/* /usr/local/bin/


missing new line at the end of file

antoninbas · 2022-01-31T18:51:35Z

build/images/base/build.sh

+    if [ "$DISTRO" == "ubuntu" ]; then
+        docker push antrea/base-ubuntu:$OVS_VERSION
+    elif [ "$DISTRO" == "ubi" ]; then
+        docker push antrea/base-ubi:$OVS_VERSION
+    fi


maybe docker push antrea/base-$DISTRO:$OVS_VERSION?

Makes sense. Thanks.

antoninbas · 2022-01-31T18:54:48Z

build/images/ovs/Dockerfile.ubi

+# https://docs.openvswitch.org/en/latest/intro/install/fedora
+FROM centos:centos7 as ovs-rpms
+
+# Some patches may not apply cleanly if another version is provided.


Comment doesn't make much sense as it is since there is no default version provided in the Dockerfile.
See build/images/ovs/Dockerfile for the "correct" comment

antoninbas · 2022-01-31T18:56:01Z

build/images/ovs/Dockerfile.ubi

+
+# Change Repository from UBI8’s to CentOS because UBI8's repository does not contain
+# enough packages required by OVS installation.
+# Use the RHEL official repository is the best choice but it's not for free.


Suggested change

# Use the RHEL official repository is the best choice but it's not for free.

# Using the official RHEL repository would be the best choice but it's not publicly accessible.

antoninbas · 2022-01-31T18:59:52Z

build/images/ovs/Dockerfile.ubi

+COPY CentOS.repo /tmp/CentOS.repo
+RUN rm -f /etc/yum.repos.d/* && mv /tmp/CentOS.repo /etc/yum.repos.d/CentOS.repo && \
+    curl https://www.centos.org/keys/RPM-GPG-KEY-CentOS-Official -o /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial && \
+    subscription-manager config --rhsm.manage_repos=0 && \
+    yum clean all -y && yum update -y && yum reinstall yum -y
+
+COPY charon-logging.conf /tmp
+COPY --from=ovs-rpms /tmp/ovs-rpms/* /tmp/ovs-rpms/
+# TODO: update the strongSwan logging config.
+RUN yum install /tmp/ovs-rpms/* -y && yum install epel-release -y && \
+    yum install iptables logrotate strongswan -y && \
+    mv /etc/logrotate.d/openvswitch /etc/logrotate.d/openvswitch-switch && \
+    sed -i "/rotate /a\    #size 100M" /etc/logrotate.d/openvswitch-switch && \
+    rm -rf /tmp/*


my recommendation would be to ensure that there is a single RUN command in the final Docker image to help keep the image small. Also, you should ensure that the last yum instruction is a yum clean all. I don't know if there are other good practices for REHL / CentOS Docker images when one wants to keep the size small.

antoninbas

PR description is no longer accurate

UBI8 image creation will be automated by the operator build and will not
be created automatically in the Antrea project scope.

antoninbas · 2022-02-11T19:08:55Z

build/images/base/Dockerfile.ubi

+FROM antrea/openvswitch-ubi:${OVS_VERSION}
+
+LABEL maintainer="Antrea <projectantrea-dev@googlegroups.com>"
+LABEL description="Takes care of building the Antrea binaries as part of building the image."


This description is invalid. If you could fix the one in build/images/base/Dockerfile as well it would be great.

antoninbas · 2022-02-11T19:10:15Z

build/images/base/Dockerfile.ubi

+
+USER root
+
+RUN yum install ipset jq -y && yum clean all 


We recently started installing iptables-wrapper in the ubuntu base image to improve portability on different Node operating systems: #3276. You probably want to do the same here?

Seems like there's a support issue with RHEL:

RHEL/CentOS 7 ship iptables 1.4, which does not support nft mode. RHEL/CentOS 8 ship a hacked version of iptables 1.8 that only supports nft mode. Therefore, neither can be used as a basis for a portable iptables-using container image.

https://github.com/kubernetes-sigs/iptables-wrappers/

I guess this also applies to CentOS 8 stream?
It should not really matter anyway if the expectation is that the Nodes will also run CentOS 8 and support nft mode.

antoninbas

LGTM

antoninbas · 2022-02-14T17:53:45Z

@ksamoray you need to ensure all commits are signed, the DCO check is failing (or you can squash and sign the resulting commit)

antoninbas · 2022-02-18T18:32:36Z

/skip-all

antoninbas · 2022-02-18T18:41:24Z

Makefile

+.PHONY: antctl-linux
+antctl-linux:


This change seems to be causing all the CI e2e tests to fail:

2022-02-15T10:45:29.5022438Z === CONT TestAntctl/testAntctlVerboseMode/RootNonVerbose 2022-02-15T10:45:29.5022857Z antctl_test.go:227: 2022-02-15T10:45:29.5023226Z Error Trace: antctl_test.go:227 2022-02-15T10:45:29.5025323Z Error: Expected nil, but got: &errors.errorString{s:"Internal error occurred: error executing command in container: failed to exec in container: failed to start exec \"301733dceb0111cd36999c637ffcbf2be462a6766c1e5d962c7a4aea3271bc6b\": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: \"antctl\": executable file not found in $PATH: unknown"} 2022-02-15T10:45:29.5026450Z Test: TestAntctl/testAntctlVerboseMode/RootNonVerbose 2022-02-15T10:45:29.5027461Z Messages: antctl stdout:

Not sure exactly why but I would guess it's a conflict with the following Makefile target:

ANTCTL_BINARIES := antctl-darwin antctl-linux antctl-windows $(ANTCTL_BINARIES): antctl-%: @GOOS=$* $(GO) build -o $(BINDIR)/$@ $(GOFLAGS) -ldflags '$(LDFLAGS)' antrea.io/antrea/cmd/antctl @if [[ $@ != *windows ]]; then \ chmod 0755 $(BINDIR)/$@; \ else \ mv $(BINDIR)/$@ $(BINDIR)/$@.exe; \ fi

With your change we end up with 2 different targets named antctl-linux. I assume the second one takes precedence in this case, but it produces a binary called antctl-linux and not a binary called antctl.

My recommendation would be to remove this target (previously antctl-ubuntu) altogether. And in the Dockerfile you can rename antctl-linux to antctl.

Makes sense, thanks

The Dockerfile needs to be updated as well, or the failure is the same.

I think this:

RUN make antrea-agent antrea-controller antrea-cni antctl

Needs to become this:

RUN make antrea-agent antrea-controller antrea-cni antctl-linux RUN cd bin && ln -s antctl-linux antctl

or

RUN make antrea-agent antrea-controller antrea-cni antctl-linux RUN mv bin/antctl-linux bin/antctl

Add the required code to build an Antrea image based on Red Hat UBI (Universal Base Image), which is in used by Red Hat platforms. Signed-off-by: Kobi Samoray <ksamoray@vmware.com>

antoninbas

LGTM

This reverts commit 43d41bb.

This reverts commit f4e10a9.

…" (antrea-io#3364)" This reverts commit f4e10a9.

Add the required code to build an Antrea image based on Red Hat UBI (Universal Base Image), which is in used by Red Hat platforms. Signed-off-by: Kobi Samoray <ksamoray@vmware.com>

…ea-io#3364) This reverts commit 43d41bb.

antoninbas reviewed Jan 31, 2022

View reviewed changes

antoninbas requested a review from salv-orlando January 31, 2022 19:02

ksamoray force-pushed the ubi8-image branch 5 times, most recently from 03390b4 to 53ccd3b Compare February 10, 2022 22:16

antoninbas reviewed Feb 11, 2022

View reviewed changes

ksamoray force-pushed the ubi8-image branch from 53ccd3b to 0c97077 Compare February 13, 2022 14:41

antoninbas previously approved these changes Feb 14, 2022

View reviewed changes

ksamoray dismissed antoninbas’s stale review via 5666e08 February 15, 2022 05:37

ksamoray force-pushed the ubi8-image branch 2 times, most recently from 5666e08 to 1e40c0a Compare February 15, 2022 10:12

antoninbas approved these changes Feb 18, 2022

View reviewed changes

antoninbas requested changes Feb 18, 2022

View reviewed changes

ksamoray force-pushed the ubi8-image branch from 1e40c0a to b434af3 Compare February 21, 2022 10:05

Support antrea-agent UBI8 based image

57454de

Add the required code to build an Antrea image based on Red Hat UBI (Universal Base Image), which is in used by Red Hat platforms. Signed-off-by: Kobi Samoray <ksamoray@vmware.com>

ksamoray force-pushed the ubi8-image branch from b434af3 to 57454de Compare February 24, 2022 14:10

antoninbas approved these changes Feb 24, 2022

View reviewed changes

antoninbas merged commit 43d41bb into antrea-io:main Feb 25, 2022

antoninbas added a commit that referenced this pull request Feb 26, 2022

Revert "Support antrea-agent UBI8 based image (#3273)"

544bfa1

This reverts commit 43d41bb.

antoninbas mentioned this pull request Feb 26, 2022

Revert "Support antrea-agent UBI8 based image" #3364

Merged

antoninbas added a commit that referenced this pull request Feb 26, 2022

Revert "Support antrea-agent UBI8 based image (#3273)" (#3364)

f4e10a9

This reverts commit 43d41bb.

ksamoray added a commit that referenced this pull request Feb 27, 2022

Revert "Revert "Support antrea-agent UBI8 based image (#3273)" (#3364)"

fb7d340

This reverts commit f4e10a9.

ksamoray added a commit to ksamoray/antrea that referenced this pull request Feb 27, 2022

Revert "Revert "Support antrea-agent UBI8 based image (antrea-io#3273)…

e37c8b3

…" (antrea-io#3364)" This reverts commit f4e10a9.

ksamoray deleted the ubi8-image branch March 1, 2022 05:27

bangqipropel pushed a commit to bangqipropel/antrea that referenced this pull request Mar 2, 2022

Revert "Support antrea-agent UBI8 based image (antrea-io#3273)" (antr…

cbc9679

…ea-io#3364) This reverts commit 43d41bb.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support antrea-agent UBI8 based image #3273

Support antrea-agent UBI8 based image #3273

ksamoray commented Jan 31, 2022 •

edited

codecov-commenter commented Jan 31, 2022 •

edited

antoninbas left a comment

antoninbas Jan 31, 2022

antoninbas Jan 31, 2022

ksamoray Feb 7, 2022

antoninbas Jan 31, 2022

antoninbas Jan 31, 2022

antoninbas Jan 31, 2022

antoninbas left a comment

antoninbas Feb 11, 2022

antoninbas Feb 11, 2022

ksamoray Feb 13, 2022 •

edited

antoninbas Feb 14, 2022

antoninbas left a comment

antoninbas commented Feb 14, 2022

antoninbas commented Feb 18, 2022

antoninbas Feb 18, 2022

ksamoray Feb 21, 2022

antoninbas Feb 22, 2022

antoninbas left a comment

	# Use the RHEL official repository is the best choice but it's not for free.
	# Using the official RHEL repository would be the best choice but it's not publicly accessible.

		.PHONY: antctl-linux
		antctl-linux:

Support antrea-agent UBI8 based image #3273

Support antrea-agent UBI8 based image #3273

Conversation

ksamoray commented Jan 31, 2022 • edited

codecov-commenter commented Jan 31, 2022 • edited

Codecov Report

antoninbas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoninbas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ksamoray Feb 13, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoninbas left a comment

Choose a reason for hiding this comment

antoninbas commented Feb 14, 2022

antoninbas commented Feb 18, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoninbas left a comment

Choose a reason for hiding this comment

ksamoray commented Jan 31, 2022 •

edited

codecov-commenter commented Jan 31, 2022 •

edited

ksamoray Feb 13, 2022 •

edited