Openshift test - install Minio Operator #1435

pjuarezd · 2023-02-06T10:25:44Z

Objective:

To test the install of Minio Operator in Openshift.
This first iteration tests "community-operators" catalog, expecting to extend it to the other 2 catalogs (redhat-marketplace, certified-operators)

Reason:

We need to ensure that introduced changes to Operator do not break the Openshift deployment installs, at least fresh clean installs should be flawless.

Environment:

This test is by far not going to be able to be ran in a github action as to run crc expectos 20gb of ram and 12 cores.
The test run expects that no install of crc or podman are present in the running host.

!!WARNING:

The test downloads the binaries and creates a symlink of the podman command, if the running host already has a podman binary in $PATH, it could be overwrited by the symlink
The test considers the boot and delete of the crc virtual machine, make sure of backup your ~/.crc/cache dir, OR setup the environment variable export OPERATOR_ENABLE_MANUAL_TESTING=ON to prevent the virtual machine deletion.
The test breaks if catalogsource or subscription resources used by the test exists in the Openshift cluster, make sure to either delete them or backup your crc vm and start with a freshcrc vm on each test (prefered).

How It looks:

> ./testing/deploy-openshift-4.sh 
Installing temporal binaries in /tmp/tmp.oCqAM0Iztd
yq
#################################################################################################################################################################################### 100.0%
opm
#################################################################################################################################################################################### 100.0%
crc
#################################################################################################################################################################################### 100.0%
Configuring crc
Successfully configured consent-telemetry to no
Successfully configured skip-check-root-user to true
Successfully configured kubeadmin-password to #######
INFO Using bundle path /home/pedro/.crc/cache/crc_libvirt_4.12.0_amd64.crcbundle 
INFO Checking if running as non-root              
WARN Skipping above check...                      
INFO Checking if running inside WSL2              
INFO Checking if crc-admin-helper executable is cached 
INFO Checking for obsolete admin-helper executable 
INFO Checking if running on a supported CPU architecture 
INFO Checking minimum RAM requirements            
INFO Checking if crc executable symlink exists    
INFO Creating symlink for crc executable          
INFO Checking if Virtualization is enabled        
INFO Checking if KVM is enabled                   
INFO Checking if libvirt is installed             
INFO Checking if user is part of libvirt group    
INFO Checking if active user/process is currently part of the libvirt group 
INFO Checking if libvirt daemon is running        
INFO Checking if a supported libvirt version is installed 
INFO Checking if crc-driver-libvirt is installed  
INFO Checking crc daemon systemd service          
INFO Checking crc daemon systemd socket units     
INFO Checking if AppArmor is configured           
INFO Checking if systemd-networkd is running      
INFO Checking if NetworkManager is installed      
INFO Checking if NetworkManager service is running 
INFO Checking if dnsmasq configurations file exist for NetworkManager 
INFO Checking if the systemd-resolved service is running 
INFO Checking if /etc/NetworkManager/dispatcher.d/99-crc.sh exists 
INFO Checking if libvirt 'crc' network is available 
INFO Checking if libvirt 'crc' network is active  
INFO Checking if CRC bundle is extracted in '$HOME/.crc' 
INFO Checking if /home/pedro/.crc/cache/crc_libvirt_4.12.0_amd64.crcbundle exists 
Your system is correctly setup for using CRC. Use 'crc start' to start the instance
INFO Checking if running as non-root              
WARN Skipping above check...                      
INFO Checking if running inside WSL2              
INFO Checking if crc-admin-helper executable is cached 
INFO Checking for obsolete admin-helper executable 
INFO Checking if running on a supported CPU architecture 
INFO Checking minimum RAM requirements            
INFO Checking if crc executable symlink exists    
INFO Checking if Virtualization is enabled        
INFO Checking if KVM is enabled                   
INFO Checking if libvirt is installed             
INFO Checking if user is part of libvirt group    
INFO Checking if active user/process is currently part of the libvirt group 
INFO Checking if libvirt daemon is running        
INFO Checking if a supported libvirt version is installed 
INFO Checking if crc-driver-libvirt is installed  
INFO Checking crc daemon systemd socket units     
INFO Checking if AppArmor is configured           
INFO Checking if systemd-networkd is running      
INFO Checking if NetworkManager is installed      
INFO Checking if NetworkManager service is running 
INFO Checking if dnsmasq configurations file exist for NetworkManager 
INFO Checking if the systemd-resolved service is running 
INFO Checking if /etc/NetworkManager/dispatcher.d/99-crc.sh exists 
INFO Checking if libvirt 'crc' network is available 
INFO Checking if libvirt 'crc' network is active  
INFO Loading bundle: crc_libvirt_4.12.0_amd64...  
INFO Starting CRC VM for openshift 4.12.0...      
INFO CRC instance is running with IP 192.168.130.11 
INFO CRC VM is running                            
INFO Configuring shared directories               
INFO Check internal and public DNS query...       
INFO Check DNS query from host...                 
WARN Wildcard DNS resolution for apps-crc.testing does not appear to be working 
INFO Verifying validity of the kubelet certificates... 
INFO Starting kubelet service                     
INFO Waiting for kube-apiserver availability... [takes around 2min] 
INFO Waiting for user's pull secret part of instance disk... 
INFO Overriding password for kubeadmin user       
INFO Starting openshift instance... [waiting for the cluster to stabilize] 
INFO 8 operators are progressing: dns, image-registry, ingress, kube-storage-version-migrator, network, ... 
INFO 8 operators are progressing: dns, image-registry, ingress, kube-storage-version-migrator, network, ... 
INFO 6 operators are progressing: image-registry, ingress, kube-storage-version-migrator, network, openshift-controller-manager, ... 
INFO 6 operators are progressing: image-registry, ingress, kube-storage-version-migrator, network, openshift-controller-manager, ... 
INFO 5 operators are progressing: image-registry, kube-storage-version-migrator, network, openshift-controller-manager, service-ca 
INFO 5 operators are progressing: image-registry, kube-storage-version-migrator, network, openshift-controller-manager, service-ca 
INFO 5 operators are progressing: image-registry, kube-storage-version-migrator, network, openshift-controller-manager, service-ca 
INFO 5 operators are progressing: image-registry, kube-storage-version-migrator, network, openshift-controller-manager, service-ca 
INFO 5 operators are progressing: image-registry, kube-storage-version-migrator, network, openshift-controller-manager, service-ca 
INFO 5 operators are progressing: image-registry, kube-storage-version-migrator, network, openshift-controller-manager, service-ca 
INFO 5 operators are progressing: image-registry, kube-storage-version-migrator, network, openshift-controller-manager, service-ca 
INFO 4 operators are progressing: image-registry, kube-storage-version-migrator, openshift-controller-manager, service-ca 
INFO Operator image-registry is progressing       
INFO All operators are available. Ensuring stability... 
INFO Operators are stable (2/3)...                
INFO Operators are stable (3/3)...                
INFO Adding crc-admin and crc-developer contexts to kubeconfig... 
Started the OpenShift cluster.

The server is accessible via web console at:
  https://console-openshift-console.apps-crc.testing

Log in as administrator:
  Username: kubeadmin
  Password: #####

Log in as user:
  Username: developer
  Password: developer

Use the 'oc' command line interface:
  $ eval $(crc oc-env)
  $ oc login -u developer https://api.crc.testing:6443
CRC version: 2.13.1+3b466f8
OpenShift version: 4.12.0
Podman version: 4.3.1
Waiting for podman vm come online (5m timeout)
REPOSITORY  TAG         IMAGE ID    CREATED     SIZE
Create Marketplace for catalog 'certified-operators'
Compiling operator in current branch
Checking dependencies
Installing golangci-lint
Installing govulncheck
?   	github.com/minio/operator	[no test files]
?   	github.com/minio/operator/pkg/apis/minio.min.io	[no test files]
?   	github.com/minio/operator/pkg/apis/minio.min.io/v1	[no test files]
ok  	github.com/minio/operator/pkg/apis/minio.min.io/v2	(cached)
?   	github.com/minio/operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/minio/operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/minio/operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/minio/operator/pkg/client/clientset/versioned/typed/minio.min.io/v2	[no test files]
?   	github.com/minio/operator/pkg/client/clientset/versioned/typed/minio.min.io/v2/fake	[no test files]
?   	github.com/minio/operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/minio/operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/minio/operator/pkg/client/informers/externalversions/minio.min.io	[no test files]
?   	github.com/minio/operator/pkg/client/informers/externalversions/minio.min.io/v2	[no test files]
?   	github.com/minio/operator/pkg/client/listers/minio.min.io/v2	[no test files]
ok  	github.com/minio/operator/pkg/controller/cluster	(cached)
?   	github.com/minio/operator/pkg/controller/cluster/certificates	[no test files]
ok  	github.com/minio/operator/pkg/resources/configmaps	(cached)
?   	github.com/minio/operator/pkg/resources/deployments	[no test files]
?   	github.com/minio/operator/pkg/resources/secrets	[no test files]
?   	github.com/minio/operator/pkg/resources/services	[no test files]
ok  	github.com/minio/operator/pkg/resources/statefulsets	(cached)
?   	github.com/minio/operator/resources	[no test files]
?   	github.com/minio/operator/subnet	[no test files]
Running lint check
Checking dependencies
Installing golangci-lint
Installing govulncheck
Building 'logsearchapi' binary
?   	github.com/minio/operator/logsearchapi	[no test files]
ok  	github.com/minio/operator/logsearchapi/server	(cached)
e6d019c454d89908693d9535810e9a2dd07817aa50f0d7b84d7a496f7cd1574e
push operator image to crc registry
Login Succeeded!
Image Stream for operator:
NAME       IMAGE REPOSITORY                                                                       TAGS   UPDATED
operator   default-route-openshift-image-registry.apps-crc.testing/openshift-operators/operator   noop   Less than a second ago
imagestream.image.openshift.io/operator image lookup updated
Compiling operator bundle for certified-operators
15e1867cc70d73af37c6ea0fa1ef3904a6d6844b7bc874f7e4519226f7cb8259
Login Succeeded!
push operator-bundle to crc registry
Image Stream for operator-bundle
NAME              IMAGE REPOSITORY                                                                                TAGS   UPDATED
operator-bundle   default-route-openshift-image-registry.apps-crc.testing/openshift-marketplace/operator-bundle   noop   Less than a second ago
imagestream.image.openshift.io/operator-bundle image lookup updated
Compiling marketplace index
WARN[0000] DEPRECATION NOTICE:
Sqlite-based catalogs and their related subcommands are deprecated. Support for
them will be removed in a future release. Please migrate your catalog workflows
to the new file-based catalog format. 
INFO[0000] building the index                            bundles="[default-route-openshift-image-registry.apps-crc.testing/openshift-marketplace/operator-bundle:noop]"
INFO[0000] Could not find optional dependencies file     file=bundle_tmp724898037/metadata load=annotations with=./bundle_tmp724898037
INFO[0000] Could not find optional properties file       file=bundle_tmp724898037/metadata load=annotations with=./bundle_tmp724898037
INFO[0000] Could not find optional dependencies file     file=bundle_tmp724898037/metadata load=annotations with=./bundle_tmp724898037
INFO[0000] Could not find optional properties file       file=bundle_tmp724898037/metadata load=annotations with=./bundle_tmp724898037
INFO[0000] Generating dockerfile                         bundles="[default-route-openshift-image-registry.apps-crc.testing/openshift-marketplace/operator-bundle:noop]"
INFO[0000] writing dockerfile: ./index.Dockerfile2935531590  bundles="[default-route-openshift-image-registry.apps-crc.testing/openshift-marketplace/operator-bundle:noop]"
INFO[0000] running podman build                          bundles="[default-route-openshift-image-registry.apps-crc.testing/openshift-marketplace/operator-bundle:noop]"
INFO[0000] [podman build --format docker -f ./index.Dockerfile2935531590 -t default-route-openshift-image-registry.apps-crc.testing/openshift-marketplace/minio-operator-index:noop .]  bundles="[default-route-openshift-image-registry.apps-crc.testing/openshift-marketplace/operator-bundle:noop]"
push minio-operator-index to crc registry
Image Stream for minio-operator-index
imagestream.image.openshift.io/minio-operator-index image lookup updated
Wait for ImageStream minio-operator-index to be local available
imagestream.image.openshift.io/minio-operator-index condition met
Create 'Test Minio Operators' marketplace catalog source
catalogsource.operators.coreos.com/minio-test-operators created
Catalog Source:
NAME                   DISPLAY                TYPE   PUBLISHER   AGE
minio-test-operators   Test Minio Operators   grpc   MinIO       5s
deleting pod minio-test-operators-gvqds -n openshift-marketplace
pod "minio-test-operators-gvqds" deleted
Waiting for Package manifest to be ready (5m timeout)
.............minio-operator-noop                                Test Minio Operators   20s
Installing Operator from catalog 'certified-operators'
subscription.operators.coreos.com/test-subscription created
Subscription:
NAME                PACKAGE               SOURCE                 CHANNEL
test-subscription   minio-operator-noop   minio-test-operators   stable
Wait subscription to be ready (10m timeout)
subscription.operators.coreos.com/test-subscription condition met
Install plan:
NAME            CSV                     APPROVAL    APPROVED
install-krz6s   minio-operator.v4.5.8   Automatic   true
Waiting for install plan to be completed (10m timeout)
installplan.operators.coreos.com/install-krz6s condition met
Deployment:
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
minio-operator   0/2     2            0           1s
Waiting for Operator Deployment to come online (5m timeout)
deployment.apps/minio-operator condition met
start - get data to verify proper image is being used
Pods:
NAME                              READY   STATUS    RESTARTS   AGE
console-5b47cd945f-nxpq5          1/1     Running   0          20s
minio-operator-5df7b8b7c9-hnrbm   1/1     Running   0          20s
Images:
                  containerImage: default-route-openshift-image-registry.apps-crc.testing/openshift-operators/operator:noop
    Image:         quay.io/minio/console@sha256:9827b3b7c89d5b809aa8b90233fd6802711e664aeba87288488255a012ba9d28
    Image ID:      quay.io/minio/console@sha256:7a985a1b8d2b8eb33c8e1fac3ebb736f54f0f68d7d34c0650ed0aad41ffbe698
                  containerImage: default-route-openshift-image-registry.apps-crc.testing/openshift-operators/operator:noop
    Image:          quay.io/minio/operator:v4.5.8
    Image ID:       quay.io/minio/operator@sha256:104e88ec29994c4cc59763e55c41def855ad852f76afb2edffa7758e5791ba6b
destroy_crc
Cluster will be destroyed for automated testing
INFO Stopping kubelet and all containers...       
INFO Stopping the instance, this may take a few minutes... 
Stopped the instance
Deleted the instance
Removing temporary binaries in: /tmp/tmp.oCqAM0Iztd

real	11m12.584s
user	1m49.586s
sys	0m11.620s

… container.

dvaldivia

LGTM.

cniackz

LGTM

* test Operator install on Openshift using crc * moar debagging * openshift install test almost complete * some bugfixing, save checkpoint * Openshift operator install test Complete * bugfix: test was stuck, test catalogsource pod was unable to pull the container. * shellcheck * rename flag to sync with minio#1441

pjuarezd added 7 commits February 4, 2023 05:13

test Operator install on Openshift using crc

d31bd9a

moar debagging

7d6b9a5

openshift install test almost complete

f5cd7bc

some bugfixing, save checkpoint

b909dc6

Openshift operator install test Complete

9f1b82d

bugfix: test was stuck, test catalogsource pod was unable to pull the…

c41ba4b

… container.

shellcheck

8747079

pjuarezd requested review from cniackz and dvaldivia February 6, 2023 10:47

pjuarezd self-assigned this Feb 6, 2023

dvaldivia previously approved these changes Feb 6, 2023

View reviewed changes

rename flag to sync with minio#1441

a2d2dce

pjuarezd dismissed dvaldivia’s stale review via a2d2dce February 7, 2023 18:18

dvaldivia approved these changes Feb 9, 2023

View reviewed changes

pjuarezd requested review from allanrogerr, dilverse, donatello and harshavardhana February 14, 2023 00:04

cniackz approved these changes Feb 28, 2023

View reviewed changes

cniackz merged commit 757a88c into minio:master Feb 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Openshift test - install Minio Operator #1435

Openshift test - install Minio Operator #1435

pjuarezd commented Feb 6, 2023

dvaldivia left a comment

cniackz left a comment

Openshift test - install Minio Operator #1435

Openshift test - install Minio Operator #1435

Conversation

pjuarezd commented Feb 6, 2023

Objective:

Reason:

Environment:

!!WARNING:

How It looks:

dvaldivia left a comment

Choose a reason for hiding this comment

cniackz left a comment

Choose a reason for hiding this comment