Stuck on Installing Percona OLM Catalog #192

thecodeassassin · 2024-04-13T19:30:44Z

Hi all,

everestctl install is stuck on this step:

verestctl install --namespaces db-prod,db-dev --operator.mongodb=true --operator.postgresql=true --operator.xtradb-cluster=true --skip-wizard
2024-04-13T14:28:16Z info install/install.go:430 Installing Operator Lifecycle Manager {"component": "install"}
2024-04-13T14:28:16Z info kubernetes/kubernetes.go:328 OLM operator is already installed {"component": "install", "component": "kubernetes"}
2024-04-13T14:28:16Z info install/install.go:435 OLM has been installed {"component": "install"}
2024-04-13T14:28:16Z info install/install.go:436 Installing Percona OLM Catalog {"component": "install"}

it fails somewhere here:

func (k *Kubernetes) InstallPerconaCatalog(ctx context.Context, version *goversion.Version) error {

But with no errors.

The text was updated successfully, but these errors were encountered:

mayankshah1607 · 2024-04-15T10:40:31Z

Hi @thecodeassassin, thanks for reporting!

Could you please also provide additional information, such as:

Everest CLI version
Environment (Kubernetes verison and cluster environment)
Status of the pods in the everest-olm namespace
Verbose logs (if you install with the -v option)

fideloper · 2024-07-08T19:14:28Z

I have a similar issue today, I can't seem to install it on a fresh EKS cluster:

cli

everestctl version
ProjectName: everestctl
Version: v1.0.1
FullCommit: 40b9e1c074f75b1452143102dcf062d91bb8ab93

cluster

EKS
k8s 1.30
IPv6 cluster

status

k logs -f -n everest-olm olm-operator-6f7b945cbb-rjcdd

# outputs this multiple times per second forever:

time="2024-07-08T18:55:05Z" level=info msg="install strategy successful" csv=packageserver id=uGZ4N namespace=everest-olm phase=Installing strategy=deployment

and:

k logs -f packageserver-6b766dcf95-nc7lz -n everest-olm

# has grpc errors since I believe the pod above isn't running yet:

time="2024-07-08T18:53:43Z" level=info msg="updating PackageManifest based on CatalogSource changes: {everest-catalog everest-olm}" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm
W0708 18:54:02.897521       1 logging.go:59] [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: "everest-catalog.everest-olm.svc:50051", ServerName: "everest-catalog.everest-olm.svc:50051", }. Err: connection error: desc = "transport: Error while dialing: dial tcp [fda1:d88a:dc1::b912]:50051: i/o timeout"
time="2024-07-08T18:54:02Z" level=warning msg="error getting bundle stream" action="refresh cache" err="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp [fda1:d88a:dc1::b912]:50051: i/o timeout\"" source="{everest-catalog everest-olm}"
time="2024-07-08T18:54:02Z" level=info msg="updating PackageManifest based on CatalogSource changes: {everest-catalog everest-olm}" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm
time="2024-07-08T18:54:02Z" level=warning msg="error getting bundle stream" action="refresh cache" err="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp [fda1:d88a:dc1::b912]:50051: i/o timeout\"" source="{everest-catalog everest-olm}"
time="2024-07-08T18:54:03Z" level=info msg="updating PackageManifest based on CatalogSource changes: {everest-catalog everest-olm}" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm
time="2024-07-08T18:54:03Z" level=warning msg="error getting bundle stream" action="refresh cache" err="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp [fda1:d88a:dc1::b912]:50051: i/o timeout\"" source="{everest-catalog everest-olm}"
time="2024-07-08T18:54:04Z" level=info msg="updating PackageManifest based on CatalogSource changes: {everest-catalog everest-olm}" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm
time="2024-07-08T19:07:32Z" level=info msg="updating PackageManifest based on CatalogSource changes: {everest-catalog everest-olm}" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm

fideloper · 2024-07-08T21:54:53Z

This is occuring within an ipv4 EKS cluster as well for me.

mayankshah1607 · 2024-07-09T07:04:02Z

@fideloper from the package server logs, it seems to me that it cannot reach the catalog server. Can you check the status of the everest-catalog-xx pod? You can also describe the catalog source object for additional clues - kubectl describe catalogsource everest-catalog -n everest-olm

fideloper · 2024-07-09T11:21:02Z

I've opened up all ports, altho the error being "connection refused" sounds like nothing is listening there rather than there being a security group in the way (which would cause a timeout typically).

Some more info:

everestctl

#> everestctl -v install

2024-07-09T07:19:22.943-0400    DEBUG   commands/root.go:34     Debug logging enabled
? Namespaces managed by Everest [comma separated] everest,percona
? What operators do you want to install? MySQL, PostgreSQL
2024-07-09T07:19:31.661-0400    DEBUG   install/install.go:552  Enabling MySQL operator {"component": "install"}
2024-07-09T07:19:31.661-0400    DEBUG   install/install.go:552  Enabling PostgreSQL operator    {"component": "install"}
2024-07-09T07:19:31.852-0400    DEBUG   install/install.go:214  Everest latest version available: 1.0.1 {"component": "install"}
2024-07-09T07:19:31.852-0400    DEBUG   install/install.go:215  Everest version information &version.MetadataVersion{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Version:"1.0.1", Recommended:map[string]string{}, Supported:map[string]string{"cli":">= 1.0.0"}}   {"component": "install"}
2024-07-09T07:19:31.852-0400    INFO    install/install.go:580  Installing Operator Lifecycle Manager   {"component": "install"}
2024-07-09T07:19:31.902-0400    DEBUG   kubernetes/kubernetes.go:430    Applying "crds/olm/crds.yaml" file      {"component": "install", "component": "kubernetes"}
2024-07-09T07:19:34.821-0400    DEBUG   kubernetes/kubernetes.go:430    Applying "crds/olm/olm.yaml" file       {"component": "install", "component": "kubernetes"}
2024-07-09T07:19:39.779-0400    INFO    install/install.go:585  OLM has been installed  {"component": "install"}
2024-07-09T07:19:39.779-0400    INFO    install/install.go:586  Installing Percona OLM Catalog  {"component": "install"}
2024-07-09T07:19:39.780-0400    DEBUG   kubernetes/kubernetes.go:398    Using catalog image docker.io/percona/everest-catalog:1.0.1     {"component": "install", "component": "kubernetes"}


## This is where it just sits/waits

catalog-operator

catalog operator logs

time="2024-07-09T11:06:28Z" level=info msg="log level info"
W0709 11:06:28.822007       1 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
time="2024-07-09T11:06:28Z" level=info msg="Using in-cluster kube client config"
time="2024-07-09T11:06:28Z" level=info msg="Using in-cluster kube client config"
W0709 11:06:28.822662       1 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="rbac.authorization.k8s.io/v1, Resource=clusterrolebindings"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="rbac.authorization.k8s.io/v1, Resource=clusterroles"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="apps/v1, Resource=deployments"
time="2024-07-09T11:06:28Z" level=info msg="detected ability to filter informers" canFilter=true
time="2024-07-09T11:06:28Z" level=info msg="registering owner reference fixer" gvr="/v1, Resource=services"
W0709 11:06:28.879504       1 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
time="2024-07-09T11:06:28Z" level=info msg="connection established. cluster-version: v1.30.2-eks-db838b0"
time="2024-07-09T11:06:28Z" level=info msg="operator ready"
time="2024-07-09T11:06:28Z" level=info msg="starting informers..."
time="2024-07-09T11:06:28Z" level=info msg="informers started"
time="2024-07-09T11:06:28Z" level=info msg="waiting for caches to sync..."
time="2024-07-09T11:06:28Z" level=info msg="starting workers..."
time="2024-07-09T11:06:28Z" level=info msg="resolving sources" id=dBSfr namespace=default
time="2024-07-09T11:06:28Z" level=info msg="checking if subscriptions need update" id=dBSfr namespace=default
time="2024-07-09T11:06:28Z" level=info msg="resolving sources" id=sIOme namespace=everest-olm
time="2024-07-09T11:06:28Z" level=info msg="checking if subscriptions need update" id=sIOme namespace=everest-olm
time="2024-07-09T11:06:28Z" level=info msg="connection established. cluster-version: v1.30.2-eks-db838b0"
time="2024-07-09T11:06:28Z" level=info msg="operator ready"
time="2024-07-09T11:06:28Z" level=info msg="starting informers..."
time="2024-07-09T11:06:28Z" level=info msg="informers started"
time="2024-07-09T11:06:28Z" level=info msg="waiting for caches to sync..."
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace everest-olm" id=sIOme namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=mTObF namespace=kube-node-lease
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=mTObF namespace=kube-node-lease
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace kube-node-lease" id=mTObF namespace=kube-node-lease
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=Af7Lz namespace=kube-public
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=Af7Lz namespace=kube-public
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace kube-public" id=Af7Lz namespace=kube-public
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=k7qHT namespace=kube-system
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=k7qHT namespace=kube-system
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace default" id=dBSfr namespace=default
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace kube-system" id=k7qHT namespace=kube-system
time="2024-07-09T11:06:29Z" level=info msg="starting workers..."
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=lpBY7 namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=lpBY7 namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace everest-olm" id=lpBY7 namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=FWc4g namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=FWc4g namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace everest-olm" id=FWc4g namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=ppQL1 namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=ppQL1 namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=6p8Zn namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=6p8Zn namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace everest-olm" id=ppQL1 namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace everest-olm" id=6p8Zn namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=HXy3Y namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=HXy3Y namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace everest-olm" id=HXy3Y namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="resolving sources" id=G7ri2 namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="checking if subscriptions need update" id=G7ri2 namespace=everest-olm
time="2024-07-09T11:06:29Z" level=info msg="No subscriptions were found in namespace everest-olm" id=G7ri2 namespace=everest-olm
time="2024-07-09T11:06:41Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=false id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="registry service status invalid, need to overwrite" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="of 0 pods matching label selector, 0 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="registry pods invalid, need to overwrite" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="creating desired pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7 pod.name= pod.namespace=everest-olm
time="2024-07-09T11:06:41Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bWAk7
time="2024-07-09T11:06:41Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=FD2Hp
time="2024-07-09T11:06:41Z" level=info msg="state.Key.Namespace=everest-olm state.Key.Name=everest-catalog state.State=CONNECTING"
time="2024-07-09T11:06:41Z" level=info msg="state.Key.Namespace=everest-olm state.Key.Name=everest-catalog state.State=TRANSIENT_FAILURE"
time="2024-07-09T11:06:41Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=W2OxN
time="2024-07-09T11:06:41Z" level=error msg="UpdateStatus - error while setting CatalogSource status" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm error="Operation cannot be fulfilled on catalogsources.operators.coreos.com \"everest-catalog\": the object has been modified; please apply your changes to the latest version and try again" id=W2OxN
E0709 11:06:41.551736       1 queueinformer_operator.go:319] sync {"update" "everest-olm/everest-catalog"} failed: Operation cannot be fulfilled on catalogsources.operators.coreos.com "everest-catalog": the object has been modified; please apply your changes to the latest version and try again
time="2024-07-09T11:06:41Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:41Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:41Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:41Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:41Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:41Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=pHbqd
time="2024-07-09T11:06:41Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:41Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:42Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:42Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:42Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:42Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:42Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=pHbqd
time="2024-07-09T11:06:42Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=tA3pc
time="2024-07-09T11:06:42Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:42Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:42Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:42Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:42Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:42Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=edjWM
time="2024-07-09T11:06:42Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:42Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:43Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:43Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:43Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:43Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:43Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=edjWM
time="2024-07-09T11:06:43Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:43Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=3e0Mt
time="2024-07-09T11:06:51Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:51Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=1TXEH
time="2024-07-09T11:06:52Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:52Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=l/Im9
time="2024-07-09T11:06:55Z" level=info msg="state.Key.Namespace=everest-olm state.Key.Name=everest-catalog state.State=READY"
time="2024-07-09T11:06:55Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="resolving sources" id=J8M3K namespace=everest-olm
time="2024-07-09T11:06:55Z" level=info msg="checking if subscriptions need update" id=J8M3K namespace=everest-olm
time="2024-07-09T11:06:55Z" level=info msg="No subscriptions were found in namespace everest-olm" id=J8M3K namespace=everest-olm
time="2024-07-09T11:06:55Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=bi2pj
time="2024-07-09T11:06:55Z" level=info msg="syncing catalog source" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="synchronizing registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="checked registry server health" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm health=true id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="registry state good" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="ensuring registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="searching for current pods" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="evaluating current pod" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="of 1 pods matching label selector, 1 have the correct images and matching hash" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm correctHash=true correctImages=true current-pod.name=everest-catalog-7pmp2 current-pod.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="ensured registry server" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=dSE4I
time="2024-07-09T11:06:55Z" level=info msg="requeuing registry server sync based on polling interval 45m0s" catalogsource.name=everest-catalog catalogsource.namespace=everest-olm id=dSE4I

everest-catalog

catalog logs

time="2024-07-09T11:06:42Z" level=info msg="starting pprof endpoint" address="localhost:6060"
time="2024-07-09T11:06:42Z" level=info msg="found existing cache contents" backend=pogreb.v1 cache=/tmp/cache configs=/configs
time="2024-07-09T11:06:43Z" level=info msg="serving registry" cache=/tmp/cache configs=/configs port=50051
time="2024-07-09T11:06:43Z" level=info msg="stopped caching cpu profile data" address="localhost:6060"

olm-operato

operator logs

time="2024-07-09T11:06:28Z" level=info msg="log level info"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="/v1, Resource=services"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="/v1, Resource=configmaps"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="/v1, Resource=pods"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="rbac.authorization.k8s.io/v1, Resource=roles"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="rbac.authorization.k8s.io/v1, Resource=rolebindings"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="batch/v1, Resource=jobs"
time="2024-07-09T11:06:28Z" level=info msg="skipping irrelevant gvr" gvr="/v1, Resource=serviceaccounts"
time="2024-07-09T11:06:28Z" level=info msg="detected ability to filter informers" canFilter=true
time="2024-07-09T11:06:28Z" level=info msg="connection established. cluster-version: v1.30.2-eks-db838b0"
time="2024-07-09T11:06:28Z" level=info msg="operator ready"
time="2024-07-09T11:06:28Z" level=info msg="starting informers..."
time="2024-07-09T11:06:28Z" level=info msg="informers started"
time="2024-07-09T11:06:28Z" level=info msg="waiting for caches to sync..."
time="2024-07-09T11:06:28Z" level=info msg="starting workers..."
time="2024-07-09T11:06:28Z" level=warning msg="csv in namespace with no operatorgroups" csv=packageserver id=OhcR1 namespace=everest-olm phase=Installing
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup incorrect" csv=packageserver error="csv in namespace with no operatorgroups" id=OhcR1 namespace=everest-olm phase=Installing
I0709 11:06:28.720224       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"220891", FieldPath:""}): type: 'Warning' reason: 'NoOperatorGroup' csv in namespace with no operatorgroups
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","source":"kind source: *v2.OperatorCondition"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","source":"kind source: *v1.Role"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","source":"kind source: *v1.RoleBinding"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","source":"kind source: *v1.Deployment"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting Controller","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.Operator"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.Deployment"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.Namespace"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.CustomResourceDefinition"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.APIService"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1alpha1.Subscription"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1alpha1.InstallPlan"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1alpha1.ClusterServiceVersion"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v2.OperatorCondition"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting Controller","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1alpha1.ClusterServiceVersion"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v2.OperatorCondition"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting Controller","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1alpha1.ClusterServiceVersion"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.Deployment"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.Namespace"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.Service"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.CustomResourceDefinition"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.APIService"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1alpha1.Subscription"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v2.OperatorCondition"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","source":"kind source: *v1.PartialObjectMetadata"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting Controller","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"subscription","controllerGroup":"operators.coreos.com","controllerKind":"Subscription","source":"kind source: *v1alpha1.Subscription"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"subscription","controllerGroup":"operators.coreos.com","controllerKind":"Subscription","source":"kind source: *v1alpha1.ClusterServiceVersion"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting EventSource","controller":"subscription","controllerGroup":"operators.coreos.com","controllerKind":"Subscription","source":"kind source: *v1alpha1.InstallPlan"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting Controller","controller":"subscription","controllerGroup":"operators.coreos.com","controllerKind":"Subscription"}
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup not found" csv=packageserver id=zWF7p namespace=everest-olm operatorgroup=olm-operators phase=Installing
E0709 11:06:28.791945       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=warning msg="csv in namespace with no operatorgroups" csv=packageserver id=yJJ8F namespace=everest-olm phase=Failed
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup incorrect" csv=packageserver error="csv in namespace with no operatorgroups" id=yJJ8F namespace=everest-olm phase=Failed
I0709 11:06:28.792239       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222873", FieldPath:""}): type: 'Warning' reason: 'NoOperatorGroup' csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup not found" csv=packageserver id=TjEWD namespace=everest-olm operatorgroup=olm-operators phase=Failed
E0709 11:06:28.813495       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=warning msg="csv in namespace with no operatorgroups" csv=packageserver id=esjGB namespace=everest-olm phase=Failed
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup incorrect" csv=packageserver error="csv in namespace with no operatorgroups" id=esjGB namespace=everest-olm phase=Failed
I0709 11:06:28.813675       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222873", FieldPath:""}): type: 'Warning' reason: 'NoOperatorGroup' csv in namespace with no operatorgroups
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting workers","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","worker count":1}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting workers","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","worker count":1}
{"level":"info","ts":"2024-07-09T11:06:28Z","logger":"controllers.operatorcondition-generator","msg":"Cleaning up resources after CSV deletion","name":"packageserver","namespace":"everest-olm","selector":"olm.owner=packageserver,olm.owner.kind=ClusterServiceVersion,olm.owner.namespace=everest-olm"}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting workers","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","worker count":1}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting workers","controller":"subscription","controllerGroup":"operators.coreos.com","controllerKind":"Subscription","worker count":1}
{"level":"info","ts":"2024-07-09T11:06:28Z","msg":"Starting workers","controller":"operator","controllerGroup":"operators.coreos.com","controllerKind":"Operator","worker count":1}
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup not found" csv=packageserver id=iaksk namespace=everest-olm operatorgroup=olm-operators phase=Failed
E0709 11:06:28.827347       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=warning msg="csv in namespace with no operatorgroups" csv=packageserver id=+6GEe namespace=everest-olm phase=Failed
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup incorrect" csv=packageserver error="csv in namespace with no operatorgroups" id=+6GEe namespace=everest-olm phase=Failed
I0709 11:06:28.827583       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222873", FieldPath:""}): type: 'Warning' reason: 'NoOperatorGroup' csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup not found" csv=packageserver id=34FF+ namespace=everest-olm operatorgroup=olm-operators phase=Failed
E0709 11:06:28.867365       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=warning msg="csv in namespace with no operatorgroups" csv=packageserver id=QqyV1 namespace=everest-olm phase=Failed
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup incorrect" csv=packageserver error="csv in namespace with no operatorgroups" id=QqyV1 namespace=everest-olm phase=Failed
I0709 11:06:28.867601       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222873", FieldPath:""}): type: 'Warning' reason: 'NoOperatorGroup' csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup not found" csv=packageserver id=t1L5Z namespace=everest-olm operatorgroup=olm-operators phase=Failed
E0709 11:06:28.893089       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=warning msg="csv in namespace with no operatorgroups" csv=packageserver id=9R0rs namespace=everest-olm phase=Failed
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup incorrect" csv=packageserver error="csv in namespace with no operatorgroups" id=9R0rs namespace=everest-olm phase=Failed
I0709 11:06:28.907895       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222873", FieldPath:""}): type: 'Warning' reason: 'NoOperatorGroup' csv in namespace with no operatorgroups
time="2024-07-09T11:06:28Z" level=info msg="operatorgroup not found" csv=packageserver id=vz+J3 namespace=everest-olm operatorgroup=olm-operators phase=Failed
E0709 11:06:28.926489       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: csv in namespace with no operatorgroups
time="2024-07-09T11:06:29Z" level=warning msg="csv in namespace with no operatorgroups" csv=packageserver id=R6j0m namespace=everest-olm phase=Failed
time="2024-07-09T11:06:29Z" level=info msg="operatorgroup incorrect" csv=packageserver error="csv in namespace with no operatorgroups" id=R6j0m namespace=everest-olm phase=Failed
I0709 11:06:29.087725       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222873", FieldPath:""}): type: 'Warning' reason: 'NoOperatorGroup' csv in namespace with no operatorgroups
time="2024-07-09T11:06:29Z" level=info msg="operatorgroup not found" csv=packageserver id=/U1vD namespace=everest-olm operatorgroup=olm-operators phase=Failed
E0709 11:06:29.106256       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: csv in namespace with no operatorgroups
time="2024-07-09T11:06:29Z" level=warning msg="csv in namespace with no operatorgroups" csv=packageserver id=xkf4a namespace=everest-olm phase=Failed
time="2024-07-09T11:06:29Z" level=info msg="operatorgroup incorrect" csv=packageserver error="csv in namespace with no operatorgroups" id=xkf4a namespace=everest-olm phase=Failed
I0709 11:06:29.106478       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222885", FieldPath:""}): type: 'Warning' reason: 'NoOperatorGroup' csv in namespace with no operatorgroups
time="2024-07-09T11:06:29Z" level=info msg="operatorgroup not found" csv=packageserver id=9D+h8 namespace=everest-olm operatorgroup=olm-operators phase=Failed
E0709 11:06:29.133323       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: csv in namespace with no operatorgroups
{"level":"info","ts":"2024-07-09T11:06:29Z","logger":"controllers.operatorcondition-generator","msg":"Cleaning up resources after CSV deletion","name":"packageserver","namespace":"everest-olm","selector":"olm.owner=packageserver,olm.owner.kind=ClusterServiceVersion,olm.owner.namespace=everest-olm"}
time="2024-07-09T11:06:29Z" level=info msg="gcing children" csv=packageserver id=PmBVA namespace=everest-olm phase=Failed
time="2024-07-09T11:06:29Z" level=info msg="gcing api service v1.packages.operators.coreos.com" csv=packageserver id=PmBVA namespace=everest-olm phase=Failed
time="2024-07-09T11:06:29Z" level=info msg="InstallModes now support target namespaces. Transitioning to Pending..." csv=packageserver id=J0twy namespace=everest-olm phase=Failed
I0709 11:06:29.341209       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"e8d796bf-0500-4ba8-8150-7c12282b328e", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222885", FieldPath:""}): type: 'Normal' reason: 'RequirementsUnknown' InstallModes now support target namespaces
time="2024-07-09T11:06:29Z" level=info msg="error updating ClusterServiceVersion status: Operation cannot be fulfilled on clusterserviceversions.operators.coreos.com \"packageserver\": StorageError: invalid object, Code: 4, Key: /registry/operators.coreos.com/clusterserviceversions/everest-olm/packageserver, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e8d796bf-0500-4ba8-8150-7c12282b328e, UID in object meta: " csv=packageserver id=TbEpU namespace=everest-olm phase=Failed
E0709 11:06:29.350346       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: error updating ClusterServiceVersion status: Operation cannot be fulfilled on clusterserviceversions.operators.coreos.com "packageserver": StorageError: invalid object, Code: 4, Key: /registry/operators.coreos.com/clusterserviceversions/everest-olm/packageserver, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e8d796bf-0500-4ba8-8150-7c12282b328e, UID in object meta: 
{"level":"error","ts":"2024-07-09T11:06:29Z","logger":"controllers.operatorcondition-generator","msg":"Removing finalizer","name":"packageserver","namespace":"everest-olm","error":"clusterserviceversions.operators.coreos.com \"packageserver\" not found","stacktrace":"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators.(*OperatorConditionGeneratorReconciler).processFinalizer\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/pkg/controller/operators/operatorconditiongenerator_controller.go:284\ngithub.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators.(*OperatorConditionGeneratorReconciler).Reconcile\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/pkg/controller/operators/operatorconditiongenerator_controller.go:103\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":"2024-07-09T11:06:29Z","msg":"Reconciler error","controller":"clusterserviceversion","controllerGroup":"operators.coreos.com","controllerKind":"ClusterServiceVersion","ClusterServiceVersion":{"name":"packageserver","namespace":"everest-olm"},"namespace":"everest-olm","name":"packageserver","reconcileID":"29b7e7fc-a679-4869-9fe0-813235ab9e6f","error":"clusterserviceversions.operators.coreos.com \"packageserver\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
time="2024-07-09T11:06:29Z" level=warning msg="error adding operatorgroup annotations" csv=packageserver error="Operation cannot be fulfilled on clusterserviceversions.operators.coreos.com \"packageserver\": the object has been modified; please apply your changes to the latest version and try again" id=0m7WS namespace=everest-olm opgroup=olm-operators phase=
time="2024-07-09T11:06:29Z" level=info msg="operatorgroup incorrect" csv=packageserver error="Operation cannot be fulfilled on clusterserviceversions.operators.coreos.com \"packageserver\": the object has been modified; please apply your changes to the latest version and try again" id=0m7WS namespace=everest-olm phase=
time="2024-07-09T11:06:29Z" level=info msg="not part of any operatorgroup, no annotations" csv=packageserver id=07Lxm namespace=everest-olm phase=
E0709 11:06:29.556823       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: Operation cannot be fulfilled on clusterserviceversions.operators.coreos.com "packageserver": the object has been modified; please apply your changes to the latest version and try again
time="2024-07-09T11:06:29Z" level=info msg="operatorgroup incorrect" csv=packageserver error="<nil>" id=gj+dr namespace=everest-olm phase=
time="2024-07-09T11:06:29Z" level=info msg="not part of any operatorgroup, no annotations" csv=packageserver id=/u6XW namespace=everest-olm phase=
{"level":"error","ts":"2024-07-09T11:06:29Z","msg":"Reconciler error","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","OperatorCondition":{"name":"packageserver","namespace":"everest-olm"},"namespace":"everest-olm","name":"packageserver","reconcileID":"4d3e5532-ba59-49f5-9722-b86de9b51be4","error":"Deployment.apps \"packageserver\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
time="2024-07-09T11:06:29Z" level=info msg="scheduling ClusterServiceVersion for requirement verification" csv=packageserver id=4l5to namespace=everest-olm phase=
I0709 11:06:29.579230       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"66d35921-db22-4506-b4d9-e55701a3b8a1", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222913", FieldPath:""}): type: 'Normal' reason: 'RequirementsUnknown' requirements not yet checked
{"level":"error","ts":"2024-07-09T11:06:29Z","msg":"Reconciler error","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","OperatorCondition":{"name":"packageserver","namespace":"everest-olm"},"namespace":"everest-olm","name":"packageserver","reconcileID":"01963e77-1f63-43e1-8c63-9cb61250aa4b","error":"rolebindings.rbac.authorization.k8s.io \"packageserver\" already exists","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":"2024-07-09T11:06:29Z","msg":"Reconciler error","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","OperatorCondition":{"name":"packageserver","namespace":"everest-olm"},"namespace":"everest-olm","name":"packageserver","reconcileID":"e64fcc67-ae2e-42fe-85b8-e44e86d2d799","error":"Deployment.apps \"packageserver\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":"2024-07-09T11:06:29Z","msg":"Reconciler error","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","OperatorCondition":{"name":"packageserver","namespace":"everest-olm"},"namespace":"everest-olm","name":"packageserver","reconcileID":"7e645883-80c3-4c8d-b2f9-f101cae827dc","error":"Deployment.apps \"packageserver\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
time="2024-07-09T11:06:29Z" level=error msg="update existing cluster role failed: nil" error="Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io \"olm.og.olm-operators.view-82gdxvBtMmhgjcWZVmNGIauWAGlyiVPmTAV5sk\": the object has been modified; please apply your changes to the latest version and try again"
{"level":"error","ts":"2024-07-09T11:06:29Z","msg":"Reconciler error","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","OperatorCondition":{"name":"packageserver","namespace":"everest-olm"},"namespace":"everest-olm","name":"packageserver","reconcileID":"a48ce53b-b0f6-4e33-b164-1451ab06c793","error":"Deployment.apps \"packageserver\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
time="2024-07-09T11:06:29Z" level=info msg="scheduling ClusterServiceVersion for install" csv=packageserver id=fecU9 namespace=everest-olm phase=Pending
I0709 11:06:29.703895       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"66d35921-db22-4506-b4d9-e55701a3b8a1", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222918", FieldPath:""}): type: 'Normal' reason: 'AllRequirementsMet' all requirements found, attempting install
time="2024-07-09T11:06:29Z" level=info msg="scheduling ClusterServiceVersion for install" csv=packageserver id=oE1VR namespace=everest-olm phase=Pending
I0709 11:06:29.724990       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"66d35921-db22-4506-b4d9-e55701a3b8a1", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222918", FieldPath:""}): type: 'Normal' reason: 'AllRequirementsMet' all requirements found, attempting install
{"level":"error","ts":"2024-07-09T11:06:29Z","msg":"Reconciler error","controller":"operatorcondition","controllerGroup":"operators.coreos.com","controllerKind":"OperatorCondition","OperatorCondition":{"name":"packageserver","namespace":"everest-olm"},"namespace":"everest-olm","name":"packageserver","reconcileID":"47ce4b56-b791-4257-b54a-48c0ad158d43","error":"Deployment.apps \"packageserver\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
time="2024-07-09T11:06:29Z" level=info msg="error updating ClusterServiceVersion status: Operation cannot be fulfilled on clusterserviceversions.operators.coreos.com \"packageserver\": the object has been modified; please apply your changes to the latest version and try again" csv=packageserver id=93h0i namespace=everest-olm phase=Pending
E0709 11:06:29.733122       1 queueinformer_operator.go:319] sync {"update" "everest-olm/packageserver"} failed: error updating ClusterServiceVersion status: Operation cannot be fulfilled on clusterserviceversions.operators.coreos.com "packageserver": the object has been modified; please apply your changes to the latest version and try again
I0709 11:06:29.931982       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"66d35921-db22-4506-b4d9-e55701a3b8a1", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222923", FieldPath:""}): type: 'Normal' reason: 'InstallSucceeded' waiting for install components to report healthy
time="2024-07-09T11:06:29Z" level=info msg="install strategy successful" csv=packageserver id=1xC5l namespace=everest-olm phase=Installing strategy=deployment
I0709 11:06:29.984727       1 event.go:298] Event(v1.ObjectReference{Kind:"ClusterServiceVersion", Namespace:"everest-olm", Name:"packageserver", UID:"66d35921-db22-4506-b4d9-e55701a3b8a1", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"222950", FieldPath:""}): type: 'Normal' reason: 'InstallWaiting' apiServices not installed
time="2024-07-09T11:06:30Z" level=info msg="install strategy successful" csv=packageserver id=jUSII namespace=everest-olm phase=Installing strategy=deployment
time="2024-07-09T11:06:30Z" level=info msg="install strategy successful" csv=packageserver id=EmA5l namespace=everest-olm phase=Installing strategy=deployment
time="2024-07-09T11:06:30Z" level=info msg="install strategy successful" csv=packageserver id=fLR5U namespace=everest-olm phase=Installing strategy=deployment
time="2024-07-09T11:06:30Z" level=info msg="install strategy successful" csv=packageserver id=Ii6Eu namespace=everest-olm phase=Installing strategy=deployment
time="2024-07-09T11:06:30Z" level=info msg="install strategy successful" csv=packageserver id=l6QJ3 namespace=everest-olm phase=Installing strategy=deployment
time="2024-07-09T11:06:30Z" level=info msg="install strategy successful" csv=packageserver id=GD27P namespace=everest-olm phase=Installing strategy=deployment
time="2024-07-09T11:06:30Z" level=info msg="install strategy successful" csv=packageserver id=eE1kI namespace=everest-olm phase=Installing strategy=deployment
time="2024-07-09T11:06:30Z" level=info msg="install strategy successful" csv=packageserver id=njvYn namespace=everest-olm phase=Installing strategy=deployment

Those bottom few log lines repeat multiple times per second forever

packageserver

packageserver logs

time="2024-07-09T11:06:31Z" level=info msg="Using in-cluster kube client config"
time="2024-07-09T11:06:31Z" level=info msg="Defaulting Interval to '12h0m0s'"
I0709 11:06:31.330754       1 handler.go:275] Adding GroupVersion packages.operators.coreos.com v1 to ResourceManager
time="2024-07-09T11:06:31Z" level=info msg="connection established. cluster-version: v1.30.2-eks-db838b0"
time="2024-07-09T11:06:31Z" level=info msg="operator ready"
time="2024-07-09T11:06:31Z" level=info msg="starting informers..."
time="2024-07-09T11:06:31Z" level=info msg="informers started"
time="2024-07-09T11:06:31Z" level=info msg="waiting for caches to sync..."
time="2024-07-09T11:06:31Z" level=info msg="starting workers..."
I0709 11:06:31.449997       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
I0709 11:06:31.450126       1 shared_informer.go:311] Waiting for caches to sync for RequestHeaderAuthRequestController
I0709 11:06:31.450019       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I0709 11:06:31.450146       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0709 11:06:31.450169       1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::apiserver.local.config/certificates/apiserver.crt::apiserver.local.config/certificates/apiserver.key"
I0709 11:06:31.450028       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
I0709 11:06:31.450399       1 shared_informer.go:311] Waiting for caches to sync for RequestHeaderAuthRequestController
I0709 11:06:31.450058       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
I0709 11:06:31.450412       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0709 11:06:31.450478       1 secure_serving.go:213] Serving securely on [::]:5443
I0709 11:06:31.450508       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I0709 11:06:31.450063       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I0709 11:06:31.450575       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0709 11:06:31.450097       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
I0709 11:06:31.450603       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0709 11:06:31.551235       1 shared_informer.go:318] Caches are synced for RequestHeaderAuthRequestController
I0709 11:06:31.551254       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0709 11:06:31.551274       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0709 11:06:31.551300       1 shared_informer.go:318] Caches are synced for RequestHeaderAuthRequestController
I0709 11:06:31.551363       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0709 11:06:31.551235       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
time="2024-07-09T11:06:41Z" level=info msg="connecting to source" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm
W0709 11:06:41.414565       1 logging.go:59] [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: "everest-catalog.everest-olm.svc:50051", ServerName: "everest-catalog.everest-olm.svc:50051", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.20.204.21:50051: connect: connection refused"
time="2024-07-09T11:06:41Z" level=info msg="updating PackageManifest based on CatalogSource changes: {everest-catalog everest-olm}" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm
time="2024-07-09T11:06:41Z" level=warning msg="error getting bundle stream" action="refresh cache" err="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 172.20.204.21:50051: connect: connection refused\"" source="{everest-catalog everest-olm}"
time="2024-07-09T11:06:42Z" level=info msg="updating PackageManifest based on CatalogSource changes: {everest-catalog everest-olm}" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm
time="2024-07-09T11:06:42Z" level=warning msg="error getting bundle stream" action="refresh cache" err="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 172.20.204.21:50051: connect: connection refused\"" source="{everest-catalog everest-olm}"
W0709 11:06:42.416792       1 logging.go:59] [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: "everest-catalog.everest-olm.svc:50051", ServerName: "everest-catalog.everest-olm.svc:50051", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.20.204.21:50051: connect: connection refused"
W0709 11:06:43.993246       1 logging.go:59] [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: "everest-catalog.everest-olm.svc:50051", ServerName: "everest-catalog.everest-olm.svc:50051", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.20.204.21:50051: connect: connection refused"
W0709 11:06:46.371165       1 logging.go:59] [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: "everest-catalog.everest-olm.svc:50051", ServerName: "everest-catalog.everest-olm.svc:50051", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.20.204.21:50051: connect: connection refused"
W0709 11:06:50.450815       1 logging.go:59] [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: "everest-catalog.everest-olm.svc:50051", ServerName: "everest-catalog.everest-olm.svc:50051", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.20.204.21:50051: connect: connection refused"
time="2024-07-09T11:06:55Z" level=info msg="updating PackageManifest based on CatalogSource changes: {everest-catalog everest-olm}" action="sync catalogsource" address="everest-catalog.everest-olm.svc:50051" name=everest-catalog namespace=everest-olm
time="2024-07-09T11:06:55Z" level=warning msg="error getting bundle stream" action="refresh cache" err="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 172.20.204.21:50051: connect: connection refused\"" source="{everest-catalog everest-olm}"

fideloper · 2024-07-09T11:31:49Z

I got this working, we had to open up the security groups so the control plane could talk to the nodes over port 5443 (where as I had previously setup security groups for just intra-node communication on port 5443).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stuck on Installing Percona OLM Catalog #192

Stuck on Installing Percona OLM Catalog #192

thecodeassassin commented Apr 13, 2024

mayankshah1607 commented Apr 15, 2024

fideloper commented Jul 8, 2024

fideloper commented Jul 8, 2024

mayankshah1607 commented Jul 9, 2024

fideloper commented Jul 9, 2024 •

edited

Loading

fideloper commented Jul 9, 2024 •

edited

Loading

Stuck on Installing Percona OLM Catalog #192

Stuck on Installing Percona OLM Catalog #192

Comments

thecodeassassin commented Apr 13, 2024

mayankshah1607 commented Apr 15, 2024

fideloper commented Jul 8, 2024

fideloper commented Jul 8, 2024

mayankshah1607 commented Jul 9, 2024

fideloper commented Jul 9, 2024 • edited Loading

everestctl

catalog-operator

everest-catalog

olm-operato

packageserver

fideloper commented Jul 9, 2024 • edited Loading

fideloper commented Jul 9, 2024 •

edited

Loading

fideloper commented Jul 9, 2024 •

edited

Loading