Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor catalog list services #4338

Merged
merged 3 commits into from
Jan 20, 2021
Merged

Refactor catalog list services #4338

merged 3 commits into from
Jan 20, 2021

Conversation

dharmit
Copy link
Member

@dharmit dharmit commented Jan 2, 2021

What type of PR is this?
/kind code-refactoring

What does this PR do / why we need it:
This PR refactors the underlying code for odo catalog list services. It makes the code more readable.

We need this because odo will be supporting both Operator Hub as well as Service Catalog backends to let users spin up services from. This PR removes if..else blocks and various variables that made the code quite confusing.

Which issue(s) this PR fixes:

Fixes NA
Part of #4298

PR acceptance criteria:

How to test changes / Special notes to the reviewer:
Check output of odo catalog list services in following scenarios:

  • a fresh minikube cluster with no Operators or Service Catalog enabled
  • enable OLM addon (minikube addons enable olm) and install some Operator (for example install Service Binding Operator)
  • enable service catalog using helm cli (download and install latest helm cli from its release page) by following below steps: 
    $ git clone https://github.com/kubernetes-sigs/service-catalog/
$ cd service-catalog/charts/catalog
# enable the CRDs related to Service Catalog
$ helm install . --generate-name

# install minibroker (a kind of ClusterServiceBroker)
$ helm repo add minibroker https://minibroker.blob.core.windows.net/charts

# make sure the pods related to minibroker are in running state before moving to next step
$ kubectl get pods

# this installs services (ClusterServiceClasses) provided by minibroker 
$ helm install minibroker --namespace default minibroker/minibroker

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dharmit

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. label Jan 2, 2021
@dharmit dharmit requested review from girishramnani, mohammedzee1000 and mik-dass and removed request for prietyc123 and rnapoles-rh January 2, 2021 09:21
@dharmit
Copy link
Member Author

dharmit commented Jan 5, 2021

All failures are attributed to the same error:

Running odo with args [odo catalog list services]
[odo] I0102 11:21:31.838297   64220 util.go:723] HTTPGetRequest: https://raw.githubusercontent.com/openshift/odo/master/build/VERSION
[odo] I0102 11:21:31.838563   64220 util.go:744] Response will be cached in /tmp/odohttpcache for 1h0m0s
[odo] I0102 11:21:31.838718   64220 util.go:757] Cached response used.
[odo] I0102 11:21:31.898810   64220 operators.go:35] Fetching list of operators installed in cluster
[odo]  ✗  clusterserviceclasses.servicecatalog.k8s.io is forbidden: User "developer" cannot list resource "clusterserviceclasses" in API group "servicecatalog.k8s.io" at the cluster scope
Deleting project: xzcksamxjy

I see the same error when I try to run odo catalog list services on my CRC box.

odo is failing to list resources of type ClusterServiceClass (a CRD provided by Service Catalog) on OpenShift. With OpenShift 4.5, support for Service Catalog is deprecated. I tried to install Service Catalog on CRC using helm. But it's not working out smoothly.

So, as far as OCP and Service Catalog is concerned, I'm getting inclined towards silently ignoring this error. The idea is to support SC smoothly on minikube/k8s.

Will update the PR accordinlgly.

@girishramnani
Copy link
Contributor

/retest

@dharmit
Copy link
Member Author

dharmit commented Jan 6, 2021

CI failures attributed to #4347

/retest

@dharmit
Copy link
Member Author

dharmit commented Jan 12, 2021

Failure log indicates that odo push is failing because it can't find the Secret that is created by SBO when we link a component to a Service. However, what's weird is that it's searching for this Secret during odo push executed after a successful odo unlink.

After odo unlink, odo should not be searching for the Secret when we're doing odo push. Able to reproduce this locally on CRC as well.

Debugging.

 Running odo with args [odo unlink EtcdCluster/example]
[odo] I0111 15:01:06.421884   65862 util.go:727] HTTPGetRequest: https://raw.githubusercontent.com/openshift/odo/master/build/VERSION
[odo] I0111 15:01:06.422073   65862 util.go:748] Response will be cached in /tmp/odohttpcache for 1h0m0s
[odo] I0111 15:01:06.422225   65862 util.go:761] Cached response used.
[odo] I0111 15:01:06.424377   65862 kclient.go:202] Checking if "clusterserviceversions" resource supported
[odo] I0111 15:01:06.474338   65862 kclient.go:202] Checking if "clusterserviceversions" resource supported
[odo] I0111 15:01:06.487596   65862 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.510129   65862 kclient.go:202] Checking if "servicebindings" resource supported
[odo] I0111 15:01:06.522987   65862 kclient.go:202] Checking if "clusterserviceversions" resource supported
[odo] I0111 15:01:06.523009   65862 operators.go:35] Fetching list of operators installed in cluster
[odo] I0111 15:01:06.560336   65862 service.go:450] Getting instances of: etcdclusters.etcd.database.coreos.com
[odo] I0111 15:01:06.576493   65862 operators.go:35] Fetching list of operators installed in cluster
[odo] I0111 15:01:06.597461   65862 service.go:450] Getting instances of: servicebindings.operators.coreos.com
[odo] I0111 15:01:06.614794   65862 operators.go:35] Fetching list of operators installed in cluster
[odo]  ✓  Successfully unlinked component "kxejpk" from service "EtcdCluster/example"
[odo] 
[odo] To apply the changes, please use `odo push`
[odo] I0111 15:01:06.653896   65862 odo.go:72] Could not get the latest release information in time. Never mind, exiting gracefully :)
Running odo with args [odo push]
[odo] I0111 15:01:06.705929   65885 context.go:115] absolute devfile path: '/tmp/035875694/devfile.yaml'
[odo] I0111 15:01:06.705980   65885 context.go:69] absolute devfile path: '/tmp/035875694/devfile.yaml'
[odo] I0111 15:01:06.705976   65885 util.go:727] HTTPGetRequest: https://raw.githubusercontent.com/openshift/odo/master/build/VERSION
[odo] I0111 15:01:06.706097   65885 util.go:748] Response will be cached in /tmp/odohttpcache for 1h0m0s
[odo] I0111 15:01:06.706222   65885 content.go:33] converted devfile YAML to JSON
[odo] I0111 15:01:06.706251   65885 util.go:761] Cached response used.
[odo] I0111 15:01:06.706264   65885 apiVersion.go:50] devfile apiVersion: '2.0.0'
[odo] I0111 15:01:06.706273   65885 helper.go:41] devfile apiVersion '2.0.0' is supported
[odo] I0111 15:01:06.710105   65885 schema.go:46] validated devfile schema
[odo] I0111 15:01:06.711213   65885 validate.go:43] Successfully validated devfile sections
[odo] I0111 15:01:06.711238   65885 validate.go:43] Successfully validated devfile sections
[odo] I0111 15:01:06.711448   65885 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.779624   65885 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.779679   65885 context.go:115] absolute devfile path: '/tmp/035875694/devfile.yaml'
[odo] I0111 15:01:06.779683   65885 context.go:69] absolute devfile path: '/tmp/035875694/devfile.yaml'
[odo] I0111 15:01:06.779907   65885 content.go:33] converted devfile YAML to JSON
[odo] I0111 15:01:06.779954   65885 apiVersion.go:50] devfile apiVersion: '2.0.0'
[odo] I0111 15:01:06.779961   65885 helper.go:41] devfile apiVersion '2.0.0' is supported
[odo] I0111 15:01:06.785656   65885 schema.go:46] validated devfile schema
[odo] I0111 15:01:06.785740   65885 validate.go:43] Successfully validated devfile sections
[odo] I0111 15:01:06.785752   65885 validate.go:43] Successfully validated devfile sections
[odo] I0111 15:01:06.785816   65885 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.785832   65885 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.787409   65885 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.787432   65885 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.807120   65885 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.807158   65885 pods.go:38] Waiting for component=kxejpk pod
[odo] I0111 15:01:06.825389   65885 pods.go:66] Status of kxejpk-5fb854f65c-kh7dn pod is Running
[odo] I0111 15:01:06.825490   65885 pods.go:70] Pod Conditions: {"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2021-01-11T15:01:02Z"}
[odo] I0111 15:01:06.825522   65885 pods.go:70] Pod Conditions: {"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2021-01-11T15:01:03Z"}
[odo] I0111 15:01:06.825528   65885 pods.go:70] Pod Conditions: {"type":"ContainersReady","status":"True","lastProbeTime":null,"lastTransitionTime":"2021-01-11T15:01:03Z"}
[odo] I0111 15:01:06.825534   65885 pods.go:70] Pod Conditions: {"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2021-01-11T15:00:54Z"}
[odo] I0111 15:01:06.825621   65885 pods.go:75] Container Status: {"name":"runtime","state":{"running":{"startedAt":"2021-01-11T15:01:03Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"registry.access.redhat.com/ubi8/nodejs-12:1-36","imageID":"registry.access.redhat.com/ubi8/nodejs-12@sha256:c691d5de6ddefe45bec53a35f6bc54a0fe420960196ddb12656c6f9331a39c03","containerID":"cri-o://4ef7555c563377883f6680d1f71763f46ba59f84387c71129bc299ee5451ff36","started":true}
[odo] I0111 15:01:06.825636   65885 pods.go:79] Pod kxejpk-5fb854f65c-kh7dn is Running
[odo] 
[odo] Validation
[odo]  •  Validating the devfile  ...
[odo] I0111 15:01:06.825757   65885 command.go:206] Build command: devbuild
[odo] I0111 15:01:06.825772   65885 command.go:213] Run command: devrun
[odo] 
 ✓  Validating the devfile [47524ns]
[odo] 
[odo] Creating Kubernetes resources for component kxejpk
[odo] I0111 15:01:06.825837   65885 command.go:98] the command group of kind "debug" is not found in the devfile
[odo] I0111 15:01:06.825856   65885 utils.go:228] Updating container runtime entrypoint with supervisord
[odo] I0111 15:01:06.825865   65885 utils.go:123] Updating container runtime with supervisord volume mounts
[odo] I0111 15:01:06.825876   65885 utils.go:133] Updating container runtime env with run command
[odo] I0111 15:01:06.825884   65885 utils.go:150] Updating container runtime env with run command's workdir
[odo] I0111 15:01:06.825895   65885 preference.go:219] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] I0111 15:01:06.825921   65885 adapter.go:339] Generating PVC name for odo-projects
[odo] I0111 15:01:06.825944   65885 utils.go:113] Checking PVC for volume odo-projects and label component=kxejpk,storage-name=odo-projects
[odo] I0111 15:01:06.833887   65885 events.go:36] Quitting collect events
[odo] I0111 15:01:06.850975   65885 utils.go:120] Found an existing PVC for volume odo-projects and label component=kxejpk,storage-name=odo-projects
[odo] I0111 15:01:06.850998   65885 adapter.go:351] Found an existing PVC for odo-projects, PVC odo-projects-kxejpk-hjoy will be re-used
[odo] I0111 15:01:06.866985   65885 adapter.go:410] Creating deployment kxejpk
[odo] I0111 15:01:06.867167   65885 adapter.go:411] The component name is kxejpk
[odo] I0111 15:01:06.867183   65885 adapter.go:415] The component already exists, attempting to update it
[odo] I0111 15:01:06.888553   65885 adapter.go:420] Successfully updated component kxejpk
[odo] I0111 15:01:06.923798   65885 adapter.go:441] Successfully update Service for component kxejpk
[odo] I0111 15:01:06.923831   65885 utils.go:113] Checking PVC for volume odo-projects and label component=kxejpk,storage-name=odo-projects
[odo]  •  Waiting for component to start  ...
[odo] I0111 15:01:06.945840   65885 utils.go:120] Found an existing PVC for volume odo-projects and label component=kxejpk,storage-name=odo-projects
[odo] I0111 15:01:06.945860   65885 deployments.go:101] Waiting for kxejpk deployment rollout
[odo] I0111 15:01:06.960267   65885 deployments.go:134] Deployment Condition: {"type":"Available","status":"True","lastUpdateTime":"2021-01-11T15:01:03Z","lastTransitionTime":"2021-01-11T15:01:03Z","reason":"MinimumReplicasAvailable","message":"Deployment has minimum availability."}
[odo] I0111 15:01:06.960289   65885 deployments.go:134] Deployment Condition: {"type":"Progressing","status":"True","lastUpdateTime":"2021-01-11T15:01:03Z","lastTransitionTime":"2021-01-11T15:00:21Z","reason":"NewReplicaSetAvailable","message":"ReplicaSet \"kxejpk-5fb854f65c\" has successfully progressed."}
[odo] I0111 15:01:06.960298   65885 deployments.go:141] Waiting for deployment "kxejpk" rollout to finish: 0 out of 1 new replicas have been updated...
[odo] I0111 15:01:06.960304   65885 deployments.go:152] Waiting for deployment spec update to be observed...
[odo] I0111 15:01:06.969102   65885 deployments.go:134] Deployment Condition: {"type":"Progressing","status":"True","lastUpdateTime":"2021-01-11T15:01:03Z","lastTransitionTime":"2021-01-11T15:00:21Z","reason":"NewReplicaSetAvailable","message":"ReplicaSet \"kxejpk-5fb854f65c\" has successfully progressed."}
[odo] I0111 15:01:06.969121   65885 deployments.go:134] Deployment Condition: {"type":"Available","status":"False","lastUpdateTime":"2021-01-11T15:01:06Z","lastTransitionTime":"2021-01-11T15:01:06Z","reason":"MinimumReplicasUnavailable","message":"Deployment does not have minimum availability."}
[odo] I0111 15:01:06.969128   65885 deployments.go:141] Waiting for deployment "kxejpk" rollout to finish: 0 out of 1 new replicas have been updated...
[odo] I0111 15:01:06.969133   65885 deployments.go:152] Waiting for deployment spec update to be observed...
[odo] I0111 15:01:07.954528   65885 deployments.go:134] Deployment Condition: {"type":"Progressing","status":"True","lastUpdateTime":"2021-01-11T15:01:03Z","lastTransitionTime":"2021-01-11T15:00:21Z","reason":"NewReplicaSetAvailable","message":"ReplicaSet \"kxejpk-5fb854f65c\" has successfully progressed."}
[odo] I0111 15:01:07.954553   65885 deployments.go:134] Deployment Condition: {"type":"Available","status":"False","lastUpdateTime":"2021-01-11T15:01:06Z","lastTransitionTime":"2021-01-11T15:01:06Z","reason":"MinimumReplicasUnavailable","message":"Deployment does not have minimum availability."}
[odo] I0111 15:01:07.954558   65885 deployments.go:152] Waiting for deployment spec update to be observed...
[odo] I0111 15:01:07.976835   65885 deployments.go:134] Deployment Condition: {"type":"Progressing","status":"True","lastUpdateTime":"2021-01-11T15:01:03Z","lastTransitionTime":"2021-01-11T15:00:21Z","reason":"NewReplicaSetAvailable","message":"ReplicaSet \"kxejpk-5fb854f65c\" has successfully progressed."}
[odo] I0111 15:01:07.976864   65885 deployments.go:134] Deployment Condition: {"type":"Available","status":"False","lastUpdateTime":"2021-01-11T15:01:06Z","lastTransitionTime":"2021-01-11T15:01:06Z","reason":"MinimumReplicasUnavailable","message":"Deployment does not have minimum availability."}
[odo] I0111 15:01:07.976871   65885 deployments.go:152] Waiting for deployment spec update to be observed...
[odo] I0111 15:01:08.000142   65885 deployments.go:134] Deployment Condition: {"type":"Progressing","status":"True","lastUpdateTime":"2021-01-11T15:01:03Z","lastTransitionTime":"2021-01-11T15:00:21Z","reason":"NewReplicaSetAvailable","message":"ReplicaSet \"kxejpk-5fb854f65c\" has successfully progressed."}
[odo] I0111 15:01:08.000163   65885 deployments.go:134] Deployment Condition: {"type":"Available","status":"False","lastUpdateTime":"2021-01-11T15:01:06Z","lastTransitionTime":"2021-01-11T15:01:06Z","reason":"MinimumReplicasUnavailable","message":"Deployment does not have minimum availability."}
[odo] I0111 15:01:08.000171   65885 deployments.go:141] Waiting for deployment "kxejpk" rollout to finish: 0 out of 1 new replicas have been updated...
[odo] I0111 15:01:08.000178   65885 deployments.go:152] Waiting for deployment spec update to be observed...
[odo] I0111 15:01:08.030294   65885 deployments.go:134] Deployment Condition: {"type":"Progressing","status":"True","lastUpdateTime":"2021-01-11T15:01:03Z","lastTransitionTime":"2021-01-11T15:00:21Z","reason":"NewReplicaSetAvailable","message":"ReplicaSet \"kxejpk-5fb854f65c\" has successfully progressed."}
[odo] I0111 15:01:08.030333   65885 deployments.go:134] Deployment Condition: {"type":"Available","status":"False","lastUpdateTime":"2021-01-11T15:01:06Z","lastTransitionTime":"2021-01-11T15:01:06Z","reason":"MinimumReplicasUnavailable","message":"Deployment does not have minimum availability."}
[odo] I0111 15:01:08.030343   65885 deployments.go:145] Waiting for deployment "kxejpk" rollout to finish: 0 of 1 updated replicas are available...
[odo] I0111 15:01:08.030351   65885 deployments.go:152] Waiting for deployment spec update to be observed...
[odo] 
I0111 15:01:53.834381   65885 events.go:52] Warning Event: Count: 5, Reason: Failed, Message: Error: secret "kxejpk-etcdcluster-example" not found
[odo] 
I0111 15:02:04.631459   65885 events.go:52] Warning Event: Count: 6, Reason: Failed, Message: Error: secret "kxejpk-etcdcluster-example" not found
[odo] 
I0111 15:02:15.803137   65885 events.go:52] Warning Event: Count: 7, Reason: Failed, Message: Error: secret "kxejpk-etcdcluster-example" not found
[odo]  ✗  Waiting for component to start [5m] [WARNING x7: Failed]
[odo]  ✗  Failed to start component with name kxejpk. Error: Failed to create the component: error while waiting for deployment rollout: timeout while waiting for kxejpk deployment roll out\nFor more information to help determine the cause of the error, re-run with '-v'.
[odo] See below for a list of failed events that occured more than 5 times during deployment:
[odo] 
[odo]  NAME                                      COUNT  REASON  MESSAGE                        
[odo] 
[odo]  kxejpk-5fb854f65c-q8pkr.16593598c6289b7c  7      Failed  Error: secret                  
[odo]                                                           "kxejpk-etcdcluster-example"   
[odo]                                                           not found                      
[odo] 
[odo] 
Deleting project: pcahtyrxtz
Running odo with args [odo project delete pcahtyrxtz -f]
[odo] I0111 15:06:07.016081   65909 util.go:727] HTTPGetRequest: https://raw.githubusercontent.com/openshift/odo/master/build/VERSION
[odo] I0111 15:06:07.016268   65909 util.go:748] Response will be cached in /tmp/odohttpcache for 1h0m0s
[odo] I0111 15:06:07.113495   65909 kclient.go:202] Checking if "projects" resource supported
[odo] I0111 15:06:07.152073   65909 kclient.go:202] Checking if "projects" resource supported
[odo] I0111 15:06:07.244058   65909 util.go:761] Cached response used.
[odo]  ⚠  Warning! Projects are deleted from the cluster asynchronously. Odo does its best to delete the project. Due to multi-tenant clusters, the project may still exist on a different node.
[odo] I0111 15:06:07.249582   65909 odo.go:72] Could not get the latest release information in time. Never mind, exiting gracefully :)
[odo]  ✓  Deleted project : pcahtyrxtz
Setting current dir to: /go/src/github.com/openshift/odo/tests/integration/operatorhub
Deleting dir: /tmp/035875694

@dharmit dharmit mentioned this pull request Jan 12, 2021
Closed
@dharmit
Copy link
Member Author

dharmit commented Jan 20, 2021

/test v4.6-integration-e2e

@girishramnani
Copy link
Contributor

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Jan 20, 2021
@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

3 similar comments
@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 9f6cf97 into redhat-developer:master Jan 20, 2021
@dharmit dharmit deleted the refactor-catalog-list-services-2 branch January 21, 2021 06:54
@dharmit dharmit mentioned this pull request Jan 21, 2021
Closed
@rm3l rm3l added the area/refactoring Issues or PRs related to code refactoring label Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@girishramnani girishramnani Awaiting requested review from girishramnani

@mohammedzee1000 mohammedzee1000 Awaiting requested review from mohammedzee1000

@mik-dass mik-dass Awaiting requested review from mik-dass

Assignees

@girishramnani girishramnani

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. area/refactoring Issues or PRs related to code refactoring lgtm Indicates that a PR is ready to be merged. Required by Prow.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@dharmit @openshift-ci-robot @girishramnani @openshift-bot @rm3l @openshift-merge-robot