Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

installing knative results in the server could not find the requested resource (post gatewaies.networking.istio.io) #1500

Closed
cppforlife opened this issue Jul 5, 2018 · 11 comments
Labels
area/test-and-release It flags unit/e2e/conformance/perf test issues for product features kind/bug Categorizes issue or PR as related to a bug.

Comments

@cppforlife
Copy link
Contributor

/area test-and-release
/kind bug

Expected Behavior

following installation instructions from docs repo should result in working isitio+knative setup.

Actual Behavior

kubectl apply of knative ymls fails with following error:

Error from server (NotFound): error when creating "https://storage.googleapis.com/knative-releases/latest/release.yaml": the server could not find the requested resource (post gatewaies.networking.istio.io)

Steps to Reproduce the Problem

  1. setup default GKE with 1.10.5-gke.0
  2. install istio successfully
  3. fail to install knative
@google-prow-robot google-prow-robot added area/test-and-release It flags unit/e2e/conformance/perf test issues for product features kind/bug Categorizes issue or PR as related to a bug. labels Jul 5, 2018
@cppforlife
Copy link
Contributor Author

cc @maximilien

@sukhil-suresh
Copy link
Contributor

sukhil-suresh commented Jul 6, 2018

@cppforlife EXCITED to see you on Knative!!

Tried just now with a fresh GKE cluster, by following Knative the easy way docs and was unable to reproduce the problem. I was also able to deploy the Hello World - Go sample on Knative.

Sorry, I know that ^ is not very helpful. My best (not great) guess is to wonder if you confirmed that the Istio components were Running or Completed, before proceeding with the Knative deploy?

Ref: As part of Step 2: Install Istio

Monitor the Istio components, until all of the components report Running or Completed

Also, confused as to why your error message refers to gatewaies.networking.istio.io. The closest match I see in the output when installing istio are the references to:

...
gateway.networking.istio.io/knative-shared-gateway created
...
customresourcedefinition.apiextensions.k8s.io/gateways.networking.istio.io created
...

Not sure if there were any changes to the repos since you last tried. But, could consider trying again since it worked on my machine? Horrible! :)

@sukhil-suresh
Copy link
Contributor

just an FYI... my cluster zone was northamerica-northeast1-a

@tcnghia
Copy link
Contributor

tcnghia commented Jul 6, 2018

@cppforlife do you still have a copy of your YAML file? I'd love to take a look to see why any reference to 'gatewaies' is still there.

@cppforlife
Copy link
Contributor Author

ill give another try later today, will report back.

@maximilien
Copy link

Attempted with a new cluster on GKE running:

 $ kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.5", GitCommit:"17d7182a7ccbb167074be7a87f0a68bd00d58d97", GitTreeState:"clean", BuildDate:"2017-08-31T19:32:09Z", GoVersion:"go1.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.5-gke.0", GitCommit:"f4c74e18e57148052c59cc0467bb7e99dcc46399", GitTreeState:"clean", BuildDate:"2018-06-21T14:11:26Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}

Same problem. Output:

 $ kubectl get node
NAME                                       STATUS    AGE       VERSION
gke-dk-jul-12-default-pool-d49794d7-rjj6   Ready     3h        v1.10.5-gke.0
gke-dk-jul-12-default-pool-d49794d7-t1rw   Ready     3h        v1.10.5-gke.0
gke-dk-jul-12-default-pool-d49794d7-zk24   Ready     3h        v1.10.5-gke.0
 $ ./perms
clusterrolebinding "cluster-admin-binding" created
 $ wget -O- https://storage.googleapis.com/knative-releases/latest/istio.yaml > istio.yml
--2018-07-12 14:06:24--  https://storage.googleapis.com/knative-releases/latest/istio.yaml
Resolving storage.googleapis.com... 216.58.194.176, 2607:f8b0:4005:804::2010
Connecting to storage.googleapis.com|216.58.194.176|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 84026 (82K) [application/octet-stream]
Saving to: 'STDOUT'

100%[==================================================================================================================>] 84,026      --.-K/s   in 0.008s

2018-07-12 14:06:24 (10.6 MB/s) - written to stdout [84026/84026]

 $ kubectl apply -f istio.yml
namespace "istio-system" created
configmap "istio-statsd-prom-bridge" created
configmap "istio-mixer-custom-resources" created
configmap "istio" created
configmap "istio-sidecar-injector" created
serviceaccount "istio-egressgateway-service-account" created
serviceaccount "istio-ingress-service-account" created
serviceaccount "istio-ingressgateway-service-account" created
serviceaccount "istio-mixer-post-install-account" created
clusterrole "istio-mixer-post-install-istio-system" created
clusterrolebinding "istio-mixer-post-install-role-binding-istio-system" created
job "istio-mixer-post-install" created
serviceaccount "istio-mixer-service-account" created
serviceaccount "istio-pilot-service-account" created
serviceaccount "istio-citadel-service-account" created
serviceaccount "istio-cleanup-old-ca-service-account" created
serviceaccount "istio-sidecar-injector-service-account" created
customresourcedefinition "rules.config.istio.io" created
customresourcedefinition "attributemanifests.config.istio.io" created
customresourcedefinition "circonuses.config.istio.io" created
customresourcedefinition "deniers.config.istio.io" created
customresourcedefinition "fluentds.config.istio.io" created
customresourcedefinition "kubernetesenvs.config.istio.io" created
customresourcedefinition "listcheckers.config.istio.io" created
customresourcedefinition "memquotas.config.istio.io" created
customresourcedefinition "noops.config.istio.io" created
customresourcedefinition "opas.config.istio.io" created
customresourcedefinition "prometheuses.config.istio.io" created
customresourcedefinition "rbacs.config.istio.io" created
customresourcedefinition "servicecontrols.config.istio.io" created
customresourcedefinition "solarwindses.config.istio.io" created
customresourcedefinition "stackdrivers.config.istio.io" created
customresourcedefinition "statsds.config.istio.io" created
customresourcedefinition "stdios.config.istio.io" created
customresourcedefinition "apikeys.config.istio.io" created
customresourcedefinition "authorizations.config.istio.io" created
customresourcedefinition "checknothings.config.istio.io" created
customresourcedefinition "kuberneteses.config.istio.io" created
customresourcedefinition "listentries.config.istio.io" created
customresourcedefinition "logentries.config.istio.io" created
customresourcedefinition "metrics.config.istio.io" created
customresourcedefinition "quotas.config.istio.io" created
customresourcedefinition "reportnothings.config.istio.io" created
customresourcedefinition "servicecontrolreports.config.istio.io" created
customresourcedefinition "tracespans.config.istio.io" created
customresourcedefinition "serviceroles.config.istio.io" created
customresourcedefinition "servicerolebindings.config.istio.io" created
customresourcedefinition "destinationpolicies.config.istio.io" created
customresourcedefinition "egressrules.config.istio.io" created
customresourcedefinition "routerules.config.istio.io" created
customresourcedefinition "virtualservices.networking.istio.io" created
customresourcedefinition "destinationrules.networking.istio.io" created
customresourcedefinition "serviceentries.networking.istio.io" created
customresourcedefinition "gateways.networking.istio.io" created
customresourcedefinition "policies.authentication.istio.io" created
customresourcedefinition "httpapispecbindings.config.istio.io" created
customresourcedefinition "httpapispecs.config.istio.io" created
customresourcedefinition "quotaspecbindings.config.istio.io" created
customresourcedefinition "quotaspecs.config.istio.io" created
clusterrole "istio-ingress-istio-system" created
clusterrole "istio-mixer-istio-system" created
clusterrole "istio-pilot-istio-system" created
clusterrole "istio-citadel-istio-system" created
role "istio-cleanup-old-ca-istio-system" created
clusterrole "istio-sidecar-injector-istio-system" created
clusterrolebinding "istio-ingress-istio-system" created
clusterrolebinding "istio-mixer-admin-role-binding-istio-system" created
clusterrolebinding "istio-pilot-istio-system" created
clusterrolebinding "istio-citadel-istio-system" created
rolebinding "istio-cleanup-old-ca-istio-system" created
clusterrolebinding "istio-sidecar-injector-admin-role-binding-istio-system" created
service "istio-egressgateway" created
service "istio-ingress" created
service "istio-ingressgateway" created
service "istio-policy" created
service "istio-telemetry" created
service "istio-statsd-prom-bridge" created
deployment "istio-statsd-prom-bridge" created
service "istio-pilot" created
service "istio-citadel" created
service "istio-sidecar-injector" created
deployment "istio-egressgateway" created
deployment "istio-ingress" created
deployment "istio-ingressgateway" created
deployment "istio-policy" created
deployment "istio-telemetry" created
deployment "istio-pilot" created
deployment "istio-citadel" created
deployment "istio-sidecar-injector" created
job "istio-cleanup-old-ca" created
horizontalpodautoscaler "istio-egressgateway" created
horizontalpodautoscaler "istio-ingress" created
horizontalpodautoscaler "istio-ingressgateway" created
mutatingwebhookconfiguration "istio-sidecar-injector" created
 $ kubectl label namespace default istio-injection=enabled
namespace "default" labeled
 $ kubectl get pods -n istio-system --watch
...
snip
...
 $ wget -O- https://storage.googleapis.com/knative-releases/latest/release.yaml > knative.yml
--2018-07-12 14:08:57--  https://storage.googleapis.com/knative-releases/latest/release.yaml
Resolving storage.googleapis.com... 216.58.194.176, 2607:f8b0:4005:804::2010
Connecting to storage.googleapis.com|216.58.194.176|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 493176 (482K) [application/octet-stream]
Saving to: 'STDOUT'

100%[==================================================================================================================>] 493,176     --.-K/s   in 0.03s

2018-07-12 14:08:57 (17.2 MB/s) - written to stdout [493176/493176]

 $ kubectl apply -f knative.yml
namespace "knative-build" created
clusterrole "knative-build-admin" created
serviceaccount "build-controller" created
clusterrolebinding "build-controller-admin" created
customresourcedefinition "builds.build.knative.dev" created
customresourcedefinition "buildtemplates.build.knative.dev" created
service "build-controller" created
service "build-webhook" created
configmap "config-logging" created
deployment "build-controller" created
deployment "build-webhook" created
namespace "knative-serving" created
clusterrole "knative-serving-admin" created
clusterrole "knative-serving-write" created
serviceaccount "controller" created
serviceaccount "autoscaler" created
clusterrolebinding "knative-serving-controller-admin" created
clusterrolebinding "knative-serving-autoscaler-write" created
service "knative-ingressgateway" created
deployment "knative-ingressgateway" created
horizontalpodautoscaler "knative-ingressgateway" created
customresourcedefinition "configurations.serving.knative.dev" created
customresourcedefinition "revisions.serving.knative.dev" created
customresourcedefinition "routes.serving.knative.dev" created
customresourcedefinition "services.serving.knative.dev" created
service "activator-service" created
service "controller" created
service "webhook" created
deployment "activator" created
configmap "config-autoscaler" created
configmap "config-controller" created
configmap "config-domain" created
configmap "config-logging" created
configmap "config-network" created
configmap "config-observability" created
deployment "controller" created
deployment "webhook" created
namespace "monitoring" created
configmap "fluentd-ds-config" created
configmap "scaling-config" created
configmap "grafana-dashboard-definition-istio" created
configmap "grafana-dashboard-definition-mixer" created
configmap "grafana-dashboard-definition-pilot" created
serviceaccount "fluentd-ds" created
clusterrole "fluentd-ds" created
clusterrolebinding "fluentd-ds" created
daemonset "fluentd-ds" created
serviceaccount "kube-state-metrics" created
role "kube-state-metrics-resizer" created
rolebinding "kube-state-metrics" created
clusterrole "kube-state-metrics" created
clusterrolebinding "kube-state-metrics" created
deployment "kube-state-metrics" created
service "kube-state-metrics" created
configmap "grafana-dashboard-definition-kubernetes-deployment" created
configmap "grafana-dashboard-definition-kubernetes-capacity-planning" created
configmap "grafana-dashboard-definition-kubernetes-cluster-health" created
configmap "grafana-dashboard-definition-kubernetes-cluster-status" created
configmap "grafana-dashboard-definition-kubernetes-control-plane-status" created
configmap "grafana-dashboard-definition-kubernetes-resource-requests" created
configmap "grafana-dashboard-definition-kubernetes-nodes" created
configmap "grafana-dashboard-definition-kubernetes-pods" created
configmap "grafana-dashboard-definition-kubernetes-statefulset" created
serviceaccount "node-exporter" created
clusterrole "node-exporter" created
clusterrolebinding "node-exporter" created
daemonset "node-exporter" created
service "node-exporter" created
service "elasticsearch-logging" created
serviceaccount "elasticsearch-logging" created
clusterrole "elasticsearch-logging" created
clusterrolebinding "elasticsearch-logging" created
statefulset "elasticsearch-logging" created
deployment "kibana-logging" created
service "kibana-logging" created
service "fluentd-ds" created
configmap "grafana-dashboard-definition-knative-efficiency" created
configmap "grafana-dashboard-definition-knative" created
configmap "grafana-datasources" created
configmap "grafana-dashboards" created
service "grafana" created
deployment "grafana" created
logentry "requestlog" created
fluentd "requestloghandler" created
rule "requestlogtofluentd" created
metric "revisionrequestcount" created
metric "revisionrequestduration" created
metric "revisionrequestsize" created
metric "revisionresponsesize" created
prometheus "revisionpromhandler" created
rule "revisionpromhttp" created
service "zipkin" created
deployment "zipkin" created
service "kube-controller-manager" created
service "prometheus-system-discovery" created
configmap "prometheus-scrape-config" created
serviceaccount "prometheus-system" created
role "prometheus-system" created
role "prometheus-system" created
role "prometheus-system" created
role "prometheus-system" created
clusterrole "prometheus-system" created
rolebinding "prometheus-system" created
rolebinding "prometheus-system" created
rolebinding "prometheus-system" created
rolebinding "prometheus-system" created
clusterrolebinding "prometheus-system" created
service "prometheus-system-np" created
statefulset "prometheus-system" created
logentry "requestlog" configured
fluentd "requestloghandler" configured
rule "requestlogtofluentd" configured
metric "revisionrequestcount" configured
metric "revisionrequestduration" configured
metric "revisionrequestsize" configured
metric "revisionresponsesize" configured
prometheus "revisionpromhandler" configured
rule "revisionpromhttp" configured
Error from server (NotFound): error when creating "knative.yml": the server could not find the requested resource (post gatewaies.networking.istio.io)

secret gist: https://gist.github.com/maximilien/ba103bcd67f4d8f733d654cf38928454 with contents of istio.yml and knative.yml

@tcnghia
Copy link
Contributor

tcnghia commented Jul 13, 2018

@maximilien I suspect this is due to older version of kubectl? I hope you can try this again with a newer version of kubectl?

My theory is that older kubectl may have interpreted our knative-shared-gateway object to have the type gatewaies.networking.istio.io because it uses convention instead of reading from the CRD defined in istio.yml.

@tcnghia
Copy link
Contributor

tcnghia commented Jul 13, 2018

I found this bug kubernetes/apiextensions-apiserver#12 . This PR kubernetes/kubernetes#50012 fixes the bug by reading from CRD instead of using convention. Based on comments in kubernetes/apiextensions-apiserver#12 I think you'll need at least kubectl 1.8 or 1.9.

Please let me know how it goes after upgrading?

@cppforlife
Copy link
Contributor Author

it does work with new CLI. cc @maximilien

thank you @tcnghia for taking a look.

google-prow-robot pushed a commit to knative/docs that referenced this issue Jul 18, 2018
* Update docs to require `kubectl` version later than v1.10.

See knative/serving#1500 for cases where
older `kubectl` fails to install Knative.

* Update based on PR feedbacks.
@Turbots
Copy link

Turbots commented Aug 17, 2018

I had the same problem so I also upgraded my CLI but it still doesn't work:

Minikube v0.28.2 configured with 8GB RAM and 4 CPUs (OSX)
Kubectl v1.11.2
Riff CLI v0.1.1

following the tutorial at https://projectriff.io/docs/getting-started-with-knative-riff-on-minikube/

edit: I apparently was using Google Cloud SDK version of kubectl and not the brew version... I was still using 1.7.6 which still has the bug. I updated Google Cloud SDK components using gcloud components update and am now using kubectl v1.9.7 and it works!

@davidmehi
Copy link

I also had this problem: the server could not find the requested resource (post gatewaies.networking.istio.io)

Even though I installed everything correctly, my path was still pointing to an old version of gcloud sdk, so my K8S client and server versions did not match. If "kubectl version" shows that the client and server versions are different, then upgrade one of them to match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/test-and-release It flags unit/e2e/conformance/perf test issues for product features kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

7 participants