Page needs improvement - https://knative.dev/docs/eventing/samples/kafka/

[mjschmidt]

Expected Behavior

I installed strimzi kafka to my cluster just to ensure that I would have a compatible/expected kafka since that is what the docs asked for. However I am unable to proceed past creating the channel for kafka in knative.

Actual Behavior

No address is passed into the knative v0.10.0 yaml file. Also, i did manually update this as the knative 10 docs still point to an older v0.9.0 yaml file.

Steps to Reproduce the Problem

1. Follow the tutorial found here https://knative.dev/docs/eventing/samples/kafka/channel/

#This code seems to have nothing to sed anymore interms of a kafka broker location? 
#https://github.com/knative/eventing-contrib/releases/download/v0.10.0/kafka-channel.yaml
NAME                                  READY   REASON      URL   AGE
broker.eventing.knative.dev/default   False   NoAddress         127m

NAME                                                     READY     REASON   URL   AGE
kafkachannel.messaging.knative.dev/default-kne-trigger   Unknown                  127m
kafkachannel.messaging.knative.dev/my-kafka-channel      Unknown                  145m
kafkachannel.messaging.knative.dev/testchannel-one       Unknown                  145m

NAME                                            READY   REASON          URL   AGE
channel.messaging.knative.dev/testchannel-one   False   EmptyHostname         145m

2. Not sure what is next (: happy to make contributions to the docs, just need a little guidance. Very cool project. Let me know if you need more details. For simplicity sake, kafka has been deployed to default namespace, along with knative. The knative components are in their usual namespaces.
3. Seems like just some doc updating is in order.

Additional Info

[knative-prow-robot]

@mjschmidt: GitHub didn't allow me to assign the following users: user.

Note that only knative members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

Expected Behavior

I installed strimzi kafka to my cluster just to ensure that I would have a compatible/expected kafka since that is what the docs asked for. However I am unable to proceed past creating the channel for kafka in knative.

Actual Behavior

No address is passed into the knative v0.10.0 yaml file. Also, i did manually update this as the knative 10 docs still point to an older v0.9.0 yaml file.

Steps to Reproduce the Problem

1. Follow the tutorial found here https://knative.dev/docs/eventing/samples/kafka/channel/

#This code seems to have nothing to sed anymore interms of a kafka broker location? 
#https://github.com/knative/eventing-contrib/releases/download/v0.10.0/kafka-channel.yaml
NAME                                  READY   REASON      URL   AGE
broker.eventing.knative.dev/default   False   NoAddress         127m

NAME                                                     READY     REASON   URL   AGE
kafkachannel.messaging.knative.dev/default-kne-trigger   Unknown                  127m
kafkachannel.messaging.knative.dev/my-kafka-channel      Unknown                  145m
kafkachannel.messaging.knative.dev/testchannel-one       Unknown                  145m

NAME                                            READY   REASON          URL   AGE
channel.messaging.knative.dev/testchannel-one   False   EmptyHostname         145m

2. Not sure what is next (: happy to make contributions to the docs, just need a little guidance. Very cool project. Let me know if you need more details. For simplicity sake, kafka has been deployed to default namespace, along with knative. The knative components are in their usual namespaces.
3. Seems like just some doc updating is in order.

Additional Info

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mjschmidt]

I am also unsure which branch I would do a pull from. would I branch master or go hot fix v0.10.0

[mjschmidt]

This command returns empty, which means that step one in these docs does functionally nothing.

curl -o kafka-channel.yaml -L https://github.com/knative/eventing-contrib/releases/download/v0.9.0/kafka-channel.yaml; cat kafka-channel.yaml | grep my-cluster

[abrennan89]

@matzew is there a way that the curl command under https://knative.dev/docs/eventing/samples/kafka/channel/ Creating a KafkaChannel channel CRD can be updated to point to a latest version at all times rather than "v0.10.0" so that we won't have to keep checking the accuracy of docs like this?

i.e. could we use something like curl -L "https://github.com/knative/eventing-contrib/releases/download/latest/kafka-channel.yaml"instead or would that not work for some reason?

@mjschmidt would this fix the issue for you?

[mjschmidt]

So the Kafka Channel actually isn't working as expected and I am not sure what to look at next. There are more details below, as well as what I plan to pull request back to the docs when I get the Kafka Channel example working again down below.

After some tweaks (simple things like specifying the correct default channel configuration since you can't have 0 partitions for kafka, but that is the default value created if you specify nothing like in the example crd)

I am getting these logs from the apiserversource-testevents container, which is deployed in the final section:

{"level":"info","ts":"2020-01-19T23:09:03.697Z","caller":"logging/config.go:108","msg":"Successfully created the logger."}
{"level":"info","ts":"2020-01-19T23:09:03.697Z","caller":"logging/config.go:109","msg":"Logging level set to info"}
{"level":"info","ts":"2020-01-19T23:09:03.697Z","logger":"apiserversource","caller":"metrics/exporter.go:99","msg":"Flushing the existing exporter before setting up the new exporter.","commit":"3bf8045"}
{"level":"info","ts":"2020-01-19T23:09:03.697Z","logger":"apiserversource","caller":"metrics/prometheus_exporter.go:37","msg":"Created Opencensus Prometheus exporter with config: &{knative.dev/sources apiserversource prometheus 5000000000 <nil> 9090 false   {   false}}. Start the server for Prometheus exporter.","commit":"3bf8045"}
{"level":"info","ts":"2020-01-19T23:09:03.697Z","logger":"apiserversource","caller":"metrics/exporter.go:108","msg":"Successfully updated the metrics exporter; old config: <nil>; new config &{knative.dev/sources apiserversource prometheus 5000000000 <nil> 9090 false   {   false}}","commit":"3bf8045"}
W0119 23:09:03.698083       1 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
{"level":"info","ts":"2020-01-19T23:09:03.698Z","logger":"apiserversource","caller":"adapter/main.go:131","msg":"Starting Receive Adapter{adapter 22 0  0xc00024e460}","commit":"3bf8045"}
W0119 23:29:39.733940       1 reflector.go:302] knative.dev/eventing/pkg/adapter/apiserver/adapter.go:225: watch of *unstructured.Unstructured ended with: The resourceVersion for the provided watch is too old.
W0119 23:39:32.767384       1 reflector.go:302] knative.dev/eventing/pkg/adapter/apiserver/adapter.go:225: watch of *unstructured.Unstructured ended with: The resourceVersion for the provided watch is too old.
W0120 00:02:27.833317       1 reflector.go:302] knative.dev/eventing/pkg/adapter/apiserver/adapter.go:225: watch of *unstructured.Unstructured ended with: The resourceVersion for the provided watch is too old.

In addition, I am getting empty logs from the display container when I go to execute the last command in the sample:

kubectl logs --selector='serving.knative.dev/service=broker-kafka-display' -c user-container

What should I be expecting at the end of the Kafka Channel Example? What are somethings I can look at to resolve the example? I am not sure where to go next with fixing things, but here are things I plan to fix in the docs so far for Kafka Channel Example:

• Updating the example Kafka channel object to work with the minikube instillation of strimzi (the example installs only a single kafka broker, so knative kafka channel must have replication of 1 for it to work)
• Updating the default channel CRD to work (you get an error of zero partitions being invalid for Kafka Channel if you don't set it in yaml)
• Tear down instructions and caviet that if you are running knative on a minikube, you cannot shut down the machine and expect knative to come back as expected (forgot the error message you get, but I will update with that once I get kafka working and can shut things down)
• Updating creating the service and trigger section to include the required service account that is missing from instructions
• Updating what versions of Kubernetes are compatible and which are not for this example if applicable (I was having problems early on when I was using 1.16, but since it was early in my understanding, 1.16.x may be fine with knative I need to do more testing)

I can now get all the way to the end of the example where I am getting the following error and I don't know what else to look at for troubleshooting. Maybe you all can provide some guidance.

[mjschmidt]

Kubernetes version 1.15.7
knative 0v.11.0 and I switched out the curl command to use v0.11.0 as well manually (one of the tings I plan to pull request)

Output from kubectl get all -n default at this point:

NAME                                                                  READY   STATUS    RESTARTS   AGE
pod/apiserversource-testevents-6ad167ff-0230-4082-8eef-dd6eab42zbmq   1/1     Running   0          5m1s
pod/default-broker-filter-5f899d7684-qd79f                            2/2     Running   5          5m
pod/default-broker-ingress-7766b55fd9-d4q7h                           2/2     Running   5          5m

NAME                                         TYPE           CLUSTER-IP       EXTERNAL-IP                                              PORT(S)                             AGE
service/broker-kafka-display                 ExternalName   <none>           cluster-local-gateway.istio-system.svc.cluster.local     <none>                              41h
service/broker-kafka-display-7fxsh           ClusterIP      10.102.146.28    <none>                                                   80/TCP                              41h
service/broker-kafka-display-7fxsh-private   ClusterIP      10.103.215.114   <none>                                                   80/TCP,9090/TCP,9091/TCP,8022/TCP   41h
service/default-broker                       ClusterIP      10.96.214.114    <none>                                                   80/TCP,9090/TCP                     2d12h
service/default-broker-filter                ClusterIP      10.108.39.116    <none>                                                   80/TCP,9090/TCP                     2d12h
service/default-kne-ingress-kn-channel       ExternalName   <none>           kafka-ch-dispatcher.knative-eventing.svc.cluster.local   <none>                              2d12h
service/default-kne-trigger-kn-channel       ExternalName   <none>           kafka-ch-dispatcher.knative-eventing.svc.cluster.local   <none>                              2d12h
service/kubernetes                           ClusterIP      10.96.0.1        <none>                                                   443/TCP                             3d21h
service/my-kafka-channel-kn-channel          ExternalName   <none>           kafka-ch-dispatcher.knative-eventing.svc.cluster.local   <none>                              2d19h
service/testchannel-one-kn-channel           ExternalName   <none>           kafka-ch-dispatcher.knative-eventing.svc.cluster.local   <none>                              2d12h

NAME                                                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/apiserversource-testevents-6ad167ff-0230-4082-8eef-dd6eab455581   1/1     1            1           41h
deployment.apps/broker-kafka-display-7fxsh-deployment                             0/0     0            0           41h
deployment.apps/default-broker-filter                                             1/1     1            1           2d12h
deployment.apps/default-broker-ingress                                            1/1     1            1           2d12h

NAME                                                                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/apiserversource-testevents-6ad167ff-0230-4082-8eef-dd6eab455581-7fb9dff67c   1         1         1       41h
replicaset.apps/broker-kafka-display-7fxsh-deployment-5dd57bf6b4                             0         0         0       41h
replicaset.apps/default-broker-filter-5f899d7684                                             1         1         1       2d12h
replicaset.apps/default-broker-ingress-7766b55fd9                                            1         1         1       2d12h

NAME                                               URL                                               LATESTCREATED                LATESTREADY                  READY   REASON
service.serving.knative.dev/broker-kafka-display   http://broker-kafka-display.default.example.com   broker-kafka-display-7fxsh   broker-kafka-display-7fxsh   True

NAME                                                      CONFIG NAME            K8S SERVICE NAME             GENERATION   READY   REASON
revision.serving.knative.dev/broker-kafka-display-7fxsh   broker-kafka-display   broker-kafka-display-7fxsh   1            True

NAME                                                     LATESTCREATED                LATESTREADY                  READY   REASON
configuration.serving.knative.dev/broker-kafka-display   broker-kafka-display-7fxsh   broker-kafka-display-7fxsh   True

NAME                                             URL                                               READY   REASON
route.serving.knative.dev/broker-kafka-display   http://broker-kafka-display.default.example.com   True

NAME                                              READY   REASON   BROKER    SUBSCRIBER_URI                                          AGE
trigger.eventing.knative.dev/testevents-trigger   True             default   http://broker-kafka-display.default.svc.cluster.local   41h

NAME                                  READY   REASON   URL                                               AGE
broker.eventing.knative.dev/default   True             http://default-broker.default.svc.cluster.local   2d12h

NAME                                                                                             TYPE                                 SOURCE                  SCHEMA   BROKER    DESCRIPTION   READY   REASON
eventtype.eventing.knative.dev/dev.knative.apiserver.reso-6ad167ff-0230-4082-8eef-dd6eab455581   dev.knative.apiserver.resource.add   https://10.96.0.1:443            default                 True

NAME                                            READY   REASON   URL                                                           AGE
channel.messaging.knative.dev/testchannel-one   True             http://testchannel-one-kn-channel.default.svc.cluster.local   2d12h

NAME                                                     READY   REASON   URL                                                               AGE
kafkachannel.messaging.knative.dev/default-kne-ingress   True             http://default-kne-ingress-kn-channel.default.svc.cluster.local   2d12h
kafkachannel.messaging.knative.dev/default-kne-trigger   True             http://default-kne-trigger-kn-channel.default.svc.cluster.local   2d12h
kafkachannel.messaging.knative.dev/my-kafka-channel      True             http://my-kafka-channel-kn-channel.default.svc.cluster.local      2d19h
kafkachannel.messaging.knative.dev/testchannel-one       True             http://testchannel-one-kn-channel.default.svc.cluster.local       2d12h

NAME                                                                                                 READY   REASON   AGE
subscription.messaging.knative.dev/default-testevents-trigger-875b1a52-ebc9-457a-8b41-068600f094ae   True             41h
subscription.messaging.knative.dev/internal-ingress-default-5dcde270-7148-46df-bd8c-db9d6e734e8c     True             2d12h

NAME                                                               READY   AGE
apiserversource.sources.eventing.knative.dev/testevents-kafka-03   True    41h
[mjschm3@ip-10-113-78-222 ~]$ kubectl get all
NAME                                                                  READY   STATUS    RESTARTS   AGE
pod/apiserversource-testevents-6ad167ff-0230-4082-8eef-dd6eab42zbmq   1/1     Running   0          16m
pod/default-broker-filter-5f899d7684-qd79f                            2/2     Running   5          16m
pod/default-broker-ingress-7766b55fd9-d4q7h                           2/2     Running   5          16m

NAME                                         TYPE           CLUSTER-IP       EXTERNAL-IP                                              PORT(S)                             AGE
service/broker-kafka-display                 ExternalName   <none>           cluster-local-gateway.istio-system.svc.cluster.local     <none>                              41h
service/broker-kafka-display-7fxsh           ClusterIP      10.102.146.28    <none>                                                   80/TCP                              41h
service/broker-kafka-display-7fxsh-private   ClusterIP      10.103.215.114   <none>                                                   80/TCP,9090/TCP,9091/TCP,8022/TCP   41h
service/default-broker                       ClusterIP      10.96.214.114    <none>                                                   80/TCP,9090/TCP                     2d12h
service/default-broker-filter                ClusterIP      10.108.39.116    <none>                                                   80/TCP,9090/TCP                     2d12h
service/default-kne-ingress-kn-channel       ExternalName   <none>           kafka-ch-dispatcher.knative-eventing.svc.cluster.local   <none>                              2d12h
service/default-kne-trigger-kn-channel       ExternalName   <none>           kafka-ch-dispatcher.knative-eventing.svc.cluster.local   <none>                              2d12h
service/kubernetes                           ClusterIP      10.96.0.1        <none>                                                   443/TCP                             3d21h
service/my-kafka-channel-kn-channel          ExternalName   <none>           kafka-ch-dispatcher.knative-eventing.svc.cluster.local   <none>                              2d19h
service/testchannel-one-kn-channel           ExternalName   <none>           kafka-ch-dispatcher.knative-eventing.svc.cluster.local   <none>                              2d12h

NAME                                                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/apiserversource-testevents-6ad167ff-0230-4082-8eef-dd6eab455581   1/1     1            1           41h
deployment.apps/broker-kafka-display-7fxsh-deployment                             0/0     0            0           41h
deployment.apps/default-broker-filter                                             1/1     1            1           2d12h
deployment.apps/default-broker-ingress                                            1/1     1            1           2d12h

NAME                                                                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/apiserversource-testevents-6ad167ff-0230-4082-8eef-dd6eab455581-7fb9dff67c   1         1         1       41h
replicaset.apps/broker-kafka-display-7fxsh-deployment-5dd57bf6b4                             0         0         0       41h
replicaset.apps/default-broker-filter-5f899d7684                                             1         1         1       2d12h
replicaset.apps/default-broker-ingress-7766b55fd9                                            1         1         1       2d12h

NAME                                               URL                                               LATESTCREATED                LATESTREADY                  READY   REASON
service.serving.knative.dev/broker-kafka-display   http://broker-kafka-display.default.example.com   broker-kafka-display-7fxsh   broker-kafka-display-7fxsh   True

NAME                                                      CONFIG NAME            K8S SERVICE NAME             GENERATION   READY   REASON
revision.serving.knative.dev/broker-kafka-display-7fxsh   broker-kafka-display   broker-kafka-display-7fxsh   1            True

NAME                                                     LATESTCREATED                LATESTREADY                  READY   REASON
configuration.serving.knative.dev/broker-kafka-display   broker-kafka-display-7fxsh   broker-kafka-display-7fxsh   True

NAME                                             URL                                               READY   REASON
route.serving.knative.dev/broker-kafka-display   http://broker-kafka-display.default.example.com   True

NAME                                              READY   REASON   BROKER    SUBSCRIBER_URI                                          AGE
trigger.eventing.knative.dev/testevents-trigger   True             default   http://broker-kafka-display.default.svc.cluster.local   41h

NAME                                  READY   REASON   URL                                               AGE
broker.eventing.knative.dev/default   True             http://default-broker.default.svc.cluster.local   2d12h

NAME                                                                                             TYPE                                 SOURCE                  SCHEMA   BROKER    DESCRIPTION   READY   REASON
eventtype.eventing.knative.dev/dev.knative.apiserver.reso-6ad167ff-0230-4082-8eef-dd6eab455581   dev.knative.apiserver.resource.add   https://10.96.0.1:443            default                 True

NAME                                            READY   REASON   URL                                                           AGE
channel.messaging.knative.dev/testchannel-one   True             http://testchannel-one-kn-channel.default.svc.cluster.local   2d12h

NAME                                                     READY   REASON   URL                                                               AGE
kafkachannel.messaging.knative.dev/default-kne-ingress   True             http://default-kne-ingress-kn-channel.default.svc.cluster.local   2d12h
kafkachannel.messaging.knative.dev/default-kne-trigger   True             http://default-kne-trigger-kn-channel.default.svc.cluster.local   2d12h
kafkachannel.messaging.knative.dev/my-kafka-channel      True             http://my-kafka-channel-kn-channel.default.svc.cluster.local      2d19h
kafkachannel.messaging.knative.dev/testchannel-one       True             http://testchannel-one-kn-channel.default.svc.cluster.local       2d12h

NAME                                                                                                 READY   REASON   AGE
subscription.messaging.knative.dev/default-testevents-trigger-875b1a52-ebc9-457a-8b41-068600f094ae   True             41h
subscription.messaging.knative.dev/internal-ingress-default-5dcde270-7148-46df-bd8c-db9d6e734e8c     True             2d12h

NAME                                                               READY   AGE
apiserversource.sources.eventing.knative.dev/testevents-kafka-03   True    41h

Output from kubectl get all -n kafka for the strimzi cluster the example suggests you deploy. I verified that this cluster was working correctly by running test messages through an example topic:

NAME                                              READY   STATUS    RESTARTS   AGE
pod/my-cluster-entity-operator-759fb44f9b-7mls5   3/3     Running   6          22m
pod/my-cluster-kafka-0                            2/2     Running   2          20m
pod/my-cluster-kafka-1                            2/2     Running   2          20m
pod/my-cluster-kafka-2                            2/2     Running   2          20m
pod/my-cluster-zookeeper-0                        2/2     Running   0          20m
pod/strimzi-cluster-operator-7ff948bcbf-5mvzv     1/1     Running   0          22m

NAME                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/my-cluster-kafka-bootstrap    ClusterIP   10.105.134.149   <none>        9091/TCP,9092/TCP,9093/TCP   2d20h
service/my-cluster-kafka-brokers      ClusterIP   None             <none>        9091/TCP,9092/TCP,9093/TCP   2d20h
service/my-cluster-zookeeper-client   ClusterIP   10.104.145.119   <none>        2181/TCP                     2d20h
service/my-cluster-zookeeper-nodes    ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   2d20h

NAME                                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/my-cluster-entity-operator   1/1     1            1           2d20h
deployment.apps/strimzi-cluster-operator     1/1     1            1           7d18h

NAME                                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/my-cluster-entity-operator-759fb44f9b   1         1         1       2d20h
replicaset.apps/strimzi-cluster-operator-7ff948bcbf     1         1         1       3d22h

NAME                                    READY   AGE
statefulset.apps/my-cluster-kafka       3/3     2d20h
statefulset.apps/my-cluster-zookeeper   1/1     2d20h

Output from kakfa itself to see if topics were created correctly

kubectl -n kafka exec -it my-cluster-kafka-0 -- bin/kafka-topics.sh --zookeeper localhost:2181 --list
Defaulting container name to kafka.
Use 'kubectl describe pod/my-cluster-kafka-0 -n kafka' to see all of the containers in this pod.
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
__consumer_offsets
knative-messaging-kafka.default.default-kne-ingress
knative-messaging-kafka.default.default-kne-trigger
knative-messaging-kafka.default.my-kafka-channel
knative-messaging-kafka.default.testchannel-one
my-topic

[mjschmidt]

Again fully happy to either get you all doing the technical writing at knative the edits needed so far to the Kafka Channel, or to submit a pull request myself, but I do seem to be having problems with the end of the sample, so some help there would be appreciated!

[matzew]

I think the document talks about putting in some numbers:

cat <<-EOF | kubectl apply -f -
---
apiVersion: messaging.knative.dev/v1alpha1
kind: KafkaChannel
metadata:
  name: my-kafka-channel
spec:
  numPartitions: 1
  replicationFactor: 3
EOF

[matzew]

I've changed the URLs to go w/ latest

see #2121

[mjschmidt]

I think the document talks about putting in some numbers:

cat <<-EOF | kubectl apply -f -
---
apiVersion: messaging.knative.dev/v1alpha1
kind: KafkaChannel
metadata:
  name: my-kafka-channel
spec:
  numPartitions: 1
  replicationFactor: 3
EOF

It does, that kafka channel actually works just fine, it is when you go to configure the CRD that things break because that CRD needs partitions and replication factor declared in the yaml it to work when specifying default channel info. I plan to put in a pull request for all of that.

[mjschmidt]

I THINK the issue seems to be with the eventing container.

[mjschmidt]

later in the example