Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka install timesout in the grpc.kafka.proxy example #85

Closed
vordimous opened this issue Oct 23, 2023 · 3 comments
Closed

Kafka install timesout in the grpc.kafka.proxy example #85

vordimous opened this issue Oct 23, 2023 · 3 comments

Comments

@vordimous
Copy link
Contributor

./setup.sh 
+ helm install zilla-grpc-kafka-proxy-kafka chart --namespace zilla-grpc-kafka-proxy --create-namespace --wait --timeout 2m
Error: INSTALLATION FAILED: timed out waiting for the condition

Kakfa looks to have started correctly so I manually finish the ./startup.sh commands which succeed. However the first grpcurl command just hangs.

I attempted to list the kafka topics:

kcat -b localhost:9092 -L
%3|1698090965.456|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv6#[::1]:9092 failed: Connection refused (after 1ms in state CONNECT)
%3|1698090966.459|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv6#[::1]:9092 failed: Connection refused (after 0ms in state CONNECT, 1 identical error(s) suppressed)
%3|1698090967.464|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 0ms in state CONNECT)
%3|1698090968.464|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv6#[::1]:9092 failed: Connection refused (after 0ms in state CONNECT)
%3|1698090969.468|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 0ms in state CONNECT)
% ERROR: Failed to acquire metadata: Local: Broker transport failure (Are the brokers reachable? Also try increasing the metadata timeout with -m <timeout>?)
@jfallows
Copy link
Contributor

@vordimous I'm not able to reproduce this on main.

% ./setup.sh 
+ docker image inspect zilla-examples/grpc-echo:latest --format 'Image Found {{.RepoTags}}'
Image Found [zilla-examples/grpc-echo:latest]
+ ZILLA_CHART=oci://ghcr.io/aklivity/charts/zilla
+ helm install zilla-grpc-kafka-proxy oci://ghcr.io/aklivity/charts/zilla --namespace zilla-grpc-kafka-proxy --create-namespace --wait --values values.yaml --set-file 'zilla\.yaml=zilla.yaml' --set-file 'configMaps.proto.data.echo\.proto=proto/echo.proto' --set-file 'secrets.tls.data.localhost\.p12=tls/localhost.p12'
Pulled: ghcr.io/aklivity/charts/zilla:0.9.55
Digest: sha256:d611cc39c99b2a4654c140f5260d8c6a9db7a9c57ee4bce80261d1ca083acc17
NAME: zilla-grpc-kafka-proxy
LAST DEPLOYED: Mon Oct 23 14:30:08 2023
NAMESPACE: zilla-grpc-kafka-proxy
STATUS: deployed
REVISION: 1
NOTES:
====
    Copyright 2021-2023 Aklivity Inc

    Licensed under the Aklivity Community License (the "License"); you may not use
    this file except in compliance with the License.  You may obtain a copy of the
    License at

      https://www.aklivity.io/aklivity-community-license/

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
    WARRANTIES OF ANY KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations under the License.
====

Zilla has been installed.

This is how to check zilla logs:
--------------------------------
$ export ZILLA_POD=$(kubectl get pods --namespace zilla-grpc-kafka-proxy --selector app.kubernetes.io/instance=zilla-grpc-kafka-proxy -o json | jq -r '.items[0].metadata.name')
$ kubectl logs --namespace zilla-grpc-kafka-proxy $ZILLA_POD

This is how to expose zilla with port-forward:
----------------------------------------------
$ export SERVICE_PORTS=$(kubectl get svc --namespace zilla-grpc-kafka-proxy zilla-grpc-kafka-proxy --template "{{ range .spec.ports }}{{.port}} {{ end }}")
$ eval "kubectl port-forward --namespace zilla-grpc-kafka-proxy service/zilla-grpc-kafka-proxy $SERVICE_PORTS" > /tmp/kubectl-zilla.log 2>&1 &

# cleanup:
$ pgrep kubectl && killall kubectl
+ helm install zilla-grpc-kafka-proxy-kafka chart --namespace zilla-grpc-kafka-proxy --create-namespace --wait --timeout 2m
NAME: zilla-grpc-kafka-proxy-kafka
LAST DEPLOYED: Mon Oct 23 14:30:21 2023
NAMESPACE: zilla-grpc-kafka-proxy
STATUS: deployed
REVISION: 1
TEST SUITE: None
++ kubectl get pods --namespace zilla-grpc-kafka-proxy --selector app.kubernetes.io/instance=kafka -o name
+ KAFKA_POD=pod/kafka-74675fbb8-778f6
+ kubectl exec --namespace zilla-grpc-kafka-proxy pod/kafka-74675fbb8-778f6 -- /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic echo-requests --if-not-exists
Created topic echo-requests.
+ kubectl exec --namespace zilla-grpc-kafka-proxy pod/kafka-74675fbb8-778f6 -- /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic echo-responses --if-not-exists
Created topic echo-responses.
+ kubectl port-forward --namespace zilla-grpc-kafka-proxy service/zilla-grpc-kafka-proxy 9090
+ kubectl port-forward --namespace zilla-grpc-kafka-proxy service/kafka 9092 29092
+ nc -z localhost 9090
+ kubectl port-forward --namespace zilla-grpc-kafka-proxy service/grpc-echo 8080
+ sleep 1
+ nc -z localhost 9090
Connection to localhost port 9090 [tcp/websm] succeeded!
+ nc -z localhost 9092
Connection to localhost port 9092 [tcp/XmlIpcRegSvc] succeeded!
+ nc -z localhost 8080
Connection to localhost port 8080 [tcp/http-alt] succeeded!

Note that I did also execute the following command first, per the README.

docker build -t zilla-examples/grpc-echo:latest .

@vordimous
Copy link
Contributor Author

ran again and didn't hit the timeout.

@attilakreiner
Copy link
Contributor

Looks good to me, too.

Bitnami only provides AMD64 images for kafka, so if you run it on an ARM64 MacBook, java/kafka will run in emulation mode, which can result in an error where kafka won't start in very rare cases (though I haven't seen this error lately at all).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants