Service Unavailable

[fabfianda]

Hi,
after installing mizu binary I run:
mizu tap ".*" -A

I can see the list of pods being tapped in the terminal.
I get:
Mizu is available at http://localhost:8899/mizu

However when I try to open that I get:
`{
"kind": "Status",
"apiVersion": "v1",
"metadata": {

},
"status": "Failure",
"message": "error trying to reach service: dial tcp 10.2.19.212:8899: i/o timeout",
"reason": "ServiceUnavailable",
"code": 503
}
`
kubectl is correctly working on my box.
I can also see the temporary mizu-collector pod running in the cluster.

Is it a port forward issue?

[fabfianda]

I've also added mizu roles using this manifest file
examples/roles/permissions-all-namespaces.yaml

I'm experiencing same issue though.

[fabfianda]

Last log line from the mizu-collector pod is:

2021-07-30T14:16:28Z INFO : Websocket Connection event - Tapper connected: PoLQpd5knkGjzEky5D6YzNkOabMOMoEmIyTBxGIZvz8p3GhBwppv8NRmXtD8BHb023I4SKGc2WDQV5v2dlVmnrYE3iFTY1c7V0Pe

[IgorGov]

Hi @fabfianda,
Thanks for reaching out.
What version of Mizu do you use? In latest stable release we use proxy instead of port forward.
It might be a bug that the proxy not ready yet, can you try to reload the page after few seconds/a minute.

BTW You are welcome to join our public slack channel at the following link:
https://join.slack.com/t/up9/shared_invite/zt-tfjnduli-QzlR8VV4Z1w3YnPIAJfhlQ

[fabfianda]

I'm using the one you get from the homepage.
Even a few minutes it still claims service is Unavailable

[IgorGov]

Can you check that kubectl proxy works for you?
https://kubernetes.io/docs/tasks/extend-kubernetes/http-proxy-access-api/

[fabfianda]

Yes, it loads and works as expected

[fabfianda]

I've just tried version 0.7.26 and the response is same:

kind	"Status"
apiVersion	"v1"
metadata	{}
status	"Failure"
message	"error trying to reach service: dial tcp 10.2.19.218:8899: i/o timeout"
reason	"ServiceUnavailable"
code	503

[fabfianda]

More information:

proxy launched by mizu works as expected too:
so if I open http://localhost:8899/ while mizu is running I can see the k8s proxy response (with list of endpoints)

[fabfianda]

endpoints created on k8s seem good too

mizu-api-server mizu 10.2.19.220:8899

[IgorGov]

Thank you for the additional info @fabfianda
The proxy seems to work fine, the next thing to check is the web socket. The "CLI" opens a web socket to the "Agent" on "ws://localhost:8899/mizu/ws" I suspect it fails to connect. It should be written in the logs. Can you please share here the logs from "mizu-collector" pod, and the CLI Stderr log?

[fabfianda]

Update: tried with other clusters and it seems to work as intended. So the issue is currently limited to a single K8s cluster. I'll try and provide more information.

[IgorGov]

Thanks for the heads-up! We are planing to release a new stable version on Sunday, which has many fixes. It might solve your issue as well.
If not, you will be able to run it with 'dump-logs' option mizu tap ".*" --set dump-logs=true - which will store the logs from cli and k8s nodes to a zip file which you can share here.

[IgorGov]

Hi @fabfianda, we have a new Mizu version: 0.10.0 which contains several fixes and new features. You can run it agains your cluster which earlier version had issues with.
If it still not working please run Mizu with dump-logs option (mizu tap --set dump-logs=true) and upload the zip file created at the end of the execution with all logs to the issue.

[IgorGov]

@fabfianda hope both of the clusters work for you with the new version. I am closing this issue because of inactivity, you are more then welcome to open a new one if you have any additional issues