Can't run server with "seldon-up.sh" - mysql status is forever Pending

[ukrbublik]

I run Kubernetes locally via Minikube 0.12.1. My OS: Ubuntu 16.04. RAM: 16G (and 16GB swap).
Problem: when I run "seldon-up.sh" (Seldon v1.3.11), I see mysql node status is forever Pending.
When I run "kubectl get events", I see this:
24m 35m 40 mysql Pod Warning FailedScheduling {default-scheduler } pod (mysql) failed to fit in any node
fit failure on node (minikube): Insufficient memory

But I have 8.4GB RAM free.

[ukclivecox]

Did you start minikube with enough memory?, e.g.
minikube start --memory=8000 --cpus=4

[ukrbublik]

Did you start minikube with enough memory?, e.g.
minikube start --memory=8000 --cpus=4

Yes, tried now with this flags, no changes.

[ukclivecox]

Did you to do a minikube delete before doing a minikube start?
If you just do minikube stop then the new memory setting will not be used.

[ukrbublik]

Did you to do a minikube delete before doing a minikube start?
If you just do minikube stop then the new memory setting will not be used.

Thanks, now mysql is running and seldon server starts ok.

But I have another problem with trying Reuters Newswire Recommendation example, can you help?
When I run "seldon-cli --quiet api --client-name reuters --endpoint /js/recommendations --item 6020 --limit 3", I get:
WARNING:kazoo.client:Connection dropped: socket connection error: No address associated with hostname
WARNING:kazoo.client:Connection dropped: socket connection error: No address associated with hostname
...
WARNING:kazoo.client:Failed connecting to Zookeeper within the connection retry policy.
Traceback (most recent call last):
File "/usr/local/bin/seldon-cli", line 5, in
seldon.cli.start_seldoncli()
File "/usr/local/lib/python2.7/dist-packages/seldon/cli/init.py", line 3, in start_seldoncli
cli_main.main()
File "/usr/local/lib/python2.7/dist-packages/seldon/cli/cli_main.py", line 346, in main
start_zk_client(opts)
File "/usr/local/lib/python2.7/dist-packages/seldon/cli/cli_main.py", line 301, in start_zk_client
gdata["zk_client"].start()
File "/usr/local/lib/python2.7/dist-packages/kazoo/client.py", line 546, in start
raise self.handler.timeout_exception("Connection time-out")
kazoo.handlers.threading.KazooTimeoutError: Connection time-out

Maybe it's related to fact that I tried to follow this guide "Manual install outside Kubernetes" (http://docs.seldon.io/seldon-cli.html#manual) when I had problem with minikube.
But now I returned "zk_hosts" value to "zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181" in ~/.seldon/seldon.conf

[ukclivecox]

This looks like either a DNS issue, but if you are running inside minikube this seems unlikely.
Can you recreate the conf - "make clean conf" from kubernetes/conf
Suggest you also ssh into seldon-control and check you can find a zookeeper instance by name, e.g. try telnet zookeeper-1 2181

[ukrbublik]

Can you recreate the conf - "make clean conf" from kubernetes/conf

Recreated. Did not help.

Suggest you also ssh into seldon-control and check you can find a zookeeper instance by name, e.g. try telnet zookeeper-1 2181

Yes, seems I can.

root@seldon-control:/home/seldon# telnet zookeeper-1 2181
Trying 10.0.0.198...
Connected to zookeeper-1.default.svc.cluster.local.
Escape character is '^]'.

I also tried to add this to /etc/hosts:
172.17.0.6 zookeeper-1
172.17.0.5 zookeeper-2
172.17.0.7 zookeeper-3
(IPs taken from kubernetes' web ui pods section)
Then I get:
WARNING:kazoo.client:Connection dropped: socket connection error: No route to host

[ukclivecox]

You shouldn't need to anything to /etc/hosts
Are you running inside minikube?

[ukrbublik]

No.
I just did these commands at local pc terminal:

minikube start
seldon-up.sh
... run import data job, recommendation microservice
seldon-cli --quiet api --client-name reuters --endpoint  /js/recommendations --item 6020 --limit 3

Sorry for my noob questions.
I should first run kubectl exec -ti seldon-control -- /bin/bash
Then at root@seldon-control I should run seldon-cli commands. Right?

[ukclivecox]

No. You can add seldon-cli which is in kubernetes/bin to your PATH.

(internally it will call seldon-cli running inside seldon-control)

[ukrbublik]

Now it works. Thanks.
seldon-cli was also in /usr/local/bin/ , this caused problem

[ukclivecox]

Great. We'll update out docs regards minikube to cover your situation.