A Problem

[gogogwwb]

No description provided.

[gaocegege]

[kubeflow][worker-0] urllib.error.URLError: <urlopen error [Errno 104] Connection reset by peer>

Seems that there is a problem about network connection.

[gogogwwb]

The network should be normal.

[gogogwwb]

I don’t know why " [kubeflow][worker-0] -----> Running code... " does not appear

[gaocegege]

Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz

yann.lecun.com is blocked in China.

[gogogwwb]

[kubeflow] Building the Docker image...
[kubeflow] Image built successfully
[kubeflow] Getting tensorflow Job jupyter-kernel-nxeof
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
[kubeflow] Waiting for all replicas (0, 1, 1)
Tried 10 times but cannot get the pods
Job jupyter-kernel-nxeof is created.

Pods are normal.It doesn't seem to show the kubeflow log.I don't know why.

[gogogwwb]

Do I need to install kubeflow locally when using Dockerized Kernel?

[gaocegege]

I think so. You need to install tf-operator at least

[gogogwwb]

But if this is the case, what are the advantages of CIAO? If Kubeflow is installed, why not use the built-in Jupyter Notebook?

[gogogwwb]

I'm a little confused.

[gaocegege]

with ciao you can write code and run it distributedly in the notebook.

[gaocegege]

If you do not want to run the training job distributedly in the notebook, you can use kubeflow jupyter or https://github.com/tkestack/elastic-jupyter-operator

[gogogwwb]

OK,thanks!

[kubeflow] Building the Docker image...
[kubeflow] Image built successfully
[kubeflow] Getting tensorflow Job jupyter-kernel-nxeof
[kubeflow] Waiting for all replicas (0, 1, 1)
........
Tried 10 times but cannot get the pods
Job jupyter-kernel-nxeof is created.

Is there a version issue?

[gaocegege]

I am not sure, Can you get the TFJob in the cluster?

[gogogwwb]

yes，It seems that the log cannot be obtained.Is it a label problem?

[gaocegege]

Maybe, I am not sure about it. Is there any log from the kernel?

[gaocegege]

Maybe it is related to labels. TFJob's labels are changed in the latest version.