Back-off pulling image "data_downloader:0.1": ImagePullBackOff

[kenhglee]

Hello!

I was following through the kubeflow_seldon_e2e tutorial to encounter the ImageBullBackOff error when running the pipeline.

To make sure my docker is running correctly by the result of

• cd ../data_downloader && ./build_image.sh

I went on to check

• docker run data_downloader:0.1

does not create any errors and went further to register it to Google Cloud Registry by tagging and push/pulling all right though it may not be relevant. Yet those double checking in the CLI proved not helpful in resolving the run failure.

Please share any thoughts or suggestions to resolve this. Thank you!

Regards, Ken

[axsaucedo]

This is due to the cluster not being able to download the image. I assume that you tagged it with the format of <GCP_REGISTRY>/<image>:<tag>.

That same format should be in two locations: the train_pipeline/nlp_pipeline.py pipeline definition, as well as in the deploy_pipeline/seldon_production_pipeline.yaml file.

Can you check that both files have the GCP_REGISTRY prefixed?

[kenhglee]

Hi Alejandro,

Thank you a million! Realizing that the images have to be renamed in nlp_pipeline.py by the GCP convention as you suggested, the pipeline started progressing without changing anything on the deployment side just yet. But, will keep in mind on the second half portion as well.

Regards, Ken

[kenhglee]

And just wanted to confirm that the renaming of images by the GCR_REGISTRY format on the deployment side proved necessary indeed, even after the pipeline ran through without, resulting in a "green" seldon-deployment-* deployment on the k8s workload. Thanks again!

[axsaucedo]

Awesome to hear @kenhglee - glad it helped 😊