GCP Initialization Action

[aashishyadavally]

@Anirudh-Kakarlapudi: Like you observed, I have tried setting up the GCP with an Initialization Action, which is failing. Can you share the gist from the logs, which we can try to debug?

Also, try GCP's dataproc-initialization-action script as well, to see if that helps.

[aashishyadavally]

Parallelly, I was hoping to explore gcloud compute ssh to see if that would help us in any way, in case this doesn't fall through.

[Anirudh-Kakarlapudi]

I tried multiple times yesterday to create a cluster using initialization action gs://uga-dsp/scripts/conda-dataproc-bootstrap.sh. But it started giving some errors.

1. Creating the conda folder
2. Downloading the anaconda
3. All the files are getting deleted (probably an error is occurring during installation and everything is getting deleted)

[Anirudh-Kakarlapudi]

Today, I was successful to create the cluster with following specs

1. region: us east
2. default worker nodes, memory
3. initialization action: gs://uga-dsp/scripts/conda-dataproc-bootstrap.sh

Please try to create a cluster and check if you are facing any problems

[aashishyadavally]

Two things I can point to there;
(a) Can you point to a specific error in the logs which might have been leading to this? We could then reach out to Dr. Quinn accordingly.
(b) Is it more like a chance thing, where the script is working some times, and the other times, it is not? If so, do you think it is worth to spend time on it, taking into consideration the deadlines?

[Anirudh-Kakarlapudi]

I don't have any specific logs relating to the error. Also I don't think it is a chance thing because the initialization action never failed today as I tested by creating 5 clusters today.

The important thing i noticed is if you try to run the VM instances as soon as you created a cluster with cluster status 'provisioning', the anaconda is not getting installed. It is taking almost 10 min before the cluster is getting ready with everything installed.

[aashishyadavally]

That's weird, considering it didn't take as much time during P1.
Okay, we will go ahead and close this issue here, and come back to addressing consistency in initialization actions when we are well ahead with the code base.