Google Cloud Platform

[datalass1]

I have a Google Cloud Platform account. Previously I used the GeoCode API to change postcodes from a housing dataset to coordinates for mapping house prices in Somerset.

https://console.cloud.google.com/home/dashboard?project=geopackage-223116

I have £230.58 credit which will end on 9th November.

[datalass1]

fastai source to set up Google Cloud SDK
fastai source to get back to work in GCP.

CLOUD SDK

Command-line interface for Google Cloud Platform products and services.
The Cloud SDK is a set of tools for Cloud Platform. It contains gcloud, gsutil, and bq, which you can use to access Google Compute Engine, Google Cloud Storage, Google BigQuery, and other products and services from the command-line. You can run these tools interactively or in your automated scripts.

gcloud Tool

gcloud manages authentication, local configuration, developer workflow, and interactions with the Cloud Platform APIs.

Step 1: Creating your account

I already have an account. Skip this.

Step 2: Install Google CLI

Install Google Cloud’s command line interface (CLI) software from Google.

# Create environment variable for correct distribution
export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)"

# Add the Cloud SDK distribution URI as a package source
echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

# Import the Google Cloud Platform public key
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

# Update the package list and install the Cloud SDK
sudo apt-get update && sudo apt-get install google-cloud-sdk

source Google Cloud SDK documentation

Run gcloud init to start the authentication process. Hit enter when prompted.
gcloud init

If at this stage I create a new project rek-fastai
Go into google cloud console Projects by billing account in manage billing to link billing account ID to the new project.

Step 3: Increase quotas

from error: ERROR: (gcloud.compute.instances.create) Could not fetch resource: - Quota 'GPUS_ALL_REGIONS' exceeded. Limit: 0.0 globally

Info on Quotas

1. Go to GCP IAM and admin > Quotas
2. Filter on location to us-west2
3. Select the service to edit
4. fill in contact details, select number of GPUs and reason.
   
5. wait for email confirmation, could take 2 days. Mine took a couple of hours, so they are pretty fast.

Step 4: SSH to the fastai instance

$ gcloud compute ssh --zone=$ZONE jupyter@$INSTANCE_NAME -- -L 8080:localhost:8080

Step 5: Connect to Jupyter Notebook

In a web-browser go to:
localhost:8080/tree
Note that this only works while you maintain the ssh connection in your terminal.

Step 6: pull from the fastai repo and update the fastai library

Now your command line which should show a prompt along the lines of jupyter@my-fastai-instance

You should make sure Github is configured and pull from the repository. You can do this by typing the following lines:

cd tutorials/fastai/course-v3
git checkout .
git pull

You should also update the fastai library:
sudo /opt/anaconda3/bin/conda install -c fastai fastai

Things to note

Preemptible instances:
We are running a preemptible instance (notice the ‘–preemptible’ parameter in our command). A preemptible GCP instance is cheaper than traditional instances but it has two main disadvantages:

It can be preempted (stopped) with a 30 second notice at any time due to high demand.
It will always be stopped after 24 hours of continuous running.
If your instance is stopped, your saved data will be kept safe but if you are running a model, the progress will be lost.

Remove it and the price of the instance will go up. But better for deeper models if I move beyond beginner.

Sources of help:

• stackoverflow
• tutorial on GCP

[datalass1]

How to connect to an existing GCP instance

Providing that gcloud is installed, see above Step 2: Install Google CLI:
$ gcloud compute ssh --zone=$ZONE jupyter@$INSTANCE_NAME -- -L 8080:localhost:8080

So my fastai instance is:
$ gcloud compute ssh --zone="us-west2-b" jupyter@"my-fastai-instance" -- -L 8080:localhost:8080

Then connect to Jupyter notebook.
In a web-browser go to:
localhost:8080/tree

Hey Pesto! You are connected to a US server with awesome compute for fastai for FREE! (for now) :)

[datalass1]

Troubleshooting

1. E: Malformed entry 1 in list file /etc/apt/sources.list.d/google-cloud-sdk.list (Component) E: The list of sources could not be read.
   Solution: Remove the /etc/apt/sources.list.d/google-cloud-sdk.list and re-try the gcloud installation steps solved the problem for my case.