Please find below some minimal knowledge to use Google Colab (jupyter notebook) efficiently.
Notebooks contain cells. These cells can be of two type:
- 'Text' cells that contain information to guide users.
- 'Code' cells that can be run.
Importantly, please always run code cells one by one from the top to the bottom of the page as every cell depends on the output of its previous cells. So please do not skip any code cell unless it is specifically indicated that you can. Skipping text cells is ok, but keep in mind that those cells are there for guidance.
-
To access Google drive from a browser click here and enter your credentials (you may have to switch id, provided you use several accounts, by clicking on the letter within a colored circle top right of the page).
-
Once loaded just drag and drop files/folders from your browser to the Google drive web interface (and wait for the upload to complete, this may take time...). Files stored in Google drive can easily be accessed from Colab (see below).
Data stored on google drive can easily be accessed from within Colab to be used for training neural networks and saving trained model and/or using neural network to segment images. To use drive you need a google account. I personnally recommend creating create a second account dedicated to deep learning to avoid issues.
- Click on the files icon on the left of the Colab window.
- Wait some time and a Google drive icon should appear. Do click it.
- Press 'Connect to google drive'.
- The Google 'drive' folder is now visible. Browse it until you see a 'My Drive' folder (this is the root of the Google drive folder).
- Browse the link provided by Google (a new page appears)
- Scroll down and click 'Authorize'
- Click the 'clipboard' icon to copy the code to the clipboard
- Get back to the colab page then paste (Ctrl/Cmd + V) the code in the 'authorization' field. Finally press 'Enter'
- The drive is now successfully mounted
- Select the cell (by clicking on it).
- Press the play button (or brackets) located top left corner of the selected code cell to run it.
- Successful execution outputs a result (failed execution raises an error that is useful to debug).
Deep learning is computation intensive and is better achieved using GPU (graphic cards) than regular computer processors. To get a Colab virtual machine that uses a GPU:
- Press the 'Runtime' menu entry.
- Choose 'Change runtime type'.
- Select 'GPU'.
- Press 'Save' (the virtual machine will connect to a GPU empowered system).
- Colab is free.
- Colab gives free access to GPU (graphic cards) and even TPU.
- Colab can access Google drive (Google drive is the most efficient way to transfer data to Colab; the Colab upload tool being extremely slow).
- Colab offers free (temporary) storage on its virtual machines.
- Colab session cannot run longer than 12 hours. After 12 hours everything saved on your Google drive is still yours, the virtual machine is reset and data that was not saved on your personal drive is lost...
- You shouldn't run several Colab notebooks in parallel (you can get banned if you do so), keep in mind it's a free service.
- You are limited by the size of your Google drive (trained models can use a lot of space and may easily fill you Google drive, making your mails non accessible, so the best is to create a second account dedicated to deep learning).
- Access to GPU or TPU is not guaranteed in the free version (if no GPU is available best is to quit and come back later).
- Colab will also disconnect the session if left idle for too long (even if a neural network is running and the 12h limit is not reached) this is why it is highly recommended to regularly press on the
icon during training (smart people may run a script in their browsers or use 3rd party software to automate that, but if you choose to do so you are doing this at your own risk).
PS: using Colab pro (commercial version) alleviates time limitation and idle disconnect timeout