Google CoLab Instructions (CPU and GPU)

[Danc2050]

(Note: This "Issues" post is under construction! I'm starting school, so I wanted to post my findings somewhere, with the plan to be periodically updated. I hope this is okay, Teal).

There are some opportunities to run your scripts under Google Colab, Teal.

Opportunities

1. Using regular CPU
   This can be done indefinitely...at least as long as the browser is open (although it is a little slow, it is faster than an Always Free Tier Google Compute Engine). Here is the code:

!sudo apt-get update # optional
!sudo apt-get dist-upgrade -y # optional
!wget https://raw.github.com/tdulcet/Distributed-Computing-Scripts/master/mprime.sh -q
!sed -i '/^nohup / s/^/# /' mprime.sh # Do not start MPrime
!sed -i '/^crontab / s/^/# /' mprime.sh # Do not create a cronjob
!bash -- mprime.sh danc2 GCodeLabs

!wget https://raw.github.com/tdulcet/Distributed-Computing-Scripts/master/mprime.sh -qO - | bash -s -- danc2 GCodeLabs
!pkill mprime # to kill the process and start it manually (see note below)

!cd /content/mprime/ && ./mprime -d # -d is optional, but good to check to see if its working.
!ps -al | grep mprime

Note: I use pkill because 1) killall is not installed and 2) GIMPS has yet to receive any info back from these machines and I wanted to use the -d command to see output.

2. Using GPU
   This must be done within a 12-hour period

3. Using a TPU
   This step would require crafting our own Prime Computation machine, likely in Python, to make use of Google Colabs TPU power.
   To-Do

• Determine if installing cron (sudo apt-get install cron) into the machine enables them to run without a browser window being opened. (My theory is the machines are ephemeral and get reused once not in use, even if a cron job is working, but I need to verify).
• Determine if, when using CUDALucas, a workload can complete in the 12-hour period maximum for a GPU workload (or if the machine can resume GPU work after whatever interval Google requires is over without previous work being lost).
• Look into how to log into a Colab remotely and then TMUX the session.

[tdulcet]

Wow! Thanks for doing all this research!

Some notes on the code:

1. You will want to replace sudo apt-get upgrade with sudo apt-get dist-upgrade -y to update everything.
2. You will likely need to replace /content/mprime/mprime -d with cd /content/mprime/ && ./mprime -d so that MPrime starts in the correct directory with its configuration files (this is probably why you have not received any info back).
3. Does the MPrime script run without errors? You mentioned cron not being installed, so I assume the crontab line produces an error. You may want to download the script, remove the lines that you do not want or that do not work and then execute it. For example:

wget https://raw.github.com/tdulcet/Distributed-Computing-Scripts/master/mprime.sh -q
sed -i '/^nohup / s/^/# /' mprime.sh # Do not start MPrime
sed -i '/^crontab / s/^/# /' mprime.sh # Do not create a cronjob
bash -- mprime.sh danc2 GCodeLabs

CUDALucas will likely take more than 12 hours depending on what GPU Google Colab is using, but you should easily be able to back up the files after the 12 hours and restore them the next day to continue working...

[Danc2050]

Answers and solutions to all my questions as usual.

I put in your updates and haven't tested them. I trust your script skills implicitly. I also added a section on using the TPU that Google Colab also offers, though I haven't researched this yet either.

[Danc2050]

I found a website that linked to a GitHub where an author has done this work already. The same author also has a script to setup an SSH tunnel, which would make things easier (e.g., run without a browser). Right now it is just doing Trial-factoring, but I am sure the script can be modified.

Furthermore, as I had mentioned in #5 this process can be done with IBM's supercomputer Watson for some amount of computing hours (they have some formula based on work done, I think). By adding to this script to sftp/scp the work off of the host machine of Watson before the allocation of compute time is finished (a suggestion you gave) then, in theory, a supercomputer could be used for a GIMPS project.