Get Your Free Google Computer!

Nick Name

Administrator
USA team member
Thanks to a recent thread over at the PrimeGrid forum I have learned that you can access Google's computing resources for free! The thread discusses GPU computing but you also get two CPU instances, if you want to use them. This is a resource offered mainly for student use, but at this time it works great for BOINC! Thanks to some scripts created by some helpful folks it's so easy to set up a cave man could do it. :D The actual hardware is one Tesla K80 GPU and two Xeon 2.30 Ghz CPU threads, not too shabby at all. If you're interested in getting this going, read on! The main limitations I'm aware of at this time are a 12 hour time limit, and of course you need a Google account.

The first thing you should do is create a Primegrid location (like Home, School, Work etc.) with reasonable settings, and set it as your default. Since the instance terminates around the 12 hour mark, you need to select apps that will complete in that time. You also want to set it as the default so your instance will get the correct work. You won't have the luxury of the usual BOINC manager, so it's best to make sure it's set to the correct location from the start. You can check the thread for comments about what people are running. I am crunching the PPSieve, both CPU and GPU. You can see my current host here. (Looks like a long-running Genefer 22 v3.21 prime search slipped in, that's going to fail. )

1. Log into your PrimeGrid account and set up your location.

2. Head on over to https://colab.research.google.com/ and log in with your Google account.

3. You will need to explicitly set the Runtime option to GPU. In the menu at the upper left you will see File | Edit | View etc. Select Runtime -> Change Runtime Type -> Set the Hardware Accelerator to GPU. If you don't do this you will only have two CPU threads available.

4. You will see a couple options beneath the menu, Code and Text. Make sure you are using a Code cell, as you will need to enter a Python script. There are a couple posted in the forum. This is the one I used.


import os.path
!apt-get update
!apt-get install boinc boinc-client
!cp /usr/bin/boinc /content
!cp /usr/bin/boinccmd /content
if not os.path.exists('/content/slots'):
!mkdir slots
if not os.path.exists('/content/slots/0'):
!boinc --daemon --attach_project "http://www.primegrid.com/" "account key here"
else:
!boinc --daemon
!boinccmd --set_gpu_mode always
!tail -f stdoutdae.txt


Paste this exactly as shown, but replace "account key here" with your PrimeGrid WEAK account key. You can find that in your Account settings when you log in.
Leave the quotes. There is a button that looks like a "Play" button beside the cell. Click it to execute the script.

5. You should see a bunch of text as BOINC is installed along with various other packages and libraries, then eventually some familiar BOINC messages about work being downloaded etc.

As I said, you don't have access to the usual BOINC tools like the manager, so if you somehow get the wrong tasks I don't know of any way to abort them other than stopping the instance. You can do that by clicking the Stop button. It's the same button that executes the script, but when the script is running it turns into a square.

As far as I know, you need to both keep the browser window open and active, meaning not minimized, to keep it running.

This should work for any project, as long as you observe the time restriction, set up your location / apps correctly, and of course use the correct project account key.
 

MrVulcan

New Member
It seems to me that I didn't need to indent the last line of the code. It's working, except the GPU computation keeps being suspended. What do I do about that?
Here's the code I'm using at the moment:

import os.path
!apt-get update
!apt-get install boinc boinc-client
!cp /usr/bin/boinc /content
!cp /usr/bin/boinccmd /content
if not os.path.exists('/content/slots'):
!mkdir slots
if not os.path.exists('/content/slots/0'):
!boinc --daemon --attach_project "http://www.primegrid.com/" "***edited for privacy***"
else:
!boinc --daemon
!boinccmd --set_gpu_mode always
!tail -f stdoutdae.txt
 
Last edited by a moderator:

Nick Name

Administrator
USA team member
It's been a little bit since I ran this, and when I did I didn't monitor it that closely. I would guess that if it's being suspended it's something to do with Google's controls overriding BOINC settings. !boinccmd --set_gpu_mode always should have it running 100% as long as the instance is running.
 

supdood

Well-Known Member
USA team member
I tested this out successfully on Einstein and it works great. I also found the below code on the forums that Nick Name mentioned. The original poster said that the code is repetitive on purpose because for some reason it doesn't always work the first time. It allows you to report all finished work and abort all unfinished work so that it doesn't sit around until the deadline.

!pkill boinc > /dev/null 2>&1
!bash -c -e "echo '<cc_config> <log_flags> <task>0</task> <file_xfer>0</file_xfer> <sched_ops>1</sched_ops> <rr_simulation>0</rr_simulation> </log_flags> <options> <abort_jobs_on_exit>1</abort_jobs_on_exit> <use_all_gpus>1</use_all_gpus> <allow_multiple_clients>0</allow_multiple_clients> <max_file_xfers_per_project>20</max_file_xfers_per_project> <max_file_xfers>30</max_file_xfers> <max_tasks_reported>1000</max_tasks_reported> <process_priority>5</process_priority> <process_priority_special>5</process_priority_special> <report_results_immediately>1</report_results_immediately> </options> </cc_config>' > /content/cc_config.xml"
!timeout 4 boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1
!bash -c -e "echo '<cc_config> <log_flags> <task>0</task> <file_xfer>0</file_xfer> <sched_ops>1</sched_ops> <rr_simulation>0</rr_simulation> </log_flags> <options> <abort_jobs_on_exit>1</abort_jobs_on_exit> <use_all_gpus>1</use_all_gpus> <allow_multiple_clients>0</allow_multiple_clients> <max_file_xfers_per_project>20</max_file_xfers_per_project> <max_file_xfers>30</max_file_xfers> <max_tasks_reported>1000</max_tasks_reported> <process_priority>5</process_priority> <process_priority_special>5</process_priority_special> <report_results_immediately>1</report_results_immediately> </options> </cc_config>' > /content/cc_config.xml"
!timeout 4 boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1
 

supdood

Well-Known Member
USA team member
Another free option from Google is their Cloud Compute Engine (https://cloud.google.com/compute). There is an always free tier (1 shared vCPU, Series: N1, Type: f1-micro) and then $300 in credit to use over 12 months. You have to sign up with your Google account and provide a credit card number, but they say that there are no charges beyond the $300 in credit unless you create a new instance after your credit expires (we shall see if that is true!). They make it very easy to see how much credit is remaining, so I'm not too concerned about that. The limitation is only 8 vCPUs at a time (in addition to the 1 always free shared vCPU) and no GPUs.

I'd recommend setting the location to Oregon and the site to us-west1-b. This puts you in one of Google's most efficient data centers and it is powered primarily by hydro.

The best value for the free credits that I've found is to create the always free micro instance and then a second instance of Series: N1, Type: custom (8 vCPUs, 7.25 GB memory). This gets you 8 Xeon Skylake vCPUs for $0.06/hour.

On the Create Instance page:
  • Select the location, series, and type as described above.
  • Change the CPU platform to Intel Skylake or later.
  • Select your boot disk OS
  • Make sure to allow HTTP and HTTPS traffic
  • Under the Management tab set Preemptibility on (this creates a spot instance at about 25% of the cost of a regular one, but you will need to check every once in a while to restart and it will automatically stop if it runs for 24 hours).
  • Make sure everything is as you want it and click create.
Here are the simple steps to get connected and set up:
  1. Connect to the VM from the Google cloud console via the SSH button (this opens a browser window connecting to the VM's terminal).
  2. If you selected Ubuntu as your boot OS, enter the following commands in sequence:
    1. sudo apt update
    2. sudo apt install boinc-client
    3. boinccmd --project_attach URLHere WeakKeyHere
  3. Use standard boinccmd commands and syntax to interact with your VM instance (https://boinc.berkeley.edu/wiki/Boinccmd_tool)
The great thing about this (other than getting 8 powerful Xeon vCPUs for free for almost 5,000 hours) is that when the spot instance is stopped, you don't lose what is on disk. So all tasks restart from the last checkpoint when you restart the VM and BOINC projects continue to recognize it as a single host.
 

MrVulcan

New Member
I tested this out successfully on Einstein and it works great. I also found the below code on the forums that Nick Name mentioned. The original poster said that the code is repetitive on purpose because for some reason it doesn't always work the first time. It allows you to report all finished work and abort all unfinished work so that it doesn't sit around until the deadline.

!pkill boinc > /dev/null 2>&1
!bash -c -e "echo '<cc_config> <log_flags> <task>0</task> <file_xfer>0</file_xfer> <sched_ops>1</sched_ops> <rr_simulation>0</rr_simulation> </log_flags> <options> <abort_jobs_on_exit>1</abort_jobs_on_exit> <use_all_gpus>1</use_all_gpus> <allow_multiple_clients>0</allow_multiple_clients> <max_file_xfers_per_project>20</max_file_xfers_per_project> <max_file_xfers>30</max_file_xfers> <max_tasks_reported>1000</max_tasks_reported> <process_priority>5</process_priority> <process_priority_special>5</process_priority_special> <report_results_immediately>1</report_results_immediately> </options> </cc_config>' > /content/cc_config.xml"
!timeout 4 boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1
!bash -c -e "echo '<cc_config> <log_flags> <task>0</task> <file_xfer>0</file_xfer> <sched_ops>1</sched_ops> <rr_simulation>0</rr_simulation> </log_flags> <options> <abort_jobs_on_exit>1</abort_jobs_on_exit> <use_all_gpus>1</use_all_gpus> <allow_multiple_clients>0</allow_multiple_clients> <max_file_xfers_per_project>20</max_file_xfers_per_project> <max_file_xfers>30</max_file_xfers> <max_tasks_reported>1000</max_tasks_reported> <process_priority>5</process_priority> <process_priority_special>5</process_priority_special> <report_results_immediately>1</report_results_immediately> </options> </cc_config>' > /content/cc_config.xml"
!timeout 4 boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1
At the risk of overlooking the glaringly obvious...where can I find this Einstein?
Thank you!
 

Nick Name

Administrator
USA team member
I kind of forgot about this for a while, recently I started trying to run it a couple times a week. Today I noticed that the instance had terminated after a short time and when I tried to reconnect I got a message saying I've exceeded the usage limit and look into Colab Pro. The last time I used it was Wednesday 7-8. I don't know if this is due specifically to BOINC, overall compute load or something else. I wasn't thinking and minimized the window so that might be the problem too. Anyway, I'm going to wait a week and try again.
 
Top