Gdistcc provides easy access to compiling 'make' based software on Google Compute Engine via distcc with local caching via ccache. The inclusion of ccache provides a local cache of compiled objects, to prevent costly repeat compilation.
Gdistcc has been designed to require minimal dependencies outside of your base distribution and the Google Cloud APIs.
- CentOS 7
- Ubuntu 16.04 LTS
- Ubuntu 14.04 LTS
- Debian 8
- python 2.7
Google Cloud SDK (Instructions Below)
google-api-python-client (Instructions Below)
- Ensure your Linux distro is currently supported, and fully updated. This is critical to ensure the development toolchain is compatible.
sudo yum upgrade -y
sudo yum install ccache distcc git
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install ccache distcc git python-pip python-googleapi
NOTE: If a new kernel is installed, please reboot before continuing.
- Authenticate with the Google Cloud
You CAN choose gdistcc as your default project and us-central-c as the default zone. But it is NOT mandatory.
- Install the google-api-python-client via your distro or pip.
sudo pip install --upgrade google-api-python-client
- Install gdistcc
sudo pip install gdistcc
Alternatively you can clone the full source from github and use the './gdistcc.py' wrapper in place of 'gdistcc' below.
git clone https://github.com/apeabody/gdistcc
Gdistcc is designed to be stateless, however there is a minimal config file to customize the project, zone, and prefix if needed.
usage: gdistcc [-h] [--settingsfile SETTINGSFILE] [--qty QTY]
[--skipfullstartup] [--globalinstances] [--version]
-h, --help show this help message and exit
Custom settings file. (default:
--qty QTY Qty of Instances to deploy during start mode.
--skipfullstartup Skip waiting for full instance startup during start
--globalinstances Use all discovered instances for this prefix and
distro, not just ones started by the local hosts
--version show program's version number and exit
Copyright 2016 Andrew Peabody. See README.md for details.
An example that starts (4) gdistcc instances in preparation for a remote compile, and polls till they are fully online and ready. This normally take a minute or two depending on the speed of the instance, and number of dependencies.
gdistcc start --qty 4
NOTE: By default gdistcc will wait for all instances to fully start, this can be skiped with
--skipfullstartup. This may be useful if the local machine is fast enough to start the compile in advance of the 1-2minute full startup.
NOTE: The first time you use glcoud on this host you may be prompted to enter a passphrase - please enter twice to use no passphrase.
Check the status of your gdistcc instances.
NOTE: This will currently fail if the instances have been preempted by Google.
Build your 'make' based application, be sure to be in the desired build root before running.
Stop your gdistcc instances.
It is recommended to add these options to your ssh client to suppress the hostkey checks, and default to the high performance firstname.lastname@example.org cipher for your gdistcc instances.
NOTE: In some cases I've found the ControlMaster mux to be unreliable with multiple streams of simulantious file transfer, but when using g1-small instances doing a single build I'm hoping this will work and greatly speed up the ssh connection.
- Always confirm all instances are shutdown after use - you are solely responsible for their cost.
- Gdistcc uses preememptible instances which offer preferred pricing, but Google may shutdown on short notice. A
gdistcc statusand fresh
gdistcc makewill check (and avoid using) an instance that has been preemempted,
gdistcc stopwill delete a terminated instance as normal. One "advantage" of preemptible instances is they won't run more than 24hr, reducing the risk of forgotten instances.
- Future versions may not require ccache.
- Only SSH is supported at the transport for distcc. Distcc's native TCP transport is not enabled due to security concerns.
- Gdistcc does NOT currently use distcc's Pump Mode for the following reasons:
- Gdistcc is intended for frequent re-compiles, so most header pre-processing will hopefully be cached by ccache anyway - mutally exclusive from pump mode.
- Gdistcc uses ssh over the internet for transfers, so minimizing the transfered file size is advantageous. (In a local/HPC setup distcc can be used over TCP for higher transfer speeds.)
- Installing the required system headers would slow the instance startup significantly.
- While system headers from normal repos are easily added to the start up script, others would require significant customization/setup time.
--pumpmodecould/might be added in the future for those so inclined to the above notes.
Based on sample code Copyright 2015 Google Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.