build: document electron goma usage and add the external binaries (#2…

…1622)

build/args/goma.gn

@@ -0,0 +1,2 @@
+goma_dir = rebase_path("//electron/external_binaries/goma")
+use_goma = true

docs/development/goma.md

@@ -0,0 +1,61 @@
+# Goma
+
+> Goma is a distributed compiler service for open-source projects such as
+> Chromium and Android.
+
+Electron has a deployment of a custom Goma Backend that we make available to
+all Electron Maintainers.  See the [Access](#access) section below for details
+on authentication.
+
+## Enabling Goma
+
+Currently Electron Goma supports both Windows and Linux, we may add macOS
+support at some point in the future.  If you are on a supported platform
+you can enable goma by importing the `goma.gn` config file when using `gn`.
+
+```bash
+gn gen out/Testing --args="import(\"//electron/build/args/testing.gn\") import(\"//electron/build/args/goma.gn\")"
+```
+
+You must ensure that you do not have `cc_wrapper` configured, this means you
+can't use `sccache` or similar technology.
+
+Before you can use goma to build Electron you need to authenticate against
+the Goma service.  You only need to do this once per-machine.
+
+```bash
+cd electron/external_binaries/goma
+goma_auth.py login
+```
+
+Once authenticated you need to make sure the goma daemon is running on your
+machine.
+
+```bash
+cd electron/external_binaries/goma
+goma_ctl.py ensure_start
+```
+
+## Building with Goma
+
+When you are using Goma you can run `ninja` with a substantially higher `j`
+value than would normally be supported by your machine.  Please do not set
+a value higher than **300**, we monitor the goma system and users found to
+be abusing it with unreasonable concurrency will be de-activated.
+
+```bash
+ninja -C out/Testing electron -j 200
+```
+
+## Monitoring Goma
+
+If you access [http://localhost:8088](http://localhost:8088) on your local
+machine you can monitor compile jobs as they flow through the goma system.
+
+## Access
+
+For security and cost reasons access to Electron Goma is currently restricted
+to Electron Maintainers.  If you want access please head to `#access-requests` in
+Slack and ping `@goma-squad` to ask for access.  Please be aware that being a
+maintainer does not *automatically* grant access and access is determined on a
+case by case basis.

script/external-binaries.json

@@ -43,6 +43,21 @@
       "url": "sccache-win32-x64.zip",
       "platform": "win32",
       "sha": "b6a20fd1c2026f3792e7286bc768a7ebc261847b76449b49f55455e1f841fecd"
+    },
+    {
+      "url": "goma-win.zip",
+      "platform": "win32",
+      "sha": "f97c88aa5d49395ae20387b6329ad406fd019f5fb4aac4ba639ca928b7151f6b"
+    },
+    {
+      "url": "goma-linux.tgz",
+      "platform": "linux",
+      "sha": "1cb3099a40f6200ae57216efa26af795c587b6ac7ae97955d1078d0b1e3011a6"
+    },
+    {
+      "url": "goma-mac.tgz",
+      "platform": "darwin",
+      "sha": "da1e7de82fbf3b99f1a9d0f9bf51b25e75e8778fec180deb72cff873813d717c"
     }
   ]
 }

script/update-external-binaries.py

@@ -5,6 +5,7 @@
 import hashlib
 import json
 import os
+import tarfile
 
 from lib.config import PLATFORM, get_target_arch
 from lib.util import add_exec_bit, download, extract_zip, rm_rf, \
@@ -51,8 +52,12 @@ def main():
 
     temp_path = download_binary(base_url, version, binary['url'], binary['sha'])
 
-    # We assume that all binaries are in zip archives.
-    extract_zip(temp_path, output_dir)
+    if temp_path.endswith('.zip'):
+      extract_zip(temp_path, output_dir)
+    else:
+      tar = tarfile.open(temp_path, "r:gz")
+      tar.extractall(output_dir)
+      tar.close()
 
     # Hack alert. Set exec bit for sccache binaries.
     # https://bugs.python.org/issue15795