Bug 1876573 - Implement new script to interface with Bitrise

Bitrisescript will be used to interface with Bitrise's API. In this
initial version, only the ability to trigger Bitrise workflows will be
added, though more capabilities may be added in the future.

Usage of bitrisescript will require at least two scopes:

- <prefix>:app:<app>
- <prefix>:workflow:<workflow>

The first will ensure the task has permission to trigger workflows for
the specific app in Bitrise. The second will ensure the task has
permission to trigger a specific workflow.

Tasks may specify multiple workflow scopes, which will trigger each
workflow asynchronously.

.dirschema.yml

@@ -8,6 +8,7 @@ docker-compose:
     - /addonscript
     - /balrogscript
     - /beetmoverscript
+    - /bitrisescript
     - /bouncerscript
     - /pushapkscript
     - /pushmsixscript
@@ -20,6 +21,7 @@ apps:
     - /addonscript
     - /balrogscript
     - /beetmoverscript
+    - /bitrisescript
     - /bouncerscript
     - /pushapkscript
     - /pushmsixscript

README.rst

@@ -80,6 +80,17 @@ beetmoverscript
 | mobile-1-beetmover      | beetmover-prod-relengworker-firefoxci-mobile-1              |
 +-------------------------+-------------------------------------------------------------+
 
+bitrisescript
+-------------
+
++-------------------------+-------------------------------------------------------------+
+| Worker type             | Deployment name                                             |
++=========================+=============================================================+
+| mobile-3-bitrise        | bitrise-prod-relengworker-firefoxci-mobile-3                |
++-------------------------+-------------------------------------------------------------+
+| mobile-1-bitrise        | bitrise-prod-relengworker-firefoxci-mobile-1                |
++-------------------------+-------------------------------------------------------------+
+
 bouncerscript
 -------------
 

bitrisescript/Dockerfile

@@ -0,0 +1,33 @@
+FROM python:3.9.7
+
+RUN groupadd --gid 10001 app && \
+    useradd -g app --uid 10001 --shell /usr/sbin/nologin --create-home --home-dir /app app
+
+COPY ["bitrisescript", "/app/bitrisescript/"]
+COPY ["scriptworker_client", "/app/scriptworker_client/"]
+COPY ["configloader", "/app/configloader/"]
+COPY ["docker.d", "/app/docker.d/"]
+
+# Copy the version.json file if it's present
+COPY ["version.jso[n]", "/app/"]
+
+RUN chown -R app:app /app && \
+    ln -s /app/docker.d/healthcheck /bin/healthcheck
+
+WORKDIR /app
+USER app
+
+RUN python -m venv /app \
+ && cd /app/scriptworker_client \
+ && /app/bin/pip install -r requirements/base.txt \
+ && /app/bin/pip install . \
+ && cd /app/bitrisescript \
+ && /app/bin/pip install -r requirements/base.txt \
+ && /app/bin/pip install . \
+ && python -m venv /app/configloader_venv \
+ && cd /app/configloader \
+ && /app/configloader_venv/bin/pip install -r requirements/base.txt \
+ && /app/configloader_venv/bin/pip install . \
+ && cd /app
+
+CMD ["/app/docker.d/init.sh"]

bitrisescript/Dockerfile.test

@@ -0,0 +1,15 @@
+ARG PYTHON_VERSION
+
+FROM python:${PYTHON_VERSION}
+
+WORKDIR /app
+
+COPY MANIFEST.in setup.py tox.ini /app/
+COPY requirements/ /app/requirements/
+
+ARG PYTHON_REQ_SUFFIX
+RUN pip install -r requirements/local${PYTHON_REQ_SUFFIX}.txt
+
+COPY src/ /app/src/
+
+ENTRYPOINT ["/usr/local/bin/tox", "-e"]

bitrisescript/MANIFEST.in

@@ -0,0 +1,16 @@
+include LICENSE
+include README.md
+include pyproject.toml
+include setup.py
+include version.txt
+
+recursive-include requirements *
+recursive-include src *
+
+exclude examples
+exclude requests
+
+recursive-exclude * __pycache__
+recursive-exclude * *.py[co]
+recursive-exclude examples *
+recursive-exclude tests *

bitrisescript/README.md

@@ -0,0 +1,86 @@
+# bitrisescript
+
+Script to handle triggering [Bitrise]() workflows and pipelines from
+Taskcluster. It's is aimed to be run with
+[scriptworker](https://github.com/mozilla-releng/scriptworker) (but runs
+perfectly fine as a standalone script).
+
+
+## Get the code
+
+
+First, you need `python>=3.9`.
+
+```sh
+# create the virtualenv in ./venv3
+virtualenv3 venv3
+# activate it
+. venv3/bin/activate
+git clone https://github.com/mozilla-releng/scriptworker-scripts
+cd scriptworker-scripts/bitrisescript
+python setup.py develop
+```
+
+### Configure
+
+#### config.json
+```sh
+cp examples/config.example.json config.json
+# edit it with your favorite text editor
+```
+
+There are many values to edit. Example values should give you a hint about what
+to provide. If not, please see [signingscript's
+README](https://github.com/mozilla-releng/scriptworker-scripts/tree/master/signingscript#config-json)
+for more details about allowing URLs, or contact the author for other unclear areas.
+
+#### Directories and File Naming
+
+If you aren't running through scriptworker, you need to manually create the
+directories that `work_dir` points to.  It's better to use new directories for
+these rather than cluttering and potentially overwriting an existing directory.
+Once you set up scriptworker, `work_dir` will be regularly wiped and recreated.
+
+
+### task.json
+
+```sh
+cp examples/task.example.json /path/to/work_dir
+# edit it with your favorite text editor
+```
+
+Ordinarily, scriptworker would get the task definition from TaskCluster, and
+write it to a `task.json` in the `work_dir`.  Since you're initially not going
+to run through scriptworker, you need to put this file on disk yourself.
+
+The important entries to edit are in the scopes:
+
+ * `project:releng:bitrise:app:*`, tells which Bitrise project should be
+   targeted.
+ * `project:releng:bitrise:workflow:*`, tells which Bitrise workflows should be
+   run.
+ * `project:releng:bitrise:pipeline:*`, tells which Bitrise pipelines should be
+   run.
+
+### Run
+
+You're ready to run bitrisescript!
+
+```sh
+bitrisescript CONFIG_FILE
+```
+
+Where `CONFIG_FILE` is the config json you created above.
+
+This should download the file(s) specified in the payload and trigger the
+specified Bitrise workflows and pipelines.
+
+### Running through Scriptworker
+
+Follow the [scriptworker
+readme](https://github.com/mozilla-releng/scriptworker/blob/master/README.rst)
+to set up scriptworker, and use `["path/to/bitrisescript",
+"path/to/script_config.json"]` as your `task_script`.
+
+:warning: Make sure your `work_dir` points to the same directories between the
+scriptworker config and the bitrisescript config!

bitrisescript/docker-compose.yml

@@ -0,0 +1,12 @@
+version: '3'
+services:
+  cli:
+    build: '.'
+    tty: true
+    command: '/bin/bash'
+    volumes:
+      - './:/src'
+    working_dir: '/src'
+    environment:
+      - PATH=/app/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+

bitrisescript/docker.d/init_worker.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+set -o errexit -o pipefail
+
+test_var_set() {
+  local varname=$1
+
+  if [[ -z "${!varname}" ]]; then
+    echo "error: ${varname} is not set"
+    exit 1
+  fi
+}
+
+case $COT_PRODUCT in
+  mobile)
+    case $ENV in
+      dev|fake-prod)
+        test_var_set 'BITRISE_ACCESS_TOKEN_STAGING'
+        ;;
+      prod)
+        test_var_set 'BITRISE_ACCESS_TOKEN_PROD'
+        ;;
+      *)
+        exit 1
+        ;;
+    esac
+    ;;
+  *)
+    exit 1
+    ;;
+esac

bitrisescript/docker.d/worker.yml

@@ -0,0 +1,11 @@
+work_dir: { "$eval": "WORK_DIR" }
+verbose: { "$eval": "VERBOSE == 'true'" }
+bitrise:
+  $switch:
+    'COT_PRODUCT == "mobile" && ENV == "prod"':
+      access_token: { "$eval": "BITRISE_ACCESS_TOKEN_PROD" }
+    $default:
+      access_token: { "$eval": "BITRISE_ACCESS_TOKEN_STAGING" }
+
+taskcluster_scope_prefix: "project:${COT_PRODUCT}:bitrise:"
+trust_domain: "${COT_PRODUCT}"

bitrisescript/examples/config.example.json

@@ -0,0 +1,9 @@
+{
+    "work_dir": "/tmp/bitrisescript/work_dir",
+    "bitrise": {
+        "access_token": "<token>"
+    },
+    "taskcluster_scope_prefix": ["project:mobile:firefox-ios:bitrise:"],
+
+    "verbose": true
+}

bitrisescript/examples/task.example.json

@@ -0,0 +1,47 @@
+{
+  "provisionerId": "scriptworker-k8s",
+  "workerType": "mobile-1-bitrise",
+  "schedulerId": "mobile-level-1",
+  "taskGroupId": "DIP_TN0kRQyfvbflprgXYw",
+  "dependencies": [
+    "OyHu5OYGTFW05iGchdqdSA"
+  ],
+  "requires": "all-completed",
+  "routes": [
+    "tc-treeherder.v2.fenix.b94cfdf06be2be4b5a3c83ab4095eb2ecde7ba71.0",
+    "checks"
+  ],
+  "priority": "highest",
+  "retries": 5,
+  "created": "2020-06-16T22:32:06.776Z",
+  "deadline": "2020-06-17T22:32:06.776Z",
+  "expires": "2021-06-16T22:32:06.776Z",
+  "scopes": [
+    "project:mobile:firefox-ios:bitrise:app:firefox-ios",
+    "project:mobile:firefox-ios:bitrise:workflow:build-and-test"
+  ],
+  "payload": {
+    "build_params": {
+      "branch": "main"
+    }
+  },
+  "metadata": {
+    "owner": "user@example.com",
+    "source": "https://github.com/mozilla-mobile/firefox-ios/blob/b94cfdf06be2be4b5a3c83ab4095eb2ecde7ba71/taskcluster/ci/bitrise",
+    "description": "Trigger Bitrise build-and-test workflow",
+    "name": "bitrise-build-and-test"
+  },
+  "tags": {
+    "os": "scriptworker",
+    "createdForUser": "user@example.com",
+    "worker-implementation": "scriptworker",
+    "kind": "bitrise",
+    "label": "bitrise-build-and-test"
+  },
+  "extra": {
+    "index": {
+      "rank": 1592346625
+    },
+    "parent": "DIP_TN0kRQyfvbflprgXYw"
+  }
+}

bitrisescript/requirements/base.in

@@ -0,0 +1,3 @@
+aiohttp
+aiohttp_retry >= 2.3
+scriptworker