taskcluster_guide.md

Taskcluster guide

This guide is designed to help you get started with Taskcluster, the Continuous Integration system we use, and to introduce you to some intermediate topics.

• Getting Started
• FAQ and HOWTOs

Getting Started

Are you totally new to Taskcluster or Taskgraph? Check out this 3-minute-video. If you want an overview of Taskgraph, you can read the blog post attached to it.

If you have any questions regarding Taskcluster or Taskgraph, feel free to join #releaseduty-mobile on Mozilla's Slack instance.

FAQ and HOWTOs

How do I test some changes I make under the taskcluster/ folder?

The easy way: create a PR and you will get results in minutes. If you want to test your changes locally, have a look at the README of Taskgraph.

How do I see test results?

• On PRs: the status badge will let you know whether it is green, running, or broken.
• On the master branch: Treeherder is a great tool to have an overview. Here are the links for:
  • fenix
  • android-components
  • reference-browser
• More generally: the status badge attached to each commit on Github will point you to each task run. For instance, here is a link to one of Fenix's release branch: https://github.com/mozilla-mobile/fenix/commits/releases/v5.0

Why is my PR taking much longer than usual? What is this toolchain task?

Major mobile projects now cache external dependencies in a Taskcluster artifact. We implemented this solution because some maven repositories couldn't keep up with the load we required from them. This also puts us one step closer to having reproducible builds. The cache gets rebuilt if any of these files change. Otherwise, build tasks just reuse the right cache: taskgraph is smart enough to know which existing one to use, we call this kind of tasks toolchain. Sadly, there is no way to ask gradle to just download dependencies - we have to compile everything. This is why such task can be really long. We will address the slowness of fenix in mozilla-mobile/fenix#10910.

How do I find the latest nightly?

If you are interested in just getting the latest APKs, use these links:

• fenix (choose the architecture you are interested in)
• android-components (choose the component you are interested in)
• reference-browser (choose the architecture you are interested in)

If you want to understand why something is wrong with nightly:

1. click on one of the aforementioned Treeherder links
2. type nightly in the search bar at the top right corner and press Enter => this will filter out any non-nightly job
3. check if there are any non-green jobs. If none are displayed, do not hesitate to load more results by clicking on one of the get next buttons.

How do I rerun jobs?

• On pull requests: just close and reopen the PR, the full task graph will be retriggered. (You cannot rerun individual tasks while bug 1641282 is not fixed)
• Otherwise: on the taskcluster UI, when looking at a task, go to the bottom right corner and select "Rerun". If it doesn't work, please contact the Release Engineering team (#releaseduty-mobile on Slack)

If you hesitate between reruns and retriggers, choose rerun first, then retrigger if the former failed.

How do I run a staging nightly/beta/release on PRs?

It is easy! Just create a new file called try_task_config.json at the root of the repository and it will change what graph taskgraph generates.

⚠️ Do not forget to remove this file before merging your patch.

Nightly

{
    "parameters": {
        "optimize_target_tasks": true,
        "target_tasks_method": "nightly"
    },
    "version": 2
}

will generate a nightly graph on your PR.

Beta

{
    "parameters": {
        "optimize_target_tasks": true,
        "release_type": "beta",
        "target_tasks_method": "release"
    },
    "version": 2
}

Release

{
    "parameters": {
        "optimize_target_tasks": true,
        "release_type": "release",
        "target_tasks_method": "release"
    },
    "version": 2
}

More generally

You can know what target_tasks_method to provide by looking at the @_target_task() sections in target_tasks.py. E.g: target_tasks.py in Fenix.

Why don't PRs open by external contributors get Tasckluster jobs?

Long story short: It's because of a limitation of the Taskcluster security model. More details in this Bugzilla bug. 2 workarounds: a. A person with write access on the repo opens a 2nd PRs with the same patch. This will trigger TC as usual. b. If the repository has bors enabled, first make sure the PR doesn't do anything suspicious, then comment bors try on the PR. Bors will run Taskcluster jobs on an integration branch.

How do I get secrets/tokens baked into an APK?

We use Taskcluster secrets to store secrets and tokens that will be injected into APKs. See this pull request as an example. Steps are:

1. Implement a similar code based on the PR mentioned just before.
2. Tell a member of the Release Engineering team you would like to add a new secret.
3. Securely send the secret by following one of these methods
4. For Releng:
   1. Go to https://firefox-ci-tc.services.mozilla.com/secrets/?search=fenix
   2. Edit existing secrets to include the new key and the new (unencrypted) value. Key is the first value of this tuple. Value is the secret it self. You usually want to edit the nightly, beta, and release secrets. nightly-simulation should be edited too but with dummy values.