Remove the use of deprecated jwt addon for TravisCI

[jdm]

From a recent TravisCI build:

JWT addon will be deprecated on April 17, 2018. Please read our announcement at https://blog.travis-ci.com/2018-01-23-jwt-addon-is-deprecated
addons_jwt
Initializing JWT

I don't know what it's used for, but it looks like we should stop using it.

[Hexcles]

I believe we use Travis' JWT (JSON Web Tokens) addon to store the secrets for Sauce Labs. See this commit 66a5cc8.

We need to find an alternative, as the official announcement says:

At the scheduled date of April 17, 2018, the addon will cease to function.

which means all the Sauce jobs (Edge & Safari) will no longer work after April 17.

The post also says:

Unfortunately, there is no replacement or workaround for the jwt addon at this time. We understand that your workflows (especially the ones involving Pull Requests) may be affected, but we ask that you phase out the use of this addon by the deadline for security reasons.

so... what now?

cc @foolip

[foolip]

I guess the setup with short-lived tokens made sense for Sauce because they don't trust their users' input anyway, so at worst someone could get some free access to Sauce. But, yeah, that's weird, I can see why it's going away.

I think we have to move to a model where the secret isn't available to Travis either, because it (1) won't work for PRs created from other repos and (2) the many people with write access to the repo would have access to the (long lived) secret.

Travis already treats everything as untrusted, so Chrome and Firefox will be fine, and there's no token for https://pulls.web-platform-tests.org/, that uses web hooks.

To get something like Sauce working again:

• Travis has no secrets, and just announces that a new PR exists (maybe a web hook)
• Some not-on-Travis controller reacts by:
  • Checking that the PR exists
  • Putting it in an untrusted sandbox to run the tests
  • Getting the results back, and making the results available somewhere

The last part would require a security token, which is why the previous step can't be trusted.

@lukebjerring @mdittmer @kereliuk, WDYT?

[foolip]

To resolve the immediate problem, we should just turn off the Sauce runs. They take up time, aren't blocking, and generally are too noisy to actually take seriously for investigating regressions. I will admit it was useful once, though:
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/12047069/

Don't know if it led to a bug that wasn't already fixed in Edge, but it might have.

I'll send a PR to get rid of the Sauce runs right away and ask @jgraham for review.

[foolip]

OK, #9954 worked. @jgraham, can you review that? I don't think we'll figure out how to get reliable results for PRs from Edge and Safari very soon, and I definitely like like having the runs is currently a net negative, the JWT deprecation notwithstanding.

[jgraham]

Right, the problem with Travis is that there's no way to keep infromation private in a way that ensures it can't be accessed by someone who can change the job configuration, which is everyone since it's fully defined in-tree. That's why only in-repo PRs get access to secure varaibles.

An alternative wouldn't involve travis at all; it would just learn about the PR from a webhook, do whatever, and then set a status on the commit. The problem with such a setup is merely that it's on closed infrastructure so it depends on ongoing maintainance by people with access to that infrastructure. That's already a problem we have with wpt.fyi and the PR dashboard, but it is a problem.

[foolip]

Setting up an alternative would indeed run into those kinds of problems and require people to be responsive to breakage. In an ideal state, the browser-neutral bits would be built in the open and maintained by the same stakeholder who effectively maintain wpt infra now, and any browser-specific bits that aren't just open infrastructure would have to be maintained by the browser teams, with some clear expectation or SLA.

[foolip]

@jdm, thanks for reporting! If you didn't we might have had a bad day on April 17 with all PRs blocked until we could figure it out...