Taskcluster can report a PR as complete if it is queued for merge while a try push is ongoing

[jdm]

I started a try push in #25309, waited until 2/3 of the try jobs were green, then self-r+ed it. As far as I can tell, as soon as the third job completed successfully, taskcluster decided that all of the jobs for the queued PR was complete and merged it while another PR was at the front of the queue.

[jdm]

Actually I think what happened is:

• the PR that was head of the queue hit an error
• taskcluster noticed that the try job had three complete tasks and was based on the same base revision as master
• ????
• merge

[SimonSapin]

There are moving pieces there interact with each other and operate somewhat independently:

• Homu makes merge commits, pushes them to branches like try or auto, gets notified of Statuses changes, and optionally pushes those same merge commits to master.
• Taskcluster-Github gets notified of pushes (such as those by Homu) and PRs, schedules task groups starting with a decision task, and creates/updates the GitHub Status based on the outcome of those task groups.
• Our decision task schedules other tasks based on in-tree config, and uses Taskcluster’s index to skip a given task when it was already successful for the same pair of parent commits (which for Homu’s merge commit means the same tree, but maybe we should use the git tree hash instead).

Note that Statuses are associated to a commit SHA, but not to a branch. So if an existing commit is also pushed to another branch, any existing Statuses is still there. This happens all the time when Homu pushes something to master after pushing it to auto, but at that point Homu is not looking at Statuses for that commit anymore. This should not happen when a try or try=something Homu command is followed by r+: at least the way I thought Homu works is that it would create a new merge commit at time r+ and push that to the auto branch.

In #25309 as far as I can tell the sequence of events was:

• jdm gave Homu the try=windows command
• Homu made merge commit 846aa5b (#25309 (comment)) and pushed it to the try-windows branch
• Taskcluster-GitHub was notified of that push and created task group https://community-tc.services.mozilla.com/tasks/groups/Mk2zjLM8SvuAPXpzTX1CCQ, starting with decision task https://community-tc.services.mozilla.com/tasks/Mk2zjLM8SvuAPXpzTX1CCQ. "GIT_REF": "refs/heads/try-windows" in the payload of the decision task and assume:repo:github.com/servo/servo:branch:try-windows in its scopes confirm the branch name. "GIT_SHA": "846aa5b56ac140c78de65e7223217c4c45550a89" and “Source fb5ab1f...846aa5b ” confirm the merge commit hash.
• At 20:52 UTC, jdm gave Homu the r+ command. At that point I would have expected Homu to forget about 846aa5b, make a new merge commit (with a different SHA because of the different commit message with r=jdm instead of r=<try> and the different date) and push that to auto. As far as I can tell this did not happen at all.
• At 21:01 UTC, the task group resolved successfully and Taskcluster-GitHub made the corresponding Status update which can be found with curl https://api.github.com/repos/servo/servo/commits/846aa5b56ac140c78de65e7223217c4c45550a89/statuses | jq '.[] |select(.created_at == "2019-12-16T21:01:30Z")'
• Homu reacted to that by making a "Test successful" comment #25309 (comment) and pushing 846aa5b to master.

As far as I can tell Homu is buggy and everything else behaved as expected here.