[Bug] Non manifest PRs are not merging

[jcscottiii]

What used to happen (using an old PR as an example):

• A single auto merge workflow would be triggered upon pull_request open event.
• It would detect that the tests were still running and wait.
• Once the tests finished and the retry poll came back around, it would merge it. (It would not wait for an approval)

The problem now (using a new PR as an example):

• A single auto merge workflow would be triggered upon pull_request open event.
• It detects that there is a minimum number of approvals needed and just stops
  • Why did it stop working? In order to support GitHub checks not running update_wpt_manifest.yml #1444, I turned on the GitHub branch requirement that at least one person needed to approve. This was to use GitHub's native auto-merge feature for the manifest PRs. (Check GitHub checks not running update_wpt_manifest.yml #1444 and Fix the new wptmetadata-bot for manifest updates #4115 for background as to why we moved to the mode of least privileged and how that affected the permissions for running pipelines) However, this conflicts with the existing auto merge feature for Automatically Triage New Metadata/Gecko sync update PRs. That is, if the workflow detects that the a review is required, it determines it is a failure and will not do a wait to check again. (which is what we see in the logs above.)

Solutions considered:

1. Leverage pull_request_review event to retrigger the automerge workflow.
   Conclusion: This is already configured but not working. This is because any event from github action bot cannot trigger anything. This is to prevent recursive github actions (Read more about GITHUB_TOKEN in these docs). I think this changed in the recent years.
   Data to show that github action bot approval doesn't trigger a pull_request_event:
   • An old PR approved by only github-actions only shows the automerge check was ran for the pull_request event.
   • An example PR where the PR did not merge. Upon approving it myself, a second automerge job ran.
     • First auto merge job with the pull_request event (logs)
     • Second auto merge job with the pull_request_review event. This was triggered by my approval and not from the github actions approval (logs)
2. Replace all triggers on the automerge workflow for an on "workflow_run" trigger. This workflow run will only trigger after the "Auto Approve".
   Preliminary conclusion: This looks like it should work and is reasonable to expect that automerge workflow should only happen after the auto approval (which happens after the successful test result)

Previous thoughts

I initially thought that this was due to a timeout issue due to the test action taking too long. I was wrong.

[jcscottiii]

Going to implement solution 2.