Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
We've been building wpt.live (previously web-platform-tests.live) as a replacement for w3c-test.org. The latter provides two distinct services:
The first is relatively straightforward, and we completed it without any integration with the WPT repository. The second is more complex, and this issue is intended to facilitate discussion on its implementation.
Our initial implementation is available under the Bocoup GitHub organization and outlined in this presentation. In brief: it's designed to poll the WPT git repository for special refs and create git working trees based on that information. This allows it to function without secrets, trusted access to the git repository, or any configuration in WPT's GitHub project.
We originally planned to create the required git refs from the GitHub "Actions" that ran in response to each pull request. Unfortunately, the feature was subsequently changed in an incompatible way (and we subsequently removed the infrastructure from WPT). We're interested in re-implementing it, and here are some alternatives we're considering:
The first approach involves granting write permission to an external service, explicit configuration with the WPT GitHub project, and maintaining a secret in an open source project.
The second approach creates noise in the GitHub Actions UI (through many no-op executions), is susceptible to API rate limiting, and will likely result in a less responsive user experience.
Suggestions for other approaches are welcome, although we probably should favor solutions that operate via git refs (since we've already built and deployed a system that works with them).
https://help.github.com/en/articles/workflow-syntax-for-github-actions#usage-limits says these are the current usage limits, subject to change of course:
At best we'd probably use 2 API requests per invocation of a cron action, one to clone the repo and one to query the 100 most recently updated PRs.
Exceeding the limits would break manifest and documentation workflows and be quite bad, but I think the limits are high enough that we can poll every 5 minutes or so.