Skip to content

Latest commit

 

History

History
198 lines (141 loc) · 10.1 KB

check-enforcer.md

File metadata and controls

198 lines (141 loc) · 10.1 KB
Raw

Check Enforcer

Table of Contents

Why did we create Check Enforcer?

The Azure SDK team maintains reusable libraries that developers use to access Azure services. These libraries are grouped together into a repository for each language/runtime. For example there are repositories for Java, .NET, Python and JavaScript - just to name a few.

Each repository contains a large number of separate libraries. Even though together these libraries constitute a single SDK, they ship separately on their own individual cadence as the underlying service evolves. As a result, we have separate build and release pipelines for say the KeyVault and the Event Hubs libraries in each repository.

Whilst Checks in GitHub are awesome, one of the limitations when setting up required checks is that you cannot make them required for just one specific path. We don't want to build all libraries for every checkin (that would take a long time and needlessly block teams if other libraries were having build reliability issues) - so we needed a way to work around it.

Check Enforcer is our solution. We use the built-in triggering w/ path filter options within Azure Pipelines (Check Enforcer is CI tool agnostic however) to control when a pipeline triggers, and we just use Check Enforcer to block until all triggered pipelines pass successfully. Each of those libraries can be optional - and you just make Check Enforcer the only required check in the repo.

Enabling Check Enforcer for a Repository

  1. Copy the following workflow files into your repo and check them into the main branch:
    • ../example-workflow.yaml (link)
  2. Create a pull request so that one of the above actions will run and post a status.
  3. Add a new branch protection rule in your repository settings: https://github.com/<org>/<repo>/settings/branches
    • NOTE: The status must be posted at least once for github to show it as a branch protection rule to select, see the above step.
    • Enter the default branch or desired branch name pattern.
    • Check Require status checks to pass before merging, search for the status check https://aka.ms/azsdk/checkenforcer and select it.
    • Save changes

Usage

Check Enforcer runs within a github actions context and is triggered by two types of events: check_suite completed and issue_comment created.

  • check_suite completed behavior: When a pull request is created, github will show a pending status check for check enforcer based on the branch protection rule configured for the default branch (main). A check_suite is the github representation of all check_runs (e.g. pipeline jobs) associated with the head commit of the pull request branch. When all registered check_runs are completed, a check_suite completed event is triggered. The check enforcer github action will run at this time, evaluate the state of the check_suite and POST the corresponding state to the check enforcer statuses API endpoint for the pull request.
  • issue_comment created behavior: When a comment is added to the pull request, check enforcer will check if that comment is a supported command. If so, it will perform the corresponding behavior (reset, evaluate or override).

NOTE: Currently, check enforcer will only handle events for check suites generated by the Azure Pipelines github app.

Onboarding a New Service

Often, new services do not have validation pipelines associated with them, in order to bootstrap pipelines for a new service, you can issue the following command as a pull request comment:

/azp run prepare-pipelines

This will run a pipeline that analyzes the source tree and creates the pipelines necessary to build and validate your pull request. Once the pipeline has been created you can trigger the pipeline using the following comment:

/azp run [lang] - [service] - ci

For more detailed information about service onboarding see pipeline setup docs (internal).

PR Comment Commands

Check Enforcer supports a limited number of commands which can by issued via PR comments.

For available commands and a link to this doc:

/check-enforcer help

If Check Enforcer appears to be stuck you can add a comment as follows to re-evaluate the pull request checks:

/check-enforcer evaluate

From time to time, Check Enforcer may be blocking a merge because no-check runs are appropriate for the PR. In these cases, you can use the following command Check Enforcer rules and park the commit as successful:

/check-enforcer override

These are the only commands that Check Enforcer supports at this time.

Need Help?

Check Enforcer is built primarily for use by the Azure SDK Engineering Systems teams for use within their mono-repositories. But we are happy for others to pick it up and start using it. If you have any issues feel free to log an issue on this GitHub repository and we'll do our best to help you out.

Contributing

Got an idea for Check Enforcer? Great - a good way to start contributing is by creating an issue and discussing with us what you want to do. We are always happy to review unsolicited pull requests and if they match our goals for Check Enforcer we'll work with you to get it merged - but its probably better if you give us a heads up on what you want to achieve first.

Testing and Debugging

Build and Run

Dependencies

Build the code

# Install dependencies
go mod tidy

# Build code
go build .

Run code

The main program takes a filepath argument that points to a github webhook event payload body. Example payloads can be found in the github docs or mock payloads in the ./testpayloads directory (most examples in there are pulled from the github docs page).

go run . <path to payload>

Run unit tests

go test .

Simulate actions locally

To simulate how the program will run as a github action from a repository, you can use act to simulate github actions.

Move to the directory containing github workflows that reference this action:

cd <path to repository with .github/workflows/event.yaml resembling example-workflow.yaml>

Update the workflow file to point to the version of the code you are testing and push your changes:

jobs:
  event-handler:
    permissions:
      statuses: write      # to set status (azure/azure-sdk-actions)
      pull-requests: write # to read pull requests and write comments (azure/azure-sdk-actions)
      checks: read         # to read check status (azure/azure-sdk-actions)
    name: Handle ${{ github.event_name }} ${{ github.event.action }} event
    runs-on: ubuntu-latest
    steps:
      - uses: <fork>/azure-sdk-actions@<dev branch>
        with:
          token: ${{ secrets.GITHUB_TOKEN }}

Set github personal access token env variable if you want to test updates. See Creating a personal access token

GITHUB_TOKEN="<pat>"

Run act. You need to pass a payload and a matching event trigger (e.g. issue_comment, check_suite, pull_request_target):

# Override the base ubuntu-latest image with the golang image. The runner VMs in github actions already have golang installed.
act -s GITHUB_TOKEN=$GITHUB_TOKEN -P ubuntu-latest=golang -e <actions repo>/testpayloads/issue_comment_event.json issue_comment

Test in the repository

  1. Set up two forked repositories, one for your changes to this action, and one for testing the github workflows.
    • For example, myusername/azure-sdk-actions and myusername/azure-sdk-test-repo
  2. Update the workflow file in the relevant repository to point to the version of the code you are testing and push your changes: 
    jobs:
  event-handler:
    permissions:
      statuses: write      # to set status (azure/azure-sdk-actions)
      pull-requests: write # to read pull requests and write comments (azure/azure-sdk-actions)
      checks: read         # to read check status (azure/azure-sdk-actions)
    name: Handle ${{ github.event_name }} ${{ github.event.action }} event
    runs-on: ubuntu-latest
    steps:
      - uses: <fork>/azure-sdk-actions@<dev branch>
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
  3. Check in the above github workflow changes to the fork with the workflow file, and push them to the main branch (github actions trigger only through main branch workflow or action files).
  4. Create a dummy PR against the above fork with the main branch changes. From there you can do things like comment on the PR to trigger events and test changes you have made to your fork of the actions repo.