Automerge action #10069

harupy · 2023-10-23T07:41:40Z

Summary

GitHub's auto-merge feature only checks required jobs. In this repo, we have jobs that are conditionally triggered. Those conditional jobs can't be marked as required (if we marked them as required, they would block the PR forever) but should block the PR. We need our own auto-merge mechanism that has the following features:

Runs every 10 minutes.
Searched for pull requests labeled with automerge
Merge the PR if all the CI checks have passed (both required and non-required ones).

Example code

ChatGPT wrote this code. I'm not sure if this really works. We need to test in a different repository:

name: Automerge PRs

on:
  schedule:
    - cron: '*/10 * * * *'  # Run every 10 minutes

jobs:
  merge:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Automerge PRs with "automerge" label created within the last month
      uses: actions/github-script@v5
      with:
        github-token: ${{secrets.GITHUB_TOKEN}}
        script: |
          const { repo: { owner, repo } } = context;
          
          // Get date from a month ago in ISO format
          const oneMonthAgo = new Date();
          oneMonthAgo.setMonth(oneMonthAgo.getMonth() - 1);
          const sinceDate = oneMonthAgo.toISOString();

          // List PRs with the "automerge" label created within the last month
          const { data: issues } = await github.issues.listForRepo({
            owner,
            repo,
            labels: 'automerge',
            since: sinceDate
          });

          // Filter for pull requests from the list of issues
          const pullRequests = issues.filter(issue => issue.pull_request);

          for (const pr of pullRequests) {
            // Fetch PR details
            const { data: pullRequest } = await github.pulls.get({
              owner,
              repo,
              pull_number: pr.number
            });

            // Check the status of all checks for the PR
            const { data: checkRuns } = await github.checks.listForRef({
              owner,
              repo,
              ref: pullRequest.head.sha
            });

            const allChecksPassed = checkRuns.check_runs.every(run => run.conclusion === 'success');

            if (allChecksPassed) {
              await github.pulls.merge({
                owner,
                repo,
                pull_number: pr.number,
                commit_title: `Merge pull request #${pr.number}`,
                commit_message: `automatic merge of PR #${pr.number}`
              });
              console.log(`Merged PR #${pr.number}`);
            } else {
              console.log(`Checks not ready for PR #${pr.number}. Skipping merge.`);
            }
          }

Notes

Make sure to open a PR from a non-master branch.

Sign off the commit using the -s flag when making a commit:

git commit -s -m "..."
         # ^^ make sure to use this

Include #{issue_number} (e.g. #123) in the PR description when opening a PR.

The text was updated successfully, but these errors were encountered:

harupy · 2023-10-23T08:34:15Z

More robust code:

name: Automerge PRs

on:
  schedule:
    - cron: '*/10 * * * *'  # Run every 10 minutes

jobs:
  merge:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Automerge PRs with "automerge" label created within the last month and are mergeable
      uses: actions/github-script@v5
      with:
        github-token: ${{secrets.GITHUB_TOKEN}}
        script: |
          const { repo: { owner, repo } } = context;

          const MAX_RETRIES = 3;
          const RETRY_INTERVAL_MS = 10000;  // 10 seconds

          // Get date from a month ago in ISO format
          const oneMonthAgo = new Date();
          oneMonthAgo.setMonth(oneMonthAgo.getMonth() - 1);
          const sinceDate = oneMonthAgo.toISOString();

          // List PRs with the "automerge" label created within the last month
          const { data: issues } = await github.issues.listForRepo({
            owner,
            repo,
            labels: 'automerge',
            since: sinceDate
          });

          // Filter for pull requests from the list of issues
          const pullRequests = issues.filter(issue => issue.pull_request);

          for (const pr of pullRequests) {
            let pullRequest;

            for (let i = 0; i < MAX_RETRIES; i++) {
              // Fetch PR details
              pullRequest = await github.pulls.get({
                owner,
                repo,
                pull_number: pr.number
              }).then(res => res.data);

              if (pullRequest.mergeable !== null) {
                break;
              }

              console.log(`Waiting for mergeability calculation for PR #${pr.number}...`);
              await new Promise(resolve => setTimeout(resolve, RETRY_INTERVAL_MS));
            }

            // If mergeable is false or still null after retries, skip this PR
            if (pullRequest.mergeable === null || pullRequest.mergeable === false) {
              console.log(`PR #${pr.number} is not mergeable. Skipping this PR.`);
              continue;
            }

            // Check the status of all checks for the PR
            const { data: checkRuns } = await github.checks.listForRef({
              owner,
              repo,
              ref: pullRequest.head.sha
            });

            const allChecksPassed = checkRuns.check_runs.every(run => run.conclusion === 'success');

            if (allChecksPassed) {
              try {
                await github.pulls.merge({
                  owner,
                  repo,
                  pull_number: pr.number,
                  commit_title: `Merge pull request #${pr.number}`,
                  commit_message: `automatic merge of PR #${pr.number}`
                });
                console.log(`Merged PR #${pr.number}`);
              } catch (error) {
                console.log(`Failed to merge PR #${pr.number}. Reason: ${error.message}`);
              }
            } else {
              console.log(`Checks not ready or PR not approved for PR #${pr.number}. Skipping merge.`);
            }
          }

harupy · 2023-10-23T09:14:30Z

@TomeHirata Would you be interested in working on this?

TomeHirata · 2023-10-23T09:56:58Z

@harupy Yes, please let me work on this. I have several questions I'd like to clarify

What is the condition when automerge label is attached? Is it a manual process?
Can I understand that we will attach the label to PR rather than issue? I asked since the pseudo-code tries to filter issues instead of PRs
Can't we have a job that is triggered when the label is attached is added instead of having a scheduled job? It may be able to save the computational resources (ref)

harupy · 2023-10-23T10:07:25Z

What is the condition when automerge label is attached? Is it a manual process?

Yes, it's manual. If your PR is approved and all you need to do is wait, then you can apply this label.

Can I understand that we will attach the label to PR rather than issue? I asked since the pseudo-code tries to filter issues instead of PRs

const pullRequests = issues.filter(issue => issue.pull_request); filters out issues.

Can't we have a job that is triggered when the label is attached instead of having a scheduled job? It may be able to save the computational resources (ref)

Good question.

You might still have running jobs when you apply the label.
What if you push a commit after you apply the label?

We can increase the interval (e.g. 30 minutes or 1 hour) if necessary.

TomeHirata · 2023-10-23T12:14:37Z

I understand, thank you for the clarification. I also learned that github.issues API can also get pull requests (ref).
Let me work on this task.

harupy · 2023-10-23T12:16:39Z

@TomeHirata Thanks! Can you create a repo (that's okay to be broken) for testing?

harupy · 2023-10-23T12:20:13Z

We can create the action above + one more job that protects master/main branch, and then file a PR with automerge label applied.

harupy · 2023-10-23T12:36:18Z

Asked ChatGPT to make a few updates:

name: Automerge PRs

on:
  schedule:
    - cron: '*/10 * * * *'  # Run every 10 minutes

jobs:
  merge:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Automerge PRs with "automerge" label created within the last month and are mergeable
      uses: actions/github-script@v5
      with:
        github-token: ${{secrets.GITHUB_TOKEN}}
        script: |
          const { repo: { owner, repo } } = context;

          const MAX_RETRIES = 3;
          const RETRY_INTERVAL_MS = 10000;  // 10 seconds
          const MERGE_INTERVAL_MS = 5000;   // 5 seconds pause after a merge

          async function sleep(ms) {
            return new Promise(resolve => setTimeout(resolve, ms));
          }

          async function logRateLimit() {
            const { data: rateLimit } = await github.rateLimit.get();
            console.log(`Rate limit remaining: ${rateLimit.rate.remaining}`);
            console.log(`Rate limit resets at: ${new Date(rateLimit.rate.reset * 1000).toISOString()}`);
          }

          async function fetchPullRequestDetails(prNumber) {
            for (let i = 0; i < MAX_RETRIES; i++) {
              const pullRequest = await github.pulls.get({
                owner,
                repo,
                pull_number: prNumber
              }).then(res => res.data);

              if (pullRequest.mergeable !== null) {
                return pullRequest;
              }

              console.log(`Waiting for mergeability calculation for PR #${prNumber}...`);
              await sleep(RETRY_INTERVAL_MS);
            }
            return null;
          }

          async function isPRApproved(prNumber) {
            const { data: reviews } = await github.pulls.listReviews({
              owner,
              repo,
              pull_number: prNumber
            });
            return reviews.some(review => review.state === 'APPROVED');
          }

          async function areAllChecksPassed(sha) {
            const { data: checkRuns } = await github.checks.listForRef({
              owner,
              repo,
              ref: sha
            });
            return checkRuns.check_runs.every(run => run.conclusion === 'success');
          }

          // Get date from a month ago in ISO format
          const oneMonthAgo = new Date();
          oneMonthAgo.setMonth(oneMonthAgo.getMonth() - 1);
          const sinceDate = oneMonthAgo.toISOString();

          // List PRs with the "automerge" label created within the last month
          const { data: issues } = await github.issues.listForRepo({
            owner,
            repo,
            labels: 'automerge',
            since: sinceDate
          });

          // Filter for pull requests from the list of issues
          const pullRequests = issues.filter(issue => issue.pull_request);

          for (const pr of pullRequests) {
            const pullRequest = await fetchPullRequestDetails(pr.number);

            if (!pullRequest || pullRequest.mergeable !== true) {
              console.log(`PR #${pr.number} is not mergeable or could not fetch details. Skipping this PR.`);
              await logRateLimit();
              continue;
            }

            if (!await isPRApproved(pr.number)) {
              console.log(`PR #${pr.number} hasn't been approved. Skipping merge.`);
              await logRateLimit();
              continue;
            }

            if (await areAllChecksPassed(pullRequest.head.sha)) {
              try {
                await github.pulls.merge({
                  owner,
                  repo,
                  pull_number: pr.number
                });
                console.log(`Merged PR #${pr.number}`);

                await sleep(MERGE_INTERVAL_MS);
                await logRateLimit();
              } catch (error) {
                console.log(`Failed to merge PR #${pr.number}. Reason: ${error.message}`);
              }
            } else {
              console.log(`Checks not ready for PR #${pr.number}. Skipping merge.`);
              await logRateLimit();
            }
          }

TomeHirata · 2023-10-24T09:21:58Z

@harupy

Can you create a repo (that's okay to be broken) for testing?

Sure, will create a small test repo. Could you assign this ticket to me so that it's easy to track?
Also, it looks like that github-action supports a retry functionality, so we can simplify the code by utilizing it. Let me try it and ask for a review.
https://github.com/actions/github-script#retries

harupy · 2023-10-24T09:38:49Z

@TomeHirata Assigned!

harupy · 2023-10-24T09:41:25Z

a retry functionality, so we can simplify the code by utilizing it

I didn't know actions/github-script supports this. We can use it if necessary.

github-actions · 2023-10-31T00:12:11Z

@mlflow/mlflow-team Please assign a maintainer and start triaging this issue.

harupy added the good first issue Good for newcomers label Oct 23, 2023

harupy assigned TomeHirata Oct 24, 2023

TomeHirata mentioned this issue Oct 29, 2023

Add a guard job to prevent accidental auto-merging of PRs when cross-version tests fail #10210

Merged

37 tasks

github-actions bot added the has-closing-pr This issue has a closing PR label Oct 29, 2023

harupy closed this as completed in #10210 Nov 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automerge action #10069

Automerge action #10069

harupy commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023

TomeHirata commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023 •

edited

Loading

TomeHirata commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023

harupy commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023 •

edited

Loading

TomeHirata commented Oct 24, 2023 •

edited

Loading

harupy commented Oct 24, 2023

harupy commented Oct 24, 2023

github-actions bot commented Oct 31, 2023

Automerge action #10069

Automerge action #10069

Comments

harupy commented Oct 23, 2023 • edited Loading

Summary

Example code

Notes

harupy commented Oct 23, 2023 • edited Loading

harupy commented Oct 23, 2023

TomeHirata commented Oct 23, 2023 • edited Loading

harupy commented Oct 23, 2023 • edited Loading

TomeHirata commented Oct 23, 2023 • edited Loading

harupy commented Oct 23, 2023

harupy commented Oct 23, 2023 • edited Loading

harupy commented Oct 23, 2023 • edited Loading

TomeHirata commented Oct 24, 2023 • edited Loading

harupy commented Oct 24, 2023

harupy commented Oct 24, 2023

github-actions bot commented Oct 31, 2023

harupy commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023 •

edited

Loading

TomeHirata commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023 •

edited

Loading

TomeHirata commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023 •

edited

Loading

harupy commented Oct 23, 2023 •

edited

Loading

TomeHirata commented Oct 24, 2023 •

edited

Loading