What criteria would be helpful in making decisions about new contributors?

[ehmatthes]

Purpose

This project aims to provide some quick context to open source maintainers when evaluating issues and PRs from new contributors, especially when it appears that the contribution is largely AI-generated. This is an attempt to help quickly surface the kind of information I've been clicking around GitHub to find when deciding how much time to invest interacting with a new contributor. It's been helpful to me so far, and seems like it's helpful to others as well.

I have no intention of ever using the information that's surfaced to make an ultimate recommendation to engage with the contribution or close it immediately. Rather, I want to give maintainers a quick overview of the kind of information that indicates a user is likely to be a well-intentioned human, a human wielding AI tools a bit too loosely, or a bot that you shouldn't spend any time engaging with.

Current output

Here's two examples of current output. First, running it against my own account:

$ uvx gh-profiler ehmatthes
GitHub user: ehmatthes
  🟢 Account age: 5059 days

  🟢 Profile information:
      name: Eric Matthes
      company:
      blog: https://www.mostlypython.com
      location: western North Carolina
      email: ehmatthes@gmail.com
      bio:

  🟢 ehmatthes has opened fewer than 10 PRs in the last 21 days.

I show green flags because I have a well-established account, I have shared some profile information, and I haven't opened an overwhelming volume of PRs recently.

Here's the output from a contribution I ended up rejecting recently:

$ uvx gh-profiler 874
GitHub user: <redacted>
  🟢 Account age: 1085 days

  🔴 No profile information has been provided.

  🟢 42 of 69 PRs have been merged in the last 21 days.
  🟡 20 of 69 PRs have been closed without merging in the last 21 days.

This user had a well-established account. However, they've shared no profile information at all. They've also produced ~70 PRs over the last three weeks, 20 of which have been closed without merging. I did interact with this user, and closed their PR because they had no personal understanding of the project they opened the PR on. Their AI-generated contribution needed work that involved developing an understanding of the project. The output shown here was helpful in knowing what to expect going into that conversation.

Discussion questions

• What do you think of the current output that's being generated by gh-profiler?
• What additional information would be helpful to show, or calculate?

I don't want to gather too much information. There are some tools that give multiple screens full of information. I really want to scope this to a small amount of consistently helpful information about the user's profile and recent activity.

[drewbrew]

What do you think about showing similar information about issues being opened? I haven't looked at the GH API in any detail, but if you get info about number of issues opened and how many get closed as done vs not planned or duplicate, that might be a good indicator of a potentially spammy contribution.

[drewbrew]

Agreed, an issue remaining open isn't a red flag, but a high percentage of issues being closed as won't fix can be

[ehmatthes]

This has been fascinating to dig into. Big red flag: opening multiple issues with the same title across a number of different repositories.

Follow #27 if you're curious, or have more specific thoughts. :)

[drewbrew]

Very cool!

[ehmatthes]

I think the first release of this is already useful:

$ uvx gh-profiler <redacted>
GitHub user: <redacted>
  🟡 Account age: 159 days

  🟢 Profile information:
      name: <redacted>
      blog: https://<redacted>.com
      email: info@<redacted>.com
     Empty fields: company, location, bio

  🟢 <redacted> has opened fewer than 10 PRs in the last 21 days.

  🔴 <redacted> has opened 6 new issues in the last 21 days.
     🟢 0 issues have been closed as NOT_PLANNED.
     🔴 6 issues were opened with the same title:
        📋 Documentation Enhancement Suggestion (6)

[drewbrew]

Excellent!

[ehmatthes]

Note from PyCon US 2026 conversation with @sheenarbw: "How can we avoid putting red flags on new human contributors?"

That's a great question. Initial thoughts:

• Maybe account age is never a red flag? A new account can only be a yellow flag.
• Maybe a new account with no other problematic concerns is a green flag for account age?
• It's tempting to think about compound analysis and then simplifying output, but it's been really helpful over a month of usage to see the full range of information about a variety of users.
• Making criteria and analysis configurable should go a long way toward finding the best ways to highlight the signals that are most helpful to maintainers, and also signals that encourage good behavior without making people who are doing the right things feel like they're doing something wrong.
• Someone asked, "How do we help someone recover if they've taken an action that evaluates as red flags?" Currently, this only looks at activity over the previous 21 days. So as it's currently implemented, the simple answer is "Avoid problematic behavior for the next three weeks." I think we can come up with a more supportive way of communicating this, that acts as a guide to good open source approaches in the AI era.
• I'll move all this to a better-articulated discussion, because this is worth digging into as a guiding principle.

[ehmatthes]

Closing this; new thoughts should go in #69.