Pull Analytics: PR turnaround time, lead time for code review and changes

[thibaudcolas]

I’m surprised there is no discussion thread for this already! I never got to try Pull Panda’s Pull Analytics, but from the marketing page this seems like invaluable functionality to keep the pull request review funnel in check.

In the case of Wagtail in particular, we pay close attention to how fast we are at acknowledging the work of new contributors submitting PRs, providing code reviews, and (hopefully) eventually getting their code merged. We see this review turnaround time as one of the most important aspects of the project’s health.

We keep track of the various bottlenecks things currently get stuck at with labels and filters, but this is very ad-hoc. It’d be lovely to have actual charts, either auto-generated like the current repository-level Insights, or with manual creation like the Project Insights charts. We also keep track of different related metrics ("number of new contributors in last week’s PRs, proportion of last week’s PRs we’ve replied to") in a spreadsheet we manually update on a weekly basis.

In the meantime, with Project Insights

In the meantime, I think we might be able to get this already if we created a private organization project, and added all pull requests to this project with a different status per step of the funnel we want to track. We could then use the Project Insights burn-down chart to see how many PRs are at different stages (but not how long a given PR has spent at each stage – mentioned here too).

This would look something like this:

The chart is useful, and being able to see all of the PRs across the different stages all on the same table view is also invaluable. However this requires a fair bit of work to move the PRs between steps to match their label.

References

Aside from Project Insights, I can see two related items in GitHub’s public roadmap

• On Insights: Engineering Leader and Organizational DORA Reports #127, the "Lead time for changes" chart. In our case, this would be more about the PR review turnaround time – but close enough.
• On Insights: Fully integrated reports & dashboards for GHAE #171 – general pull request health metrics.

[thibaudcolas]

For anyone interested in this, two relevant things I stumbled upon:

• Another Discussions thread asking for feedback on pull request analytics: Feedback on GitHub Pull Request analytics #14270
• A third-party project implementing the "Average time to close pull requests": https://www.repotrends.com/

[hbd]

+1 this would be very helpful for measuring improvements in review process within a project.

In addition, would be great to get an average time taken to respond to a review request on an individual level. This is similar to a "usually response in X minutes" in an Intercom chat. For example, "hbd's average response time for a PR request is X minutes". Exactly what a response is (first response after initial review request?), is up for debate.

[abhishekbedi1432]

Thanks everyone for inputs.
I was wondering if there is any Github Action or Tool where we can see the Lead Time ( PR Review time / pipeline action time / etc) on the PR itself to see how fast our CI pipeline is performing.

[c12k]

I experimented with looking through commit history, but with pr squash and merge and branch deletion you lose the history time when a brach is created from main ( the key date for tracking time branch started to compare against merge time).