Public Metrics

[gregdek]

We should be keeping some public metrics. Some I can think of:

• Number of PRs opened vs. closed
• Number of issues opened vs. closed
• Number of modules maintained by core team vs. individual maintainers

...just a few ideas.

[gundalow]

A few other suggestions

• Elapsed time be ticket raised and triaged
• Number of modules with tests (I'm working on this)
• Number of modules and modules-extra

[gundalow]

I found http://pgjones.github.io/push-pull/ Not been updated in a couple of years

However when add a GitHub API key you can see some stats

[MichaelBaydoun]

Additional ideas

• Total number of contributors
• Number of plugins
• Percent of code submitted by non employees
• Stars/Forks over time

[robynbergeron]

Okay, I've started vaguely drafting something up for what this list looks like -- still have a bunch of unwritten ideas in my head.

I think we can probably discuss at some point the relationship between the statistics and goals, or, the statistics and "how we use them", what they indicate, at what point they should become action triggers, etc.

https://github.com/robynbergeron/thingsandstuff/blob/master/monthlyreportstats

[robynbergeron]

@gregdek -- you're the assignee right now, I could move this over to me totally in a heartbeat, unless you have stuff specific you'd like to do here.

[mckerrj]

Since gregdek and I are sposed to chat about this this week, one that I've been thinking about is average age of a PR?

[robynbergeron]

I will note that if we're doing this monthly I'd love to start with some (even rough) tracking items this month and a clear path forward, since ... once a month is usually the last day of the month, and it only happens once a month

[robynbergeron]

@mckerrj: I've got that tracked in here.

[gregdek]

Take it pls.

[gregdek]

https://jctanner.github.io/

And @jctanner is now working on this. :)

[jctanner]

closure ages: https://jctanner.github.io/ansible/stats/ages.html

[jctanner]

All of the content on my user page is being generated by the code in https://github.com/jctanner/pr-triage

I will eventually move it to a separate repo which is not a fork of sivel's

[gregdek]

@jctanner is this something we can/should link from somewhere?

[robynbergeron]

Labelling for discussion in Community WG meeting today.

[robynbergeron]

0:16 <@RBergeron> If nothing else: having a static page with "if you need the basic metrics about ansible, here's where
they are" might be useful, and we could get most of that stuff by API.
10:16 <@RBergeron> # of contributors, stars, forks, contributions, # of meetups, members, # of ... blah blah blah
10:16 <@gregdek> Yeah, that kind of low-hanging fruit thing would probably be good
10:16 <@gregdek> I do worry about vanity metrics though
10:16 <@RBergeron> that people always cite like, 85 years old versions of
10:17 <@RBergeron> Well, we could always do a blog post about ... something relating to "meaning of metrics" or
whatever, and just link it to the top of the page.
10:17 <@gregdek> true.
10:18 <@RBergeron> 'here are the metrics you cite, here are the metrics we actually (hopefully) use to improve ourselves"
10:18 <@RBergeron> I can poke at the issue. Do we want to do that? Maybe come up with a proposal or two + wtf are other
people doing?
10:18 <@gregdek> So. jctanner collects some metrics. We need to decide if we're using them.
10:18 <@gregdek> Yep, I like that.
10:19 <@RBergeron> where we == me i guess
10:19 <@gregdek> Maybe put together a quick+dirty metrics page somewhere in the short term, figure out a good strategy
long-term?
10:19 <@RBergeron> Yeah. I mean -- we could just preschedule a quarterly "metrics review" type of meeting in this hour
and just .. see if we're alarmed by anything, if the metrics are useful, whatever.
10:19 <@RBergeron> Yeah.
10:19 <@gregdek> If you want to assign that issue to yourself, that definitely wfm :)

[gregdek]

I may now believe this is a monthly news column by jtanner :)
1:48 PM C +1
1:48 PM G<•gregdek> Greg DeKoenigsberg It would be interesting to actually talk about what things we measure and why.
1:48 PM Raw numbers are meaningless. Metrics exist solely for analysis.
1:51 PM C you could even break the metrics down by topic, i.e. cloud modules, aws, docker, etc.
1:51 PM P other number: number of "automerged" PR. number of shipit by roles (module maintainers, namespace maintainers, others)
1:52 PM G<•gregdek> Greg DeKoenigsberg Yeah.
1:52 PM And all of these metrics are really asking the question: how are we doing at managing all of the requests to improve Ansible? That's kind of the overarching question.
1:53 PM jtanner are these the kinds of questions that we have the metrics to answer?

[dagwieers]

There's a list of services or stats:

• http://ghv.artzub.com/#repo=ansible&climit=100&user=ansible
• http://git-awards.com/users/search?login=ansible (Ansible in 11th place for python)
• https://githubstats.com/ansible/ansible
• https://octoverse.github.com/ (Ansible listed a few times in top 10 lists)

[gregdek]

Thanks @dagwieers . I created this page:

https://github.com/ansible/community/wiki/Community:-Metrics

For the most part, though, the question of "how to use metrics to improve Ansible" is a question for the core team to answer. I think we've got what we can reasonably get from this conversation, so I'm going to close this issue.