Investigate status and health of Exercism language tracks #97

Closed
kytrinyx opened this Issue Nov 20, 2016 · 56 comments

Projects

None yet
@kytrinyx
Member
kytrinyx commented Nov 20, 2016 edited

I've used Sarah Sharp's FOSS Heartbeat project to generate stats for each of the language track repositories, as well as the x-common repository.

The Exercism heartbeat data is published here: https://exercism.github.io/heartbeat/

The most important question that I would like to answer right now is to what degree each track is maintained, and if it is, who is actively maintaining. I've made up a scale:

  • ORPHANED - Nobody (other than me) has merged anything in the past year.
  • ENDANGERED - Somewhere between ORPHANED and AT RISK.
  • AT RISK - Two people (other than me) are actively discussing issues and reviewing and merging pull requests.
  • MAINTAINED - Three or more people (other than me) are actively discussing issues and reviewing and merging pull requests.

In addition to this, it would be interesting to know if there is any degree of activity. If here are fewer than 20 issues/PRs in the past year, the track can probably be considered inactive. (I'm making the threshold up).

For tracks that have activity, it would be useful to know if there are any contributors who should be nominated for commit.

Summary

Based on the comments below, here is a snapshot of the track health and status.

Maintained

  • Ruby
  • C
  • Java
  • Crystal
  • Elixir
  • Haskell

At Risk

  • Swift
  • Rust (but very active)
  • OCaml
  • JavaScript
  • ECMAScript
  • Python
  • Scala
  • F#
  • C#
  • Clojure
  • Objective-C

Endangered

  • Lua
  • Lisp
  • Erlang
  • Go
  • Factor
  • Perl 6
  • PHP
  • C++
  • Elm
  • Emacs Lisp
  • Lisp Flavored Erlang
  • MIPS Assembly
  • Racket
  • Scheme
  • Prolog

Orphaned

  • Bash
  • CoffeeScript
  • D
  • Groovy
  • Kotlin
  • PL/SQL
  • Perl 5
  • Pony
  • R
  • Standard ML

Inactive Tracks

There has not been enough activity to generate a heartbeat for the following tracks:

  • Ceylon
  • ColdFusion
  • Haxe
  • Idris
  • The Netwide Assembler
  • Nim
  • Pascal
  • PowerShell
  • Purescript
  • Tcl
  • TECO
  • VB.NET
This was referenced Nov 20, 2016
@NobbZ
NobbZ commented Nov 20, 2016 edited

I'm not quite sure if I do understand your request right.

I took a quick look into the list of issues (search by is:issue created:>2015-11-20) which has 14 open and 14 closed issues also I did search PRs (is:pr created:>2015-11-20) and had 2 open and 25 closed. This is above the 20 threshold, so we can consider xerlang as an active track. But I have to admit, that most of the issues might be due to my planned overhaul as well as blazon stuff, but I did not analyze it deeply.

Also aside of @kytrinyx it seems as if I was the only person merging or commenting over the last year.

I did even realize a couple of months ago that non of the former maintainers did respond to any team- or named mention and emailed @kytrinyx privately about this issue. Since then I try to do housekeeping on that track and also try to stem an overhaul and re-evaluation of the track on my own.

In the younger time, there have only been two or three people that offered help, but from all of them I didn't hear anything again after explaining what kind of help is most welcome at the current stage.


edit

According to my semi-analyzis from above the current status of the track is ENDANGERED or even ORPHANED!

@ryanplusplus
Member

xlua

Endangered based on the number of contributors, though I'll note that issues do get resolved quickly

xc

Maintained

@Insti
Insti commented Nov 20, 2016

xruby

Maintained

exercism/xruby#488

@kytrinyx
Member

@NobbZ I think you're probably right about the Erlang track being orphaned. You've been contributing a fair amount, though, and if you want to have commit on it, let me know, I'll add you.

@NobbZ
NobbZ commented Nov 20, 2016

@kytrinyx, actually I do have the necessary rights to commit directly, as well as I am able to do merges. Still, I can't stem the work necessary around this track on my own.

When I created the first issues about the overhaul everything was different. Back then I still hoped that the other three members of @exercism/erlang might take a look every now and then, but they didn't show up. Also I was unemployed back then, which changed as well. Last but not least, even if unborn, our youngest family member does take a lot of my free time.

@dvberkel
Member

xocaml

Endangered

exercism/xocaml#91

@verdammelt
Member
verdammelt commented Nov 20, 2016 edited

xlisp

Endangered

Single active maintainer. Few issues or PRs. No new exercises unless I submit them.

exercism/xlisp#125

(...and sorry for closing this issue. I hate when I do that..)

@verdammelt verdammelt closed this Nov 20, 2016
@verdammelt verdammelt reopened this Nov 20, 2016
@jtigger
Member
jtigger commented Nov 20, 2016

Java

MAINTAINED

Since June:

  • maintainers are: @matthewmorgan, @jtigger, @sit;
  • maintainers respond to PRs and issues within a day;
  • the average open time of PRs is down from 63 days to 5 days;
  • maintainers are actively and expeditiously participating in cross-track efforts;
  • notably, @stkent has made numerous contributions in the form of high quality PRs.
@mhelmetag
Member
mhelmetag commented Nov 20, 2016 edited

xcrystal

Maintained

  • maintainers are: @mhelmetag, @elorest , @bmulvihill
  • PRs have had numerous reviewers: @mhelmetag, @petertseng (has reviewed his own PRs πŸ˜† ), @elorest, @asterite and others
  • the average open time of PRs is less than a day
  • recently, @bmulvihill has been making awesome progress towards adding more exercises with the new spec generator
  • we're also pretty close to having rikki make some automated comments on crystal code

Update: Added two more track maintainers so we're all happy here in the Crystal Palace.

@masters3d
Member

Swift

AT RISK

  • Two maintainers
  • Occasional issue or comment. Not a lot of activity outside of maintainers.
@IanWhitney
IanWhitney commented Nov 21, 2016 edited

Rust

AT RISK (? I guess, I somewhat disagree with the ranking)

  • Two maintainers
  • We get some issues & sometimes get a PR from other Rust track students.
  • @ijanos has contributed a few exercises, so he might be a good candidate for adding to the team
@kytrinyx
Member

I can't stem the work necessary around this track on my own.

@NobbZ Yeah, that makes sense. The point of this exercise is to kick off some work to look into how to help find new contributors and ramp them up.

@kytrinyx
Member

I somewhat disagree with the ranking

@IanWhitney I don't have very strong feelings about the rankings. My reasoning for making 2 "at risk" is that if one person gets busy or goes on vacation or moves on, we're left with one person. When there's one person, there is much less discussion and suddenly a much larger burden. With 3 it's easier to have the kind of activity that means that others are more likely to get involved (quick turnaround, responsiveness to requests and issues, lively discussions, mentorship). So with 3 I think that we're more likely to have something that can be sustainable.

That said I've been incredibly excited about (and impressed with) the level of activity in the Rust track. I just don't want that to stop!

@kytrinyx
Member

[ijanos] might be a good candidate for adding to the team

@IanWhitney would you reach out to them and see if they'd be interested in joining the team? (There's some documentation in https://github.com/exercism/exercism.io/blob/master/docs/maintaining-a-track.md and other documents in that directory)

@kytrinyx
Member

I updated OCaml to at risk (from endangered) as there are two maintainers.

@elorest
elorest commented Nov 21, 2016

Add me back and I'll spend more time on it. I took a new job a couple
months back and have been a bit preoccupied.

On Nov 20, 2016 4:29 PM, "Max Helmetag" notifications@github.com wrote:

xcrystal

** Endangered ** (based on the amount of active maintainers; just me)

β€”
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#97 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHmpBZ0h8vSga1AVLnADhL1iPXwucNQks5rANfZgaJpZM4K3j8H
.

@behrtam
behrtam commented Nov 21, 2016

Python

AT RISK

  • maintainers are: @Dog, @behrtam
  • we lack reviews/reviewers
  • activity status: 62 prs, 43 issues in 2016
@rchavarria

JavaScript

AT RISK:

  • Two maintainers
  • Issues and PRs from time to time, but I feel that it takes too much time for me to answer questions, merge PRs or take care of other aspects

ECMAScript

Exactly the same as JavaScript.

AT RISK:

  • Two maintainers
  • Issues and PRs from time to time
@rbasso
Member
rbasso commented Nov 21, 2016

Haskell

AT RISK:

  • Two active maintainers.
  • Sporadic contributions from non-maintainers.
  • No frequent contributor to receive write access yet.
  • Still under active reformulation: 579 commits in the last year.
@matthewmorgan

@rchavarria @kytrinyx I would agree with RubΓ©n's assessment above for Ecmascript and Javascript. I've been helping a little once in a while on both those tracks but a third maintainer would help. I've gotten busy over the last few months and it seems like RubΓ©n is in the same boat.

@kytrinyx
Member

@mhelmetag I'm excited about the progress with both generators and rikki! (I owe you a code review there, the past couple of weeks have been terribly hectic).

@elorest Thank you! Expect an email about this in the next day or so.

@petertseng
Member
petertseng commented Nov 21, 2016 edited

Go

Go is Endangered

Rationale:

  • Active improvements to the track and fixes for issues are not being made by the maintainers themselves, but in general pull requests from contributors are being accepted.
  • Of all maintainers on the team, only @kytrinyx (whose activity we've been instructed to disregard) and I are currently merging pull requests for at least the last few months.
  • I don't have any particularly notable instances of non-maintainers reviewing others' pull requests. I notice one instance at exercism/xgo#403, but unsure of continued activity. Once again, just me and @kytrinyx . Maybe non-maintainers are unsure whether their opinion would be welcome!
  • The rate of outside contributions seemed a bit low, usually about 1 a month to 1 a week. Hacktoberfest is an exception.
  • I am grateful for all contributors who have contributed to the track for Hacktoberfest. If any of you are interested, feel free to let us know. I do note that @ferhatelmas has contributed a few times with good quality PRs.

I agree with the previously-written reports on Rust and Haskell. Both tracks are at risk due to having two maintainers, though there is a fair bit of activity on both, so if a third maintainer were to come on and activity were to remain the same, the tracks would qualify as Maintained. I do have a potential Haskell contributor in mind, unless this contributor has already declined to come on as a maintainer

@petertseng
Member
petertseng commented Nov 21, 2016 edited

re: crystal track report

PRs have had numerous reviewers: @petertseng

Doh, I am pretty sure that was me commenting on my own PR (and as you can see, I made a large number of them). It might be good if the reviewer stats would exclude self-reviews? I mention this because I can't remember ever having commented on anyone else's PRs on the Crystal track (I'm interested in the language, but my other tracks have my hands full)

That could probably do with a report upstream, but I can't right now - limited access.

@catb0t
catb0t commented Nov 21, 2016

Factor

  • To what degree is the track maintained?

It's up to date with current Exercism standards in terms of config file and folder structure format, but I haven't been updating it recently, unfortunately. (Also because I've been trying to figure out how to automate more things.)

  • Who (if anyone) is merging pull requests?

kytrinyx and I can merge requests that come through because I check my GH notifs often, but it's not like we get a lot of activity anyways.

  • Who (if anyone) is reviewing pull requests?

Same answer as the last.

  • Is there someone who is not merging pull requests, but who comments on issues and pull requests, has thoughtful feedback, and is generally helpful? If so, maybe we can invite them to be a maintainer on the track.

No, because this track is low-traffic and low-activity which is less stress for me but boring otherwise :)

This track is "endangered" because I'm the only maintainer aside from kytrinyx.

There's not even the occasional issue/comment, though.

@ricemery
Member

Scala

  • To what degree is the track maintained?

More time should be spent on updating the config file to include the exercise metadata. But, in general the track is in good shape.

  • Who (if anyone) is merging pull requests?

Only myself and @kytrinyx on occasion.

  • Who (if anyone) is reviewing pull requests?

Only myself, @abo64 and occasionally @kytrinyx .

  • Is there someone who is not merging pull requests, but who comments on issues and pull requests, has thoughtful feedback, and is generally helpful? If so, maybe we can invite them to be a maintainer on the track.

I think we should invite @abo64 and @ErikSchierboom to be Scala track maintainers.

I would say the Scala track is "at risk". I am the only person actively merging pull requests. But, there have been quite a few pull requests recently from @abo64 and @ErikSchierboom .

@kytrinyx
Member

@ricemery I'm going to put Scala as endangered, but if @abo64 and/or @ErikSchierboom join, we could update that. Ric, would you reach out to them via email to discuss that and let me know if I should add anyone to the team? (If useful: https://github.com/exercism/exercism.io/blob/master/docs/maintaining-a-track.md)

@ErikSchierboom
Member

I'd be happy to join the Scala track as a maintainer, even though I am no Scala expert by any means. However, I do know Exercism and manage the C# and F# tracks, so the types of issues we might receive would be familiar. If @abo64 would also join, that would be a fantastic addition to the team. He has been incredibly helpful reviewing my submissions.

@ErikSchierboom
Member
ErikSchierboom commented Nov 21, 2016 edited

C#

ENDANGERED

  • To what degree is the track maintained?

The track is actively maintained. It has the second-most implemented exercises and issues are usually quite swiftly responded to. There will be some substantial rewriting when the .NET Core CLI is finished, making the track fully cross-platform with much simpler instructions.

  • Who (if anyone) is merging pull requests?

Mostly, it's me, with occasionally the help of @bressain and @jwood803

  • Who (if anyone) is reviewing pull requests?

Again, it's mostly me, with occasionally the help of @bressain and @jwood803.

  • Is there someone who is not merging pull requests, but who comments on issues and pull requests, has thoughtful feedback, and is generally helpful? If so, maybe we can invite them to be a maintainer on the track.

@robkeim has added a lot of great exercises the last couple of months.

@ErikSchierboom
Member
ErikSchierboom commented Nov 21, 2016 edited

F#

ENDANGERED

  • To what degree is the track maintained?

The track is actively maintained. It has the most implemented exercises and issues are usually quite swiftly responded to. There will be some substantial rewriting when the .NET Core CLI is finished, making the track fully cross-platform with much simpler instructions.

  • Who (if anyone) is merging pull requests?

Mostly, it's me, with occasionally the help of @jwood803

  • Who (if anyone) is reviewing pull requests?

Again, it's mostly me, with occasionally the help of @jwood803.

  • Is there someone who is not merging pull requests, but who comments on issues and pull requests, has thoughtful feedback, and is generally helpful? If so, maybe we can invite them to be a maintainer on the track.

@robkeim has added a some exercises the last couple of months, which was very helpful. Furthermore, @rmunn, @rebelwarrior and @pminten have participated in several discussions, and @mattcbaker has indicated he wants to help.

@robkeim
robkeim commented Nov 21, 2016

@ErikSchierboom - Let me know if there's anything that I can do to help the C#/F# track because I'd definitely hate to see them get left behind!

@jwood803

@robkeim If you become an extra maintainer I think that's the boost we need to get out of Endangered, I believe.

@ricemery
Member

@ErikSchierboom, great! Your help will be greatly appreciated.

@kytrinyx , can you add @ErikSchierboom as a maintainer on the Scala track. I don't think that I have the rights to add maintainers to the team.
Thx

@abo64
abo64 commented Nov 22, 2016

Thank you for the invitation, I happily accept to join the Scala team! :-)

@jtigger
Member
jtigger commented Nov 22, 2016

I too am a @abo64 fan; I've learned more about FP from that guy...

@ErikSchierboom
Member

@robkeim Would you be interested in become a maintainer of the C# and/or F# track? We could really use your help, and you've already made some great contributions.

p.s. there is this nice document on what it means to maintain a track, for those interested.

@kytrinyx
Member

Fantasticβ€”thanks everyone for coordinating this.

I've added the following people to the following tracks:

There are a few more open requests pending as well.
I'll update the status of these tracks in the issue.

@ErikSchierboom
Member

Welcome on board @robkeim! πŸŽ‰

@robtimp
robtimp commented Nov 22, 2016 edited

Objective-C

AT RISK

  • Two maintainers: @masters3d, @robtimp
  • Fairly light activity. I’ve been adding the occasional new exercise.
@haus
haus commented Nov 22, 2016

Clojure

Clojure is probably endangered.

To what degree is the track maintained?

The track does get a good amount of contributions in the form of new exercises and improvements to existing exercises from a bunch of contributors (17 unique PR authors in 2016).

Who (if anyone) is merging pull requests?

@yurrriq is the only maintainer besides @kytrinyx that has merged PRs in 2016.

Who (if anyone) is reviewing pull requests?

Generally review on PRs comes from @yurrriq.

@kytrinyx
Member

@ferhatelmas has accepted an invitation to be co-maintainer of the Go track πŸŽ‰

@kytrinyx
Member

@haus Thanks for taking a look at the status of Clojure. I think you're right about it being endangered.

@yurrriq
Member
yurrriq commented Nov 22, 2016

πŸ‘ to adding @haus as maintainer of xclojure. I agree it's endangered.

@robkeim
robkeim commented Nov 22, 2016

Thanks @ErikSchierboom and @kytrinyx I'm happy to help out on the C#/F# tracks!

@haus
haus commented Nov 22, 2016

I'd be delighted to help out with xclojure.

@kytrinyx
Member

Thank you, @haus, I've added you to the Clojure team ✨ and upgraded the track from endangered to at risk.

@kytrinyx
Member

From @parkerl about Elixir:

MAINTAINED

  • To what degree is the track maintained? We generally have several PRs every month. Many are two add new exercises.
  • Who (if anyone) is merging pull requests? @devonestes and @parkerl are the most active maintainers at this time.
  • Who (if anyone) is reviewing pull requests? We have great help reviewing PRs. Notably, @martinsvalin @lpil @dalexj have been great.
@petemcfarlane
petemcfarlane commented Nov 23, 2016 edited

PHP

ENDANGERED

To what degree is the track maintained?

7 pull requests in the past month from 5 authors

Who (if anyone) is merging pull requests?

I am the only maintainer besides @kytrinyx that has merged PRs in 2016.

Who (if anyone) is reviewing pull requests?

Mostly me again

Is there someone who is not merging pull requests, but who comments on issues and pull requests, has thoughtful feedback, and is generally helpful?

@brettsantore has contributed and has been useful, he may be interested?

@brettsantore

@petemcfarlane Next week, I'll be able to review the track again and take a look at the common track. I'll see if I can spare some time to help out.

@masters3d
Member
masters3d commented Nov 23, 2016 edited

I am not sure if something like "how many tracks is a person currently maintaining" could affect the rating but if somebody is maintaining 3+ tracks and that person leaves, all of the sudden we have 3+ tracks that go from endangered to ORPHANED. There are only so many hours in a day.

@kytrinyx
Member

@brettsantore ✨ that is much appreciated. If you want commit say the word!

@kytrinyx
Member

@masters3d That's a very good point. I'm hoping that we can put some things into place to make it more likely that all the are in a good, safe, sustainable place. Getting this overview is a first step, and it's been very encouraging to have this conversation and see how many people have been actively helping out across the board in the past year.

@kytrinyx
Member

Thanks, everyone, for weighing in on this!

@kytrinyx kytrinyx closed this Nov 24, 2016
@rbasso
Member
rbasso commented Nov 28, 2016

@kytrinyx, could you please add @abo64 as a maintainer in the Haskell track too?

Both @petertseng and I agree that he would be a great addition to the Haskell team, and this would also move the Haskell track from "At risk" to "Maintained".

@kytrinyx
Member

@rbasso done! @abo64 Thank you!

@kytrinyx kytrinyx referenced this issue in exercism/exercism.io Dec 4, 2016
Open

Add exercism to up-for-grabs? #3124

@Theodus
Member
Theodus commented Jan 2, 2017

@kytrinyx I would like to be added as a maintainer for the Pony track so that I can help get it off the ground.

@kytrinyx
Member
kytrinyx commented Jan 2, 2017

@Theodus done, and thank you ❀️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment