The case of gems not needing active maintenance

[skatkov]

There is a recurring theme that comes up during my work with different teams (I'm "consulting"). And I have a feeling that ruby-toolbox probably indirectly contributes to this theme and I'm wondering if we can do something about it.

So here is an issue

• I'm offering some low-magic gem with a small footprint (not much code, nut much dependencies), that solves exactly what is requested. I can easily glance through code myself. I'm pretty sure it's very unlikely to disrupt our rails update. Or I can take over this even if it's unmaintained.
• But team asks me: Last commit was 4 years ago... Let's pick something else?

First gem that I heard this argument pop-up was searchlight, it almost got rejected by a team. It was actually adopted and for a couple of years, it was successfully used without much changes.

It's somehow expected that gem should always have a steady stream of contributions.

And I think this is not a healthy expectation from open-source. I've been playing around with Clojure/ClojureScript and noticed that a lot of libraries have remarks about it's "completness"

"this library is complete. it works, there is no needed to fix anything"

so @nathanl notes in searchlight gem as well.

I consider searchlight "done". It has no dependencies, so there's no reason it shouldn't work indefinitely.

Searchlight looks scary on ruby-toolbox page, a lot of red colors. like REALLY RED.
https://www.ruby-toolbox.com/projects/searchlight

But downloads are growing steadily. It's clearly being actively used.

My question here is the following

Since a lot of prominent open source contributors burned out, moved away. Should we better represent 'finished' gems on ruby-toolbox? To create a bit healthier expectations of open-source?

[FranklinYu]

I understand that this gem is finished, but at least I think the repository shouldn’t be archived, and issue tracker should be open, even if it is very unlikely that there would be any issue. If you are so confident that there won’t be any issue, leaving the issue tracker open won’t create much (if any) maintenance burden. The project page says things correctly:

• It has steady download, so it probably also works for you.
• The author has stated the desire not to maintain the project, for whatever reason (including the reason that the project doesn’t need maintenance).

I would argue that issue tracker is still useful. For example, people may discuss about feature request, and exchange information about forks. Also if Ruby 3.0 is released someday, this gem may or may not work with it. Anyway, as long as the issue tracker is open, we have a place to talk.

It is left to users of the gem to decide what these metrics means. The only thing worth discussion is (IMO) color of the two label; we may make “Repository is archived” and “No release in over 3 years” yellow label instead of red, because that is somewhat subjective. (I’m not saying that it should be turned yellow; I only mean that it can be discussed.)

[nathanl]

Thanks for the mention @skatkov. I agree that "done" projects exist, as long as the language itself doesn't break compatibility. PriorityQueue is another example. Not touched since 2009, but it's just a data structure, and it worked great when I used it in 2016.

Good points @FranklinYu - I unarchived Searchlight in case people want to create issues and discuss in the future.

[skatkov]

There are a actually a lot of examples, but another favorite of mine is discard.

https://github.com/jhawthorn/

A lot of stuff that dry-rb guys are doing will probably at some point will be "done".

I think it's a healthy to have a backbone of such gems in eco-system. And it would be nice to motivate people to consider this possibility.

[nathanl]

I wasn't familiar with discard, but given that it depends on ActiveRecord, it seems like it may need updates in the future.

Some signals that a gem may be "done" might be:

• Are people downloading it, showing that it's useful and not half-baked or obsolete?
• Does it have 0 dependencies? That's a strong sign, because in that case it shouldn't break unless the language itself changes. (Though it's possible for it to have undeclared dependencies, or depend on an interface that may change; with Searchlight I note that it depends on the ORM supporting a "chaining" model of querying, like ActiveRecord's .where(...).where(...).limit(...), or similar chains with Sequel, Mongoid, etc, which I think are unlikely to change but could.)
• If it has dependencies, are they stable?

This could be tricky to work out programmatically. An easier approach would be to just let humans mark gems as stable.

[skatkov]

Adding more to examples, @dkubb has a lot of nice gem's that rarely get updated.

https://github.com/dkubb/adamantium
https://github.com/dkubb/axiom

Those have dependencies, but still -- feels that there are not many reasons to maintain those all the time.

In a case of discard, it does rely on AR... but it touches API surface so minimally, that Rails 6 update doesn't take that much.

jhawthorn/discard#45

An easier approach would be to just let humans mark gems as stable.

Not sure how signaling of such a status could be accomplished.

P.S. I'll cross-post this on Reddit, to see if we will have other input. :)

https://www.reddit.com/r/ruby/comments/bc2ih3/rubytoolbox_the_case_of_gems_not_needing_active/

[vidarh]

Some way of more prominently signalling number of open pull requests and number of open issues might be an interesting metric. Maybe relative to number of downloads. It would seem that if a gem has a high number of downloads but a very low number of open issues or open pull requests that's a good signal that few people are having issues that are going unaddressed.

[ylluminate]

I think @vidarh's concept is relatively valuable in this sense. A comparison of open issues vs PRs vs recent downloads is a usable metric to determine "lifeness" (at the very least interest). These metrics could be decoupled into lifeness vs interest (heck this would be good to also quickly determine effort focus for folks looking to help out at various places).

I think the idea of something being usable with current codebases might also be valuable and it could be a simple "vote" that could be recorded by people who are hitting the project. For example, a rubytoolbox gem were available that would record the consumer codebase's version info vs what gems are being used, that would be super useful to determine this really valuable metric and breath new life into many projects. Otherwise this metric could be manually entered by site visitors who feel inclined to indicate version info they're using it on (but clearly much less input would be expected with this manual method).

Hmmm.

[FranklinYu]

There are a actually a lot of examples, but another favorite of mine is discard.

@skatkov I think the case of Discard isn’t as typical as Searchlight. Searchlight doesn’t have release in over 3 years, because there is nothing to release. If you compare the master branch with last release you will see that only markdown files are modified. In contrast, in the case of Discard, I see several Ruby file updated. The package isn’t really “finished”; not as much as Searchlight. Author is still maintaining it.

[colszowka]

Hey all, I read through the discussion a bit here and on Reddit. I don't have time right now for an appropriate in depth comment, but I really appreciate the lively discussion and the great ideas brought up, so thanks for your contributions, keep it up! I'll be back soon :)

[colszowka]

Also I'd like to ping @maxim due to #355 (comment) where we already had a brief discussion about the same matter :)

[maxim]

Thanks for keeping that discussion in mind @colszowka. I guess there's no need to copy my comment here, but having re-read it, I still agree with it. Here're some additional thoughts, but take them with a grain of salt since I don't have experience maintaining something like ruby toolbox.

In general, I think it's important to embrace both: the automated data gathering/calculation, as well as allowing community to make custom edits.

It's hard to combine these 2, because automated data can keep itself updated while community edits get stale.

If I was to imagine an ideal world (without any knowledge of how ruby toolbox is currently implemented), I would consider something like the following:

1. Internally, keep auto-calculated data separate from custom edits.
2. Introduce the concept of "expiration" of custom edits, where after certain amount of time they would become stale.
3. Send periodic notification to the author of a custom edit to check if it needs updating.
4. If notification didn't get a response, and there was no other custom edit activity, fallback on auto-calculated data again.

In other words, expiring custom edits could help solve the issue of having gems that cannot be easily judged by algorithms. However, I recognize the above is quite a lot of work.

[skatkov]

Maybe a wild idea, but I think that easiest way is to have 'gem: complete' badge. It would be very easy to propose such badge for any gem through Pull Request.

It would be easy to 'track' such gems (maybe even add a separate category "timeless gems'? :) ), it will also lead back to ruby-toolbox for an explanation page.

[gettalong]

Better than a badge would be a meta-data field in the gem file itself. Then this information doesn't depend on the location of the source repository and could even be easily displayed on rubygems.org.

[maxim]

I like the "completed" mark. Would a gem that only accepts bugfixes, and will never change API be
considered completed? Or does it have to have absolutely no activity? I imagine it would be difficult to make a distinction if a gem is perpetually open to bugfixes, and sees usage, but nobody's filing anything.

[FranklinYu]

@gettalong If you mean .gemspec then this need to be implemented by RubyGems. Gem author cannot add random metadata into RubyGems AFAIK.

And I seriously doubt whether RubyGems team would like this feature, according to their previous reactions.

[nathanl]

Or does it have to have absolutely no activity? I imagine it would be difficult to make a distinction if a gem is perpetually open to bugfixes, and sees usage, but nobody's filing anything.

I don't think this has to be absolute. Right now we have an indicator of "few / no recent commits" which is taken to mean "abandoned". All we need is something that says "yeah there's a reason for that, and it's nothing to worry about in this case."

[AxelTheGerman]

I don't think this has to be absolute.

Agreed. Also in regards of dependencies. Even if you have a few dependencies, e.g. ActiveRecord. As long as your dependencies are stable and you use only small parts of their API the gem can still be complete.
Complete doesn't necessarily have to mean you will never ever have to do anything. If there might have to be a small commit to add a new dependency version to your CI build and bump the version that's fine IMO

[CharlesHi]

A very good point, but with a lot of angles.
I'd like a message along the lines of "validated against this, that, and the other", where "this" would be a version(s) of the Ruby language, and "that" and "the other" would be dependencies with versions. It's more work, but more useful. And if the original author had to drop the project, that kind of message could be updated by someone else fairly simply.

[PeterCamilleri]

I also thought that my code would not ever need updating but I was wrong. Changes to minitest, rake, rdoc, reek and other tools plus the addition of codes of conduct mean that the gem needs TLC on occasion.

Also, since my source code is backed on GitHub, I am free to add a comment to the repo. Code that is no longer under development is noted as such.

If you don't want to rely on the source code repo, I suppose you could always release a new version of the gem with a new description. Though it seems a bit pointless to release a BRAND NEW VERSION that only says go away! ;-)

[brandondees]

I've also encountered a disconcerting amount of the OP's scenario... I would settle for just visually de-emphasizing the scary visuals associated with low repo activity. The stats should be explicitly for the user to interpret, not suggestive of a "good/not-good" quality.

Over the years I've encountered a variety of tools that are not popular, not actively maintained, maybe have a few open issues that are unimportant, but are absolutely stable and the ideal solution for a specific problem scenario, but teams will still preemptively reject them on the basis of "hasn't been updated lately, so we can't use it" and won't even open up discussion about how the stats are subject to interpretation. I've seen blog posts coaching devs to adopt and adhere to these types of policies, but at the end of the day, it's the tools like ruby-toolbox and similar that reinforce those notions instead of promoting more critical thought in dependency evaluation, at least in my opinion and experiences.

[6temes]

I think that there is a point that maybe has not been commented yet. We are using the words Active, Abandoned and Completed, and perhaps we should define them first. So let me try.

Active Development: The Gem may or may not be ready for production. The scope or the Gem gem may or may not be wholly defined, and people are currently working on building this gem.

Completed: The Gem is feature complete, according to the author, and there will never be changes to the API. There may be bugfixes, and the author is periodically updating the dependencies of the gem and making sure that it works with the latest versions of Ruby.

Abandoned: The author does not answer to Issues opened in the repository, or the gem is not compatible with the latest versions of Rails and/or Ruby.

So, the main problem to solve is to find a way to differentiate between Completed and Abandoned gems.

[aledalgrande]

Just adding my 2 cents here, after reading the above comments: the Ruby community has always been very good with tests.

How about running, for each gem in the toolbox, the associated tests in a matrix of different Ruby/Rails versions on Travis? If the repo doesn't offer tests, yellow badge. If the gem fails with recent combinations, red badge (indicates it has not been maintained). If the gem fails with the upcoming version/combination yellow badge (fix might be underway). If the gem passes all the combinations, green badge.

This for me shows more clearly if work is being done to maintain the gem, over a certain amount of time, or if maintenance work is ever needed, abstracting from having to define "complete".

Coupled with the number/age of open issues, it would tell a complete story.

[suhlig]

A piece of code may be done in terms of features, but I doubt that it can ever be done in terms of maintenance. Even for a gem with zero dependencies (hard to imagine if it has tests), there is a new version of Ruby every now and then.

So yes, let’s find a way to mark those projects that have green tests with the latest releases of their dependencies incl. Ruby itself. In a time where services like @dependabot exist, it should not be too hard to automate this as a maintainer (I hope).

[nathanl]

Abandoned: The author does not answer to Issues opened in the repository, or the gem is not compatible with the latest versions of Rails and/or Ruby.

A piece of code may be done in terms of features, but I doubt that it can ever be done in terms of maintenance. Even for a gem with zero dependencies (hard to imagine if it has tests), there is a new version of Ruby every now and then.

I understand these concerns. But even abandoned gems can be stable and useful.

I would again point to PriorityQueue. It has not been touched in a decade and was written against Ruby 1.8. It has no public repo, much less an issue tracker or CI. It really is abandoned, and certainly one could wish it were updated, given better documentation, etc. (When I used it, I attempted to contact the author to offer to take over maintenance, but never heard back.) Yet it works just fine with Ruby 2.6.2p47, released about a month ago, according to the specs of mendel, which uses it. It depends on nothing but basic Ruby syntax, and this is the secret to its stability.

Its Ruby Toolbox page flags it as having no release in over 3 years, suggesting you shouldn't use it. But it continues to be downloaded because it works, and barring breaking changes to Ruby itself, it will keep working indefinitely.

If I were on your team and you needed a priority queue, I would advise you to use this one rather than write your own, especially considering that the guts are implemented in C. It's abandoned, definitely. But it works well. If no algorithmic way can be found to indicate this, maybe a manual flag or vote is best?

[PeterCamilleri]

Until such time as Travis is cross platform, I don't think it can be a requirement for gem health.

[andhapp]

This is a really interesting discussion!

About 4 - 5 years ago, I came across quite a few gems which were feature complete yet weren't upgraded to work with newer version of rails or something else. There was no information to say if it was okay to use it or not. As a developer, you want something to work and work then and there, otherwise it's very frustrating. I comprehend how a gem can fall into disrepair with life and other things taking over.

I thought I could write something quick to fix this issue, something like a travis-badge on each repository. I decided to write a script and take into account a few metrics, something @vidarh alluded to. I was planning on using:

1. Last commit date,
2. Number of open issues,
3. Number of open PRs,
4. Number of open issues where the repository owner/committer has responded
5. Number of open PRs where the repository owner/committer has responded
   and a few more that I can't remember now

I can't remember if I considered downloads at the time, but it seems like a good idea now that it's been mentioned here. Reason I added 4 and 5 was to catch out some very popular gems like Rails for instance, that has a lot of open issues/PRs but are actively dealt with. I started writing scripts to collect this information then realised that this could suffer from my personal prejudice and gave up on the idea altogether.

Couple of years down the line, I started doing a bit of machine learning and decided to pick this project up again and leverage machine learning for the same. I wanted to use the supervised machine learning and for that I needed to label the repositories correctly and I wanted to get the community involved and have a collective opinion in marking something as abandoned. Also, give an opportunity to the actual author to revive it, if he/she is interested to or pass the baton along. Keep the whole process transparent. Could have opted for unsupervised learning but I suspect it will lump something very popular with something very unpopular depending on the feature vector it's fed and again I decided against it. (I am no expert an machine learning.)

Now, I had 2 challenges; 1) how will someone label? 2) Not being a very popular member of the community, how do I get the word out?

For 1, I decided to write a chrome plugin, which I nearly completed. Here are couple of screenshots from it:

I called it github-abandonware to start with, but found the name incredibly mundane and changed it to NeedsUpdate. I never really got to the 2 challenge, like someone famous said: I made plans - then life happened.

Anyways, fast-forward now, if there's interest in the community for something like this, I don't mind pairing up with few more active members who can get the word out and make this thing a success. Perhaps, it will be a fitting denouement to something I thought of but could never quite action in the right way.

Phew.

Update:

There are other challenges when it comes to machine learning part of this project, which I've adroitly taken out. One that springs to mind is false-positive and how do we fix that?

[FranklinYu]

How about running, for each gem in the toolbox, the associated tests in a matrix of different Ruby/Rails versions on Travis?

How to do that technically? Regular builds, or builds triggered by reverse-dependency?

[skatkov]

Thanks to Ruby weekly for picking this up and spreading the link. As an OP - I'm happy that this resonated and love reading all the replies. But I'd like to summarize and maybe find some possible action points:

• I agree with @6temes, it seems important to nail down definition precisely. It feels that feature complete status seems most appropriate and explains it well.
• I really like @gettalong idea about meta-tag in gemfile. It seems like a great way for an author to signal status to all the possible parties involved. We can certainly ask RubyGems if they consider this a good idea, it doesn't hurt to ask :)
• It also seems that @FranklinYu might be a right path with coloring proposal and a lot of people have voiced support for it:

It is left to users of the gem to decide what these metrics means. The only thing worth discussion is (IMO) color of the two label; we may make “Repository is archived” and “No release in over 3 years” yellow label instead of red, because that is somewhat subjective. (I’m not saying that it should be turned yellow; I only mean that it can be discussed.)

[simi]

I left a comment at rubygems/rubygems#2734 according to adding this meta to Gemfile.

Anyway I had an different idea about introducing new metric based on amount of downloads (recent, for last version, overall) combined with amount of issues when available (opened, closed), releases and dependencies status. Isn't that actually the project score?

[joelvh]

Great discussion. Reading this thread and thinking about my own observations over the years, the ideas discussed make me wonder if looking at the age/activity of reverse-dependencies can be a proxy for how relevant an old gem is? Maybe this is what others have suggested, but if currently, active projects are using an old gem such as PriorityQueue, then that could definitely raise the value of it through some sort of rating/indicator that PriorityQueue is used in current projects.

Looking forward to see where these ideas land.

[FranklinYu]

Maybe this is what others have suggested, but if currently, active projects are using an old gem such as PriorityQueue, then that could definitely raise the value of it through some sort of rating/indicator that PriorityQueue is used in current projects.

This can be done in two ways: including “reverse dependency” when calculating scores, and/or mention the number of reverse dependencies in project page.

[simi]

...and/or mention the number of reverse dependencies in project page.

@FranklinYu I think it is in there already.

[joelvh]

Just to be clear, the number of reverse dependencies is interesting, but knowing the age of those is probably more helpful to indicate that newer projects are using this gem. Maybe it could be framed as some popularity rating among current projects. I'd have to give it more thought, but it's more about the time-relevancy of those reverse dependencies than the absolute number of them.

Maybe a little graph indicating popularity over time to show it's still "popular" or in use? That said, this is a suggestion specific to helping understand the value of what look like old gems.

[joelvh]

Maybe something to consider is the last release or average age of those reverse-dependent projects as a proxy/metric for relevancy of an old gem?

[mrleeio]

For most small gems I almost always recommend inlining the functionality to avoid this mess.

For larger gems I am mostly interested in the Authors ability / desire to maintain the project. I am often approached about using a gem and the first things I typically look for, in no particular order, are:

• Time since last commit
• Time since last release
• Open issues titled "Is this project dead?!"

Sometimes none of that information is present so I have a few options:

1. Be that guy and create the "Is this project dead" issue
2. Fork the gem
3. Use something else

The problem I have with a badge or label for "complete" is that it is static. Two years down the road when a breaking change happens how can we know that the Author is still updating the gem to account for these changes?

I would like to see a badge or label tied to an automatic reminder. Ask the Author every 2, 6, 12 months if they are still maintaining the project. If they do not reply, or say "no", place a warning on the project. Otherwise people are left guessing and look for other indicators.

[nathanl]

For most small gems I almost always recommend inlining the functionality to avoid this mess.

Good option.

Ask the Author every 2, 6, 12 months if they are still maintaining the project. If they do not reply, or say "no", place a warning on the project.

This replaces "have they actually made any commits" with "would they theoretically make any if needed", which is an improvement. But it doesn't account for situations the OP described - gems that do not need maintenance anymore. Searchlight has 0 production dependencies, so I don't see how it could break unless Ruby makes backwards-incompatible syntax changes, which I think is unlikely. It requires only an ORM which supports method chaining as a means of querying (eg, .where(...).limit(...), but has no knowledge of those specific ones), so in theory ActiveRecord or Sequel or Mongoid could break compatibility with Searchlight, but it would remain compatible with those that still use chaining.

(Caches also has no dependencies and Mendel depends only on PriorityQueue, so I think they won't need updates either, but they're much less used and therefore more likely to have bugs.)

Maybe it's hubris to say "I will never need to update Searchlight again", and I could be proven wrong. But that's my expectation. So if I don't respond to your issues, I assert that this doesn't mean Searchlight needs to be avoided. I thought that was the situation we were trying to address.

[PeterCamilleri]

Another metric could be to check for unresolved issues. It would show a definitive lack of required attention.

Having said that; The problem with that is that most people don't bother raising issues. They just solve the problem some other way.

[nathanl]

Another metric could be to check for unresolved issues. It would show a definitive lack of required attention.

Not all libraries have open issue trackers, and not all issues indicate problems with the library. Sometimes they are questions, requests for help with a problem the library didn't create, or requests for features the maintainer doesn't want.

I think this is going to be hard to solve with an algorithm. Maybe it's more of an issue for moderation or voting.

[mrleeio]

Maybe it's hubris to say "I will never need to update Searchlight again", and I could be proven wrong.

Having just gone through a Ruby on Rails upgrade, and Ruby upgrade, I beg to differ.

The Ruby update itself revealed around five dependencies which had to be removed, replaced, inlined, or updated due to changes in Ruby. Most of these projects would fall within your definition of complete, but the author had essentially abandoned the project.

It’s hard to say what will change in the coming years. I also feel that expecting authors to indefinitely support a project is a big ask.

[FranklinYu]

The Ruby update itself revealed around five dependencies which had to be removed, replaced, inlined, or updated due to changes in Ruby.

I think examples for such dependencies would benefit this conversation. Several example for “gems didn’t need update” has been mentioned above.

[mrleeio]

I think examples for such dependencies would benefit this conversation. Several examples for “gems didn’t need update” has been mentioned above.

Ruby 2.4, specifically the unification of Fixnum and Bignum into Integer, is a good example of a breaking change which resulted in abandoned projects. I don't think it is useful or necessary to name and shame specific gems, many of our favorites had to address this at some point.

With Ruby 2.3 having reached EOL I'm seeing a lot of these examples related to Ruby 2.4 upgrades pop up.

[vfonic]

Interesting conversation! Thanks @skatkov!

/disclaimer: Not sure if I'm getting off-topic here.

There's one more issue that's hard to address:
When the gem is unmaintained, but there are people interested in maintaining the gem (but can't get access to original repo). How to solve this issue?
Think of cancan vs cancancan. It took me about a year until I discovered that there's cancancan gem forked and maintained.
Maybe rubytoolbox can help addressing this issue as well? That would be really amazing! 🙏

[FranklinYu]

@vfonic I think that’s off-topic, but sounds like a great feature request. Create an issue for it?

[vfonic]

You're right @FranklinYu! I've created a separate issue: #488

[janko]

PriorityQueue is another example. Not touched since 2009, but it's just a data structure, and it worked great when I used it in 2016.

I agree that really old gems can still be useful if they work, but we shouldn't forget that people can find these gems by googling as well (which automatically searches RubyGems.org and GitHub). PriorityQueue isn't on Ruby Toolbox, but you still found it 😃

Maybe it's not necessary for Ruby Toolbox to cover 100% of the ground, but keep its focus to "actively maintained & popular" gems, like this repo's description states.

Gem author cannot add random metadata into RubyGems AFAIK.

There is Gem::Specification#metadata, it seems like it could accept any kind of metadata.