Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flag abandoned projects #7

Open
brodock opened this issue Jul 17, 2017 · 23 comments
Open

Flag abandoned projects #7

brodock opened this issue Jul 17, 2017 · 23 comments

Comments

@brodock
Copy link

@brodock brodock commented Jul 17, 2017

Can we have a feature to flag abandoned projects? By abandoned I mean anything that haven't been updated in X years (1 or 2 for example).

This can be a valuable search filter as people would normally want to exclude unmaintained/abandoned projects from results (while others supporting legacy, may be interested in this specific projects)

@NicolasWebDev
Copy link

@NicolasWebDev NicolasWebDev commented Jul 17, 2017

Maybe it would be better to have two separate functionalities:

  1. see when was the last commit. This is a really needed and natural feature to have.
  2. let users flag a project as abandoned. This would enable to really find out what are the abandoned projects. If in the last 5 years, their was a unique commit, 3 months ago, the gem should be certainly still flagged as abandoned.

Loading

@brodock
Copy link
Author

@brodock brodock commented Jul 17, 2017

I agree that it should be manually flagged, but relying on commits may not be the best solution as people may be just updating the README file to say it's abandoned / unmaintained. Relying on gem release data makes more sense.

Loading

@NicolasWebDev
Copy link

@NicolasWebDev NicolasWebDev commented Jul 17, 2017

I guess I was not clear in my proposition ^^. I agree with you @brodock , abandoned projects should be flagged by users (that will know of the project's real state) and not automatically (relying on commit information).

Loading

@FranklinYu
Copy link

@FranklinYu FranklinYu commented Jul 19, 2017

How about both manually and automatically? We can have a "deprecated" badge depending on user's flag, as well as a "last release X years/months ago" which gets red font when it is over N years. The "deprecated" flag can have a tooltip "flagged by 35 users as deprecated".

Loading

@hmistry
Copy link

@hmistry hmistry commented Oct 4, 2017

What if I had a gem that's not needed any update (i.e. features frozen and magically bug-free for 5+ yrs) but it's actively being used? It should not be flagged as abandoned. It's challenging to distinguish an abandoned gem vs a useful gem that's feature frozen, bug free, and low volume.

I think the last commit metric is one indicator, and monthly downloads is another good metric to have. The latter will allow one to see that even though last commit is old, the gem is actively being used. It might be good to distinguish a gem that always had an average of say 10 downloads per month vs a gem that had millions downloads per month at it's peak but has now dropped to hundreds of downloads. The former gem has not been abandoned while the latter is being abandoned as users move to other solutions.

If we give control to users to flag "abandoned" gems, we should consider how can we prevent malintent e.g some people hate the gem and flag it or there's gem wars between maintainers? I think an "abandoned" gem that works is ok to use if it meets your needs and we should avoid discouraging it's use in the UI. With that said, it's better to track if the gem that hasn't been updated suddenly stops working on the latest rubies and no one is fixing it. Now I know this is not practical, but it's methodology makes a better indicator as an example - perhaps this can be achieved another way...

cc @colszowka

Loading

@FranklinYu
Copy link

@FranklinYu FranklinYu commented Oct 4, 2017

@hmistry I think we can check open issue counts. Most "features frozen and magically bug-free" (if there is any) gem should not have any open issues, since issues indicate "things to be done".

Loading

@hmistry
Copy link

@hmistry hmistry commented Oct 4, 2017

@FranklinYu Not necessarily as people also use issues for solving their "how do I do this" questions or "my (wrong) implementation does not work" - basically think Stackoverflow support type questions. Also some maintainers don't bother clearing their open issues... one can't assume open issues = things to be done.

My point is let's avoid drawing any conclusions from data like "this gem is abandoned or open issues = things to be done", but simply present the raw relevant data to the user and let them investigate and draw their own conclusions about the gems.

Loading

@colszowka
Copy link
Member

@colszowka colszowka commented Oct 5, 2017

Hey everyone, some thoughts on this can be found in rubytoolbox/catalog#2 (comment)

Loading

@weedySeaDragon
Copy link

@weedySeaDragon weedySeaDragon commented Oct 10, 2017

I agree that issue counts are not a reliable indicator of the state of a gem. Different maintainers use the issue queue differently. Repos that allow anyone to file an issue will also have a wide variety of types and uses of issues.

However, how long since the code was last touched is a good indicator as to whether a gem has been abandoned. This is called "staleness" as in "how stale is this code?"

@hmistry You presented this scenario:

What if I had a gem that's not needed any update (i.e. features frozen and magically bug-free for 5+ yrs) but it's actively being used?

Even if there is a gem out there that has been "perfect" for 5 years and so the code hasn't been touched, it is certainly going to still be stale due to incompatibilities, security, etc. So it's still a valid criteria.

(And I challenge you to find a real gem that supports your scenario. :-) )

If we have flagged a gem as "stale," given some reasonable criteria, then there's no problem with the semantics of whether it's been totally 'abandoned' or not.

Having this data presented to me 'raw' would not be helpful. (And by "raw" I mean just as another piece of text in the details about a gem.) The whole point of a tool such as this is to provide value above and beyond raw data. There are plenty of other places/tools that simply aggregate raw data about gems and code. (Not to mention it's straightforward to gather that yourself.) Forgive me if we're thinking the same thing despite the word "raw." Clearly I have strong opinions and experiences about this.

Any critical piece of information should be quick to visually ingest, and this is one of them. It should be a simple visual piece (chunk) presented as an icon, or an easily discernible part of a chart or graph, etc. Visuals are quickly ingested, words are slower.

Loading

@hmistry
Copy link

@hmistry hmistry commented Oct 10, 2017

@weedySeaDragon I liked your feedback, thanks for sharing.

I like your word choice of staleness and I agree it's hard-pressed for a gem to survive long without issues especially complex functionality gems. To be clear, I'm not thinking of the case like cancan vs cancancan gems - for this I'm with all of you - show that cancancan is the better choice and cancan is no longer popular.

I'd like to see a differentiation between a gem maintainer saying "don't use this gem, it's been deprecated" vs a perfectly good gem that has no visible activity on it. The challenge is how to differentiate a bad stale gem vs a good gem that has no activity on it. If RubyToolbox flags them both as stale or unmaintained or doesn't show them, then I see it as incorrectly labeling the good gem (for example a specific math gem or a parser for circuit netlist or something very niche).

To your other point - by raw, I'm not saying present some form of a "CSV dump" to the user... oh no. By all means, we should present the data in charts, visuals, tables, etc that makes it easy to read and understand the data.

If we take the data and for user value say we do a blanket thing like for example, anything with less than 100 downloads is dead or has no activity for 1 year is stale or use a scoring system to rank the gems, then I question the accuracy of it and therefore its true value. For example, Cocoapods has a quality factor (and ruby toolbox had a score if I remember correctly) but I've rarely used it to decide whether that library meets my needs because it's scoring is different from how I'd score it in that moment. If scores and blanket rules works for you, then we're different people and it's fine... I understand it simplifies things. Then I'd like to have the option to turn off the scoring/rules and let me decide on the metrics.

The scoring system has tradeoffs and lacks a personalization to a users intent in that moment. If I'm looking for an authentication gem, then my weighting on security, downloads, issues closure, etc will be very high. If I'm looking for say a parser for circuit netlist (assuming it exists), then my weighting on security, downloads, or how long ago it was written won't be of high concern. The other issue of a scoring system is that popular gems become a self-fulfilling thing and even though some one introduces a better gem, users may never give it a chance because it lacks the activity or popularity or high score. If there's a way to solve these in a scoring system, I'm open to it. I guess I don't want any gem to get left off the list because of a bad scoring system or it got incorrectly labeled due to some general rule.

Last thing I'd like to mention for the discussion to separate the database catalog of gems from the UI presentation. I think RubyToolbox should have a complete database catalog of all gems if possible. However the UI part needs to be user friendly and present things in a digestible manner so using filters, sorting, pagination, charts, visuals, etc are very important and if one wishes to sit there and paginate through the complete list of gems in a category, they should be able to do it (I've done it before).

I hope this clarifies where I feel we should draw the line on what is "abandoned/stale" projects. It's only my opinions based on my experience of using RubyToolbox, Cocoapods, and the likes. However I acknowledge we're all different and neither is right or wrong, but in sharing our opinions, we can hopefully come up with a solution that caters to all.

Loading

@weedySeaDragon
Copy link

@weedySeaDragon weedySeaDragon commented Oct 10, 2017

@hmistry What a great response. Thank you. I really appreciate your diplomacy and your ability to make your thoughts and viewpoint clear. I think we're very much in agreement. Which is nice, but -- as you stated well -- is not necessary by any means. :-)

Here are my thoughts on your key points. (I'm trying to format them so they're clear to all. )


  • "Stale" is a candidate for a critical score

  • Gem owners should be able to add info about a score

Using some score for "stale" is important. And I think you've hit upon an important feature: gem owners need to be able to provide information specific to a RubyToolbox 'score' or 'flag' so that they can provide information as to why that score is either 'wrong' or that there is some important context/info that RubyToolbox site visitors should know.

Ex:

Scenario:

  • RubyToolbox has a score for "staleness" based on how long ago the gem was released. Anything older than 3 years is marked as "stale" (perhaps it'd really be based on more data, but just for the sake of this example.)
  • The gem "blorfo" was last released April 1st, 2014
  • the gem owner knows that this gem is still widely and actively used and wants people to know

RubyToolbox Feature:

  • The gem owner can enter a comment about the staleness score. The score will show up with some sort of indicator (an asterisk + color or other formatting?) so folks will understand that this gem is very much alive and active and well

  • A single score isn't helpful. Users need to see the overall pattern for a gem

I also agree that showing a single score for a gem isn't very helpful. Showing each of the "scores" (or whatever) for the "main criteria" is what is helpful because that way, each site visitor can determine what gems are best for their particular needs.
Perhaps someone doesn't care about 'staleness' but instead is interested in 'number of downloads.' They can quickly see (and sort, hopefully) based on the criteria(-ion) that is important to them.

In the world of charts, a "spider graph" shows this information most succinctly. However, they turn out to be both ugly and not easy to understand for those not used to them.

Perhaps a simple bar chart can be used so people can see how each criteria score and the overall pattern for a gem.
Imagine there are 2 gems, one named "Blorfo" and a gem named "Foop". And that RubyToolbox by default will show these criteria in a little 'summary chart': staleness, activity, # downloads, mimmz, and blarg. ("mimmz" and "blarg" are just nonesense words for some other criterion scores)

Here's what a user might see for the 2 gems:

rubytoolbox-gems-score-barchart-ex

A user can quickly see which criteria are 'low' or 'high'.

Perhaps there's some criterion that doesn't lend itself to a numerical score. It might be represented with some icon that goes with/on the chart.

(I don't know why that chart has a big gap in the left. Definitely my problem. I just tossed one together quickly as an example.)

(Perhaps a user could choose which criteria are shown in the summary sparkline or chart or whatever visual indicator.)


  • A user should be able to see all of the information about a gem
    And I totally agree that the visual display of information should be separate from the data. (That's good design/architecture.) There should be as much data as possible. Users should always be able to view all of the details possible if they want to do so.

What do others think?

Loading

@hmistry
Copy link

@hmistry hmistry commented Oct 11, 2017

@weedySeaDragon I like this direction. Your bar chart is a great example and we can add filter options for each criteria when listing gems. I think this is a great way to start if we don't have solid understanding on all the different user preferences because we can collect the analytics for filter options in searches and then use that later on to build a better recommendation algorithm or fine tune our criteria(ions).

I think in another issue we should list all the different data that can be collected for gems e.g. downloads, loc, issues, pr, etc. And also list what types of data would greatly help RubyToolbox e.g. like the gem is deprecated. This can help ideate what informatics we can create or identify what data would be really handy to have.

Loading

@weedySeaDragon
Copy link

@weedySeaDragon weedySeaDragon commented Oct 11, 2017

I think issues #4 and #5 are where the ideas about 'what data' are percolating.

Loading

@Schwad
Copy link

@Schwad Schwad commented Nov 26, 2017

My two cents - sorry if it's a repetition here - we have tooling available that 'Grades' gems out there based on such things as staleness and maintenance support. Would be nice to bring something along those lines within a view.

There are times I don't care how old a gem is or about its maintenance if it works - but if it is becoming a mission critical component of a money-reliant application, knowing how stale a gem is would be incredibly helpful before working with it. Loving the conversation happening here :)

Loading

@Schwad
Copy link

@Schwad Schwad commented Nov 26, 2017

My comment is redundant and covered in much more length here: #4

Loading

@GBH
Copy link

@GBH GBH commented Feb 9, 2018

Is it still an option to make a PR that removes stuff from catalog?

Some projects are pretty obviously abandoned by author and users: Rails 3 is the last supported version, no commits in last 5 years, pile of issues where the last one is asking for Rails 4 support in 2013. This is not a bug-free project that didn't need updates for years and still being widely used.

Loading

@colszowka
Copy link
Member

@colszowka colszowka commented May 11, 2018

@GBH Yes, definetely it's still an option. I'm however still thinking about "better" ways to do this, i.e. manually flagging projects as dead, using github archived repo flag or detecting from the project data itself, and then displaying them less-prominent, or with a clear warning etc. I haven't really reached a conclusion what to do best here, but it's an ongoing topic that needs to be dealt with eventually :)

Loading

@brodock
Copy link
Author

@brodock brodock commented May 12, 2018

I think there is value against removing stuff. Not everyone is living on the bleeding edge. We could "hide" stale/unmaintained gems etc, but keep then available under a separate filter etc.

It is still valid use case to be able to search on google for something that is compatible with "rails 3" and get results or something that works in ruby 1.8.7, etc (just for the sake of few examples).

Loading

@Schwad
Copy link

@Schwad Schwad commented May 12, 2018

Loading

@janko
Copy link

@janko janko commented May 10, 2019

I've opened some PRs on the catalog repo for removing "outdated" gems (e.g. rubytoolbox/catalog#278), and was directed here.

My main question: is Ruby Toolbox meant to keep track of all possible gems that fit a given category, or only the "actively maintained & popular" ones like this repository's description says (like awesome-ruby)? My understanding is that it's the latter.

Currently, one big downside of Ruby Toolbox for me is the outdated content, which hurts its usefulness.

  1. There is a big amount of outdated/deprecated/obsolete gems, some even explicitly state that in their READMEs (e.g. Tire, girl_friday).
  2. There is a redundancy with gem extensions being listed as well (e.g. elasticsearch + elasticsearch-rails, sunspot + sunspot_solr, resque + resque-mongoid). IMO that introduces a slippery slope, because it's not clear how would we decide which extensions are worth adding (some gems like CarrierWave have a lot of extensions, so it's not feasible to add all of them). That can be a separate discussion, though.

Showing "outdated" gems can make it hard to find the good stuff. For example, in the Background Jobs category, the RocketJob gem is a nice modern alternative to Sidekiq, but it's currently buried underneath outdated and unmaintained gems.


Related to this discussion, I think it would be difficult to design a system which determines whether a gem is outdated and should be flagged/removed from Ruby Toolbox, I think it still needs to be a manual process, at least for now.

It's fine if we separated these gems into an "outdated" list (as opposed to removing them), as suggested by @mediafinger in rubytoolbox/catalog#2 (comment). I don't think Ruby Toolbox needs to be showing them, at least not by default. I would start with not showing them, and then later possibly add the checkbox filter for including them in the list.

name: Ruby Version Management
description:
projects:
  - postmodern/chruby
  - postmodern/ruby-install
  - rvm/rvm
  - senny/rvm.el
  - sstephenson/rbenv
  - sstephenson/ruby-build
outdated_projects:
  - hmans/rbfu
  - phoet/which_ruby
  - vertiginous/pik

The idea of additionally including the rationale and replacement, suggested by @colszowka in rubytoolbox/catalog#2 (comment), is also interesting. But that also introduces more work for anyone "outdating" the gem on Ruby Toolbox, especially the "replacement" part. I think simply moving outdated gems to a separate list inside the YAML file is more than feasible.


I think that the outdated/abandoned projects significantly hurt usefulness of Ruby Toolbox, so we should start flagging those sooner rather than later. I started identifying those already (to illustrate, the "Background Jobs" section has been reduced from 40 to 13 gems):

If it's ok with you, I can move those into the outdated_projects section in the YAML (instead of removing them as I did currently).

Loading

@colszowka
Copy link
Member

@colszowka colszowka commented Jul 28, 2019

Thanks for your input @janko!

My main question: is Ruby Toolbox meant to keep track of all possible gems that fit a given category, or only the "actively maintained & popular" ones like this repository's description says (like awesome-ruby)? My understanding is that it's the latter.

Ideally as already stated before, imho it's a good thing to also see "outdated" libraries as they still might provide value. I do see the point of more up to date, upcoming libraries gettting buried though. To generally indicate the healthiness of a library more clearly I introduced the project health indicators a while ago

In order to not lose the previous categorization of a library by removing it from the catalog altogether, I think it would be much better to lean on the project health tags and provide some filtering capabilities on top of those, but then again we have #477 which is in a somewhat conflicting position to this approach.

Regarding the topic of mere "adapters" or "plugins": Yes, probably those don't belong into the catalog, at least not for general-purpose categories. My take on this would be that in general the library should have "standalone" functionality for it's purpose. If extensions exist (and the library is popular enough) then it might make sense to add a dedicated extensions / plugins category - we do have this for example for active record connection adapters.

I probably should try to write down some guidelines regarding this so there's a central place to refer back to / discuss when editing the catalog, especially with regard to the cleanups that are definetely neccessary - it's a ton of work though, so I'm trying to make things work automated based on the data as much as possible.

Loading

@skatkov
Copy link

@skatkov skatkov commented Jan 23, 2020

@weedySeaDragon

Even if there is a gem out there that has been "perfect" for 5 years and so the code hasn't been touched, it is certainly going to still be stale due to incompatibilities, security, etc. So it's still a valid criteria.

(And I challenge you to find a real gem that supports your scenario. :-) )

We had already such a discussion before, about gems that "look abandoned", but in really they just still work. With bunch of examples too -> #477

This seems like a continuation of this discussion to me.

@colszowka seems like non of possible solutions have been accepted in that thread by you. any chance you can combine these issues and find a single solution?

Loading

@colszowka
Copy link
Member

@colszowka colszowka commented Jul 31, 2020

@skatkov Yes. Although I have to say that I still have to go through the huge amount of feedback given in #477 and try to find a reasonable solution.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants