New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove CPANRatings #1653

Closed
kraih opened this Issue Mar 2, 2016 · 46 comments

Comments

Projects
None yet
@kraih

kraih commented Mar 2, 2016

Please remove CPANRatings from MetaCPAN, it is a terrible service and does not add any value. Aside from being practically unmaintained and easy to manipulate, it is filled with outdated information, personal attacks, and what should have been bug reports.

Just take a look at this. Yea, maybe it's funny for a moment, but is that really how we want to present the Perl community to the outside world?

@jberger

This comment has been minimized.

Show comment
Hide comment
@jberger

jberger Mar 2, 2016

Contributor

I much prefer the metacpan ++ system. Cream rises to the top without appearing mean.

Contributor

jberger commented Mar 2, 2016

I much prefer the metacpan ++ system. Cream rises to the top without appearing mean.

@oalders

This comment has been minimized.

Show comment
Hide comment
@oalders

oalders Mar 2, 2016

Member

@abh what is the current status of CPAN Ratings? From the discussion in #mojo it seems that hidden malicious reviews are re-appearing.

Member

oalders commented Mar 2, 2016

@abh what is the current status of CPAN Ratings? From the discussion in #mojo it seems that hidden malicious reviews are re-appearing.

@kraih

This comment has been minimized.

Show comment
Hide comment
@abh

This comment has been minimized.

Show comment
Hide comment
@abh

abh Mar 3, 2016

Patches are welcome. I haven’t heard about the “hidden reviews appearing” thing.

Ask

abh commented Mar 3, 2016

Patches are welcome. I haven’t heard about the “hidden reviews appearing” thing.

Ask

@oalders

This comment has been minimized.

Show comment
Hide comment
@oalders

oalders Mar 3, 2016

Member

Cool. Looks like setup instructions are at https://github.com/perlorg/perlweb

Member

oalders commented Mar 3, 2016

Cool. Looks like setup instructions are at https://github.com/perlorg/perlweb

@kraih

This comment has been minimized.

Show comment
Hide comment
@kraih

kraih Mar 3, 2016

The hidden reviews reappearing thing is just someone manipulating the votes a little i assume. Which is extremely easy, since "Yes" votes are valued much higher than "No" votes, and actual human participation in voting is usually very low.

kraih commented Mar 3, 2016

The hidden reviews reappearing thing is just someone manipulating the votes a little i assume. Which is extremely easy, since "Yes" votes are valued much higher than "No" votes, and actual human participation in voting is usually very low.

@kraih

This comment has been minimized.

Show comment
Hide comment
@kraih

kraih Mar 3, 2016

Honestly, i never understood why it is featured so prominently on MetaCPAN.
cpanratings
Sure, i could understand a link in the sidebar, but those stars carry a lot of weight right now.

kraih commented Mar 3, 2016

Honestly, i never understood why it is featured so prominently on MetaCPAN.
cpanratings
Sure, i could understand a link in the sidebar, but those stars carry a lot of weight right now.

@oalders

This comment has been minimized.

Show comment
Hide comment
@oalders

oalders Mar 3, 2016

Member

My strong preference would be to fix any outstanding issues in CPAN Ratings, especially given that @ask is open to receiving patches. That said, the ad hominems do concern me. Perhaps we could allow distributions to opt out of the ratings being displayed using some key in META.json, similar to the way we now display the IRC banner for distributions which list an IRC channel.

Member

oalders commented Mar 3, 2016

My strong preference would be to fix any outstanding issues in CPAN Ratings, especially given that @ask is open to receiving patches. That said, the ad hominems do concern me. Perhaps we could allow distributions to opt out of the ratings being displayed using some key in META.json, similar to the way we now display the IRC banner for distributions which list an IRC channel.

@kraih

This comment has been minimized.

Show comment
Hide comment
@kraih

kraih Mar 3, 2016

Would definitely opt out. 👍

kraih commented Mar 3, 2016

Would definitely opt out. 👍

@abh

This comment has been minimized.

Show comment
Hide comment
@abh

abh Mar 3, 2016

The hidden reviews reappearing thing is just someone manipulating the votes a little i assume. Which is extremely easy since "Yes" votes are valued much higher than "No" votes.

What reviews are you talking about?

abh commented Mar 3, 2016

The hidden reviews reappearing thing is just someone manipulating the votes a little i assume. Which is extremely easy since "Yes" votes are valued much higher than "No" votes.

What reviews are you talking about?

@kraih

This comment has been minimized.

Show comment
Hide comment
@kraih

kraih Mar 3, 2016

@abh This ad hominem attack on me from 2011 for example, with 75 "No" votes you'd think it would stay hidden. But in its current form, CPANRatings is nothing but a bullying tool. There is no way to respond and ratings never disappear, no matter how old or inappropriate they are, all you can do is remove your distribution from CPAN.

For a big project like Mojolicious it might not matter much, but imagine you're new to Perl and have just uploaded your very first module. An angry one star rating would be devastating, and discourages incremental improvements.

kraih commented Mar 3, 2016

@abh This ad hominem attack on me from 2011 for example, with 75 "No" votes you'd think it would stay hidden. But in its current form, CPANRatings is nothing but a bullying tool. There is no way to respond and ratings never disappear, no matter how old or inappropriate they are, all you can do is remove your distribution from CPAN.

For a big project like Mojolicious it might not matter much, but imagine you're new to Perl and have just uploaded your very first module. An angry one star rating would be devastating, and discourages incremental improvements.

@abh

This comment has been minimized.

Show comment
Hide comment
@abh

abh Mar 3, 2016

abh commented Mar 3, 2016

@trinitum

This comment has been minimized.

Show comment
Hide comment
@trinitum

trinitum Mar 3, 2016

I personally would miss it. I occasionally find reviews on cpanratings useful. What I don't find useful though is the rating itself, the number of reviews that are rubbish is quite high and as a result calculating the average rating doesn't make much sense. I would suggest to replace stars with a simple link to cpanratings, that way people will actually have to read reviews and decide which of them worth taking into account.

trinitum commented Mar 3, 2016

I personally would miss it. I occasionally find reviews on cpanratings useful. What I don't find useful though is the rating itself, the number of reviews that are rubbish is quite high and as a result calculating the average rating doesn't make much sense. I would suggest to replace stars with a simple link to cpanratings, that way people will actually have to read reviews and decide which of them worth taking into account.

@Smylers

This comment has been minimized.

Show comment
Hide comment
@Smylers

Smylers Mar 4, 2016

Contributor

i could understand a link in the sidebar, but those stars carry a lot of weight right now.

Please keep at least the ratings link and the number of reviews in the sidebar, cos it can be useful to read reviews of a module you haven't encountered before. (I'm quite capable of ignoring personal attacks and checking whether criticisms made in very old reviews still apply.)

The stars in search results may be suggesting that the ratings have more value than they actually do. Removing them from there may be warranted.

Contributor

Smylers commented Mar 4, 2016

i could understand a link in the sidebar, but those stars carry a lot of weight right now.

Please keep at least the ratings link and the number of reviews in the sidebar, cos it can be useful to read reviews of a module you haven't encountered before. (I'm quite capable of ignoring personal attacks and checking whether criticisms made in very old reviews still apply.)

The stars in search results may be suggesting that the ratings have more value than they actually do. Removing them from there may be warranted.

@leejo

This comment has been minimized.

Show comment
Hide comment
@leejo

leejo Mar 4, 2016

I much prefer the metacpan ++ system. Cream rises to the top without appearing mean.

The problem is that cream goes sour. Look at File::Slurp: 59++'s but utterly broken, unmaintained, and full of issues. The problem with both stars in the reviews and ++'s is that they're trying to implement a tl;dr and that simply isn't possible for evaluating a CPAN module. I'd argue that any simple rating system needs to decay those ratings over time, and if users want they can come back and re ++ it when it has decayed.

As for CPANRatings and the reviews, i'd argue there's not much value in the prominence of the stars but the actual reviews can be useful. To make them more relevant hide any reviews that are more than, say, one year before the last release of the module (and mark those reviews as potentially outdated) and remove the stars from the listings. Also an opt-out would probably be useful too.

leejo commented Mar 4, 2016

I much prefer the metacpan ++ system. Cream rises to the top without appearing mean.

The problem is that cream goes sour. Look at File::Slurp: 59++'s but utterly broken, unmaintained, and full of issues. The problem with both stars in the reviews and ++'s is that they're trying to implement a tl;dr and that simply isn't possible for evaluating a CPAN module. I'd argue that any simple rating system needs to decay those ratings over time, and if users want they can come back and re ++ it when it has decayed.

As for CPANRatings and the reviews, i'd argue there's not much value in the prominence of the stars but the actual reviews can be useful. To make them more relevant hide any reviews that are more than, say, one year before the last release of the module (and mark those reviews as potentially outdated) and remove the stars from the listings. Also an opt-out would probably be useful too.

@xdg

This comment has been minimized.

Show comment
Hide comment
@xdg

xdg Mar 4, 2016

Problems with ++: no granularity; ratchets upwards; no explanation of what it means or why

Problem with reviews: get stale; have to be policed

I think most of the time, having CPAN Ratings is a positive. That someone took the time to write one is a signal (positive or negative) that carries more weight than a "++". I would rather try to fix CPAN ratings than eliminate it from metacpan search results.

xdg commented Mar 4, 2016

Problems with ++: no granularity; ratchets upwards; no explanation of what it means or why

Problem with reviews: get stale; have to be policed

I think most of the time, having CPAN Ratings is a positive. That someone took the time to write one is a signal (positive or negative) that carries more weight than a "++". I would rather try to fix CPAN ratings than eliminate it from metacpan search results.

@openstrike

This comment has been minimized.

Show comment
Hide comment
@openstrike

openstrike Mar 4, 2016

I agree entirely with @xdg - keep the ratings and fix any real problems within the CPAN Ratings system.

openstrike commented Mar 4, 2016

I agree entirely with @xdg - keep the ratings and fix any real problems within the CPAN Ratings system.

@ranguard

This comment has been minimized.

Show comment
Hide comment
@ranguard

ranguard Mar 4, 2016

Member

I use MetaCPAN's ++ as a reminder to look at modules (until we implement a 'shortlist' feature), so my ++'s are not always that useful for others!

+1 for keep CPAN Ratings but improve CPAN Ratings if it needs it

Member

ranguard commented Mar 4, 2016

I use MetaCPAN's ++ as a reminder to look at modules (until we implement a 'shortlist' feature), so my ++'s are not always that useful for others!

+1 for keep CPAN Ratings but improve CPAN Ratings if it needs it

@Grinnz

This comment has been minimized.

Show comment
Hide comment
@Grinnz

Grinnz Mar 4, 2016

I don't have much of an opinion on the matter but it seems like most people are agreeing that

  1. the stars should be removed from the main view or at least the search results
  2. it would be nice to allow opting out in metadata
  3. old reviews are often more harmful than they are relevant, so they should be hidden or clearly marked as outdated

Grinnz commented Mar 4, 2016

I don't have much of an opinion on the matter but it seems like most people are agreeing that

  1. the stars should be removed from the main view or at least the search results
  2. it would be nice to allow opting out in metadata
  3. old reviews are often more harmful than they are relevant, so they should be hidden or clearly marked as outdated
@xdg

This comment has been minimized.

Show comment
Hide comment
@xdg

xdg Mar 4, 2016

I don't agree they should be removed from the search results as they help "real" modules stand out from random stuff. If people think it has too much visual weight, then make the stars black instead of gold.

If the stars have to be dropped, then at least keep the number of reviews and links, as having reviews is again a signal of real module that people have used (for good or ill).

xdg commented Mar 4, 2016

I don't agree they should be removed from the search results as they help "real" modules stand out from random stuff. If people think it has too much visual weight, then make the stars black instead of gold.

If the stars have to be dropped, then at least keep the number of reviews and links, as having reviews is again a signal of real module that people have used (for good or ill).

@book

This comment has been minimized.

Show comment
Hide comment
@book

book Mar 4, 2016

Contributor

CPAN Reviens are one of the datapoints I use when picking a module to use. Most of the time it's not enough data to make a decision, but it's still enough to form a preliminary opinion.

In fact, I should say "opinion" rather than "data" when talking about the reviews. Still, I find it interesting to know what others think about a specific distribution. Which brings me to: what about listing the reviews one has made on their author page on metacpan? That would give some context on the reviewer.

I also think older reviews should be given less weight, or even be hidden entirely/deleted, but maybe not before asking the original author to review them (so that they could confirm or revise their opinion).

Contributor

book commented Mar 4, 2016

CPAN Reviens are one of the datapoints I use when picking a module to use. Most of the time it's not enough data to make a decision, but it's still enough to form a preliminary opinion.

In fact, I should say "opinion" rather than "data" when talking about the reviews. Still, I find it interesting to know what others think about a specific distribution. Which brings me to: what about listing the reviews one has made on their author page on metacpan? That would give some context on the reviewer.

I also think older reviews should be given less weight, or even be hidden entirely/deleted, but maybe not before asking the original author to review them (so that they could confirm or revise their opinion).

@ido50

This comment has been minimized.

Show comment
Hide comment
@ido50

ido50 Mar 4, 2016

I mostly agree with @kraih. Reviews are very often for extremely outdated versions, and people do tend to bash into other people because they don't like something about the module.

For example after my first upload of my localization module Wolowitz, I immediately got a zero (or was it one) star review from some extremely annoyed person who simply couldn't stand it I did not namespace the module with "Locale::". I ceded, because I didn't want my new module to start out with a terrible review that had absolutely nothing to do with the module itself. That person had his own view of how CPAN should be structured. An outdated view in my opinion, but it doesn't matter. In retrospect, I should have kept it the way it was and told him to go f*** himself.

Anyway, I still think both reviews and metacpan's ++ systems have a place, but they both have their shortcomings. At the end of the day I think the real problem has less to do with these two systems and more to do with the fact that the Perl community is super silent. This is one of the main reasons Perl no longer has the cool factor of other languages.

ido50 commented Mar 4, 2016

I mostly agree with @kraih. Reviews are very often for extremely outdated versions, and people do tend to bash into other people because they don't like something about the module.

For example after my first upload of my localization module Wolowitz, I immediately got a zero (or was it one) star review from some extremely annoyed person who simply couldn't stand it I did not namespace the module with "Locale::". I ceded, because I didn't want my new module to start out with a terrible review that had absolutely nothing to do with the module itself. That person had his own view of how CPAN should be structured. An outdated view in my opinion, but it doesn't matter. In retrospect, I should have kept it the way it was and told him to go f*** himself.

Anyway, I still think both reviews and metacpan's ++ systems have a place, but they both have their shortcomings. At the end of the day I think the real problem has less to do with these two systems and more to do with the fact that the Perl community is super silent. This is one of the main reasons Perl no longer has the cool factor of other languages.

@xdg

This comment has been minimized.

Show comment
Hide comment
@xdg

xdg Mar 4, 2016

I suspect opinions about the value of CPAN ratings correlate strongly with the number of bad reviews people have received.

I would possibly suggest discounting the opinions of authors and getting feedback from non-author users.

xdg commented Mar 4, 2016

I suspect opinions about the value of CPAN ratings correlate strongly with the number of bad reviews people have received.

I would possibly suggest discounting the opinions of authors and getting feedback from non-author users.

@xsawyerx

This comment has been minimized.

Show comment
Hide comment
@xsawyerx

xsawyerx Mar 4, 2016

Contributor

My relationship with CPAN Ratings is a love/hate. Dancer saw two reviews which were made up (one was under "Su-Shee" and one by the developers of Sinatra). Someone simply put someone else's name and bad-mouthed with no actual relevant content. Some people...

On the other hand, CPAN Ratings can also be useful. I don't view the first referenced review as BS, to be honest. It reflects the current state. MLEHMANN decided not to apply patches and forked perl instead. It's exact and accurate. It did not attack Marc, only reflect what happened.

The MetaCPAN ++ button has similar problems and brings its own:

  • You don't actually know who provided a ++ in many cases. This can easily be gamed. Easily.
  • Most people don't use the ++ anyway. (OTOH, many don't provide ratings nowadays either.)
  • Many ++ to just make a note of it ("Why did you favorite that Tweet? Do you agree with it?" "No, I just wanted to find it again later." - This happens with ++ too.)
  • You have no knowledge of why. There is no actual "review" going on there. Do you ++ because it's good? Because you use it? Because it's cool? Because you want to find it later? There is no room to provide more in-depth opinion and opinion on a module from someone who reviews it objectively matters.

Despite being on the abused end (at least twice by now) by made up BS reviews, I still think CPAN Ratings has value. I would like to see it monitored and have reviews flagged and checked for authenticity, added value, and objectivity. Provided that, I believe it serves a good purpose.

The only remaining problem are reviews which are too old and no longer reflect the current state. Perhaps there should be a mechanism for an author to rebuke a review and allow them to prove it wrong. I know we're getting into a difficult situation, but this, IMHO, would provide the most value.

Oh, and in case anyone thinks I was exaggerating about the abuse we received at Dancer by some anonymous poo-poo'er, you can read the following: http://www.sinatrarb.com/2011/07/21/sinatra-loves-dancer.html.

Contributor

xsawyerx commented Mar 4, 2016

My relationship with CPAN Ratings is a love/hate. Dancer saw two reviews which were made up (one was under "Su-Shee" and one by the developers of Sinatra). Someone simply put someone else's name and bad-mouthed with no actual relevant content. Some people...

On the other hand, CPAN Ratings can also be useful. I don't view the first referenced review as BS, to be honest. It reflects the current state. MLEHMANN decided not to apply patches and forked perl instead. It's exact and accurate. It did not attack Marc, only reflect what happened.

The MetaCPAN ++ button has similar problems and brings its own:

  • You don't actually know who provided a ++ in many cases. This can easily be gamed. Easily.
  • Most people don't use the ++ anyway. (OTOH, many don't provide ratings nowadays either.)
  • Many ++ to just make a note of it ("Why did you favorite that Tweet? Do you agree with it?" "No, I just wanted to find it again later." - This happens with ++ too.)
  • You have no knowledge of why. There is no actual "review" going on there. Do you ++ because it's good? Because you use it? Because it's cool? Because you want to find it later? There is no room to provide more in-depth opinion and opinion on a module from someone who reviews it objectively matters.

Despite being on the abused end (at least twice by now) by made up BS reviews, I still think CPAN Ratings has value. I would like to see it monitored and have reviews flagged and checked for authenticity, added value, and objectivity. Provided that, I believe it serves a good purpose.

The only remaining problem are reviews which are too old and no longer reflect the current state. Perhaps there should be a mechanism for an author to rebuke a review and allow them to prove it wrong. I know we're getting into a difficult situation, but this, IMHO, would provide the most value.

Oh, and in case anyone thinks I was exaggerating about the abuse we received at Dancer by some anonymous poo-poo'er, you can read the following: http://www.sinatrarb.com/2011/07/21/sinatra-loves-dancer.html.

@ido50

This comment has been minimized.

Show comment
Hide comment
@ido50

ido50 Mar 4, 2016

That was actually the only bad review I've ever gotten for one of my modules. All others were very positive. I wear two hats: one an author's hat, the other a user's. As a user I've learned that CPAN reviews are not good indicators of a module's quality. Sometimes they are very confusing. I often see a module with one star, enter to read why this module is so bad, and find the last review is from 2006, but many versions have been released since then. Are these reviews still relevant? How many users have seen this one star and moved on, passing up on that module?

Reviews are important, but If I wanted to gauge a module's quality before trying it, I give documentation and test coverage the greatest importance.

ido50 commented Mar 4, 2016

That was actually the only bad review I've ever gotten for one of my modules. All others were very positive. I wear two hats: one an author's hat, the other a user's. As a user I've learned that CPAN reviews are not good indicators of a module's quality. Sometimes they are very confusing. I often see a module with one star, enter to read why this module is so bad, and find the last review is from 2006, but many versions have been released since then. Are these reviews still relevant? How many users have seen this one star and moved on, passing up on that module?

Reviews are important, but If I wanted to gauge a module's quality before trying it, I give documentation and test coverage the greatest importance.

@xdg

This comment has been minimized.

Show comment
Hide comment
@xdg

xdg Mar 4, 2016

@abh Someone just made me aware of this: http://cpanratings.perl.org/user/matrinapoint

Looks like someone is trying to make a point about CPAN Ratings

xdg commented Mar 4, 2016

@abh Someone just made me aware of this: http://cpanratings.perl.org/user/matrinapoint

Looks like someone is trying to make a point about CPAN Ratings

@kraih

This comment has been minimized.

Show comment
Hide comment
@kraih

kraih Mar 4, 2016

@xdg And, does your opinion about the value of CPANRatings strongly correlate with the number of bad reviews you have received?

kraih commented Mar 4, 2016

@xdg And, does your opinion about the value of CPANRatings strongly correlate with the number of bad reviews you have received?

@oalders

This comment has been minimized.

Show comment
Hide comment
@oalders

oalders Mar 4, 2016

Member

This kind of childishness pushes me in the direction of not removing the ratings. This "matrinapoint" business serves no real purpose other than to annoy. We know that the ratings can be gamed. We don't need a demonstration. We need someone to step up and fix it rather than sit back and make it worse with a one-off script.

Member

oalders commented Mar 4, 2016

This kind of childishness pushes me in the direction of not removing the ratings. This "matrinapoint" business serves no real purpose other than to annoy. We know that the ratings can be gamed. We don't need a demonstration. We need someone to step up and fix it rather than sit back and make it worse with a one-off script.

@ranguard

This comment has been minimized.

Show comment
Hide comment
@ranguard

ranguard Mar 4, 2016

Member

If someone wants to improve the CPAN Ratings algorithm then they should submit a patch here https://github.com/perlorg/perlweb MetaCPAN will continue to use CPAN Ratings as it stands for now.

Member

ranguard commented Mar 4, 2016

If someone wants to improve the CPAN Ratings algorithm then they should submit a patch here https://github.com/perlorg/perlweb MetaCPAN will continue to use CPAN Ratings as it stands for now.

@ranguard ranguard closed this Mar 4, 2016

@kraih

This comment has been minimized.

Show comment
Hide comment
@kraih

kraih Mar 4, 2016

Isn't a kneejerk reaction not to do anything just as childish?

kraih commented Mar 4, 2016

Isn't a kneejerk reaction not to do anything just as childish?

@kraih

This comment has been minimized.

Show comment
Hide comment
@kraih

kraih Mar 4, 2016

The irony in all of this, the very thing that made me start this thread ended up derailing it. 👏

kraih commented Mar 4, 2016

The irony in all of this, the very thing that made me start this thread ended up derailing it. 👏

@ranguard

This comment has been minimized.

Show comment
Hide comment
@ranguard

ranguard Mar 4, 2016

Member

We have decided to do something... keep using CPAN Ratings and recommend that any issue be resolved there.

Member

ranguard commented Mar 4, 2016

We have decided to do something... keep using CPAN Ratings and recommend that any issue be resolved there.

@leejo

This comment has been minimized.

Show comment
Hide comment
@leejo

leejo Mar 4, 2016

It's a shame this conversation has broken down. I may see if i can spend some time to put a PR together in the next couple of weeks if people want to make sensible suggestions. I'm not advocating for sweeping changes, just something that improves the situation. My initial thoughts are the same as i state above: reviews should stay but the prominence of the stars is questionable and older reviews should be hidden and marked as potentially outdated. Whilst @kraih has concerns i understand about bad reviews, i think false positives (or outdated positives) have the potential to be far more damaging.

We were bitten by problems with the review system just this week. A colleague had chosen a module in good faith after looking at the perldoc and reviews (4 stars!), but it turns out that module is fundamentally broken and there is a much better alternative available. Whilst the alternative has a few ++'s it doesn't have any reviews.

This is misleading, but is the nature of a system as large as and with as much history as CPAN. If it can bite, what i like to think are, competent perl developers then what good is it for the newbies?

leejo commented Mar 4, 2016

It's a shame this conversation has broken down. I may see if i can spend some time to put a PR together in the next couple of weeks if people want to make sensible suggestions. I'm not advocating for sweeping changes, just something that improves the situation. My initial thoughts are the same as i state above: reviews should stay but the prominence of the stars is questionable and older reviews should be hidden and marked as potentially outdated. Whilst @kraih has concerns i understand about bad reviews, i think false positives (or outdated positives) have the potential to be far more damaging.

We were bitten by problems with the review system just this week. A colleague had chosen a module in good faith after looking at the perldoc and reviews (4 stars!), but it turns out that module is fundamentally broken and there is a much better alternative available. Whilst the alternative has a few ++'s it doesn't have any reviews.

This is misleading, but is the nature of a system as large as and with as much history as CPAN. If it can bite, what i like to think are, competent perl developers then what good is it for the newbies?

@oalders

This comment has been minimized.

Show comment
Hide comment
@oalders

oalders Mar 5, 2016

Member

@leejo I think one very simple way to solve part of the problem would be for CPAN Ratings to export all of the actual ratings in a large dump (JSON?) as opposed to http://cpanratings.perl.org/csv/all_ratings.csv That would allow us to say "X reviews in last Y months" rather than just "X reviews".

This really does need to be addressed at the source, but if CPAN Ratings were to publish more information, with appropriate datestamps, then the consumers could choose what they wanted to do with it.

From what I gather it's all in a MySQL database, so running a dump cron every few hours or once per day shouldn't be complicated.

That doesn't solve the problem as a whole, but it starts to make the available data easier to work with.

Member

oalders commented Mar 5, 2016

@leejo I think one very simple way to solve part of the problem would be for CPAN Ratings to export all of the actual ratings in a large dump (JSON?) as opposed to http://cpanratings.perl.org/csv/all_ratings.csv That would allow us to say "X reviews in last Y months" rather than just "X reviews".

This really does need to be addressed at the source, but if CPAN Ratings were to publish more information, with appropriate datestamps, then the consumers could choose what they wanted to do with it.

From what I gather it's all in a MySQL database, so running a dump cron every few hours or once per day shouldn't be complicated.

That doesn't solve the problem as a whole, but it starts to make the available data easier to work with.

@abh

This comment has been minimized.

Show comment
Hide comment
@abh

abh Mar 5, 2016

Maybe supporting /dist/FooBar.json to return the reviews as JSON?

Again, patches welcome. :-)

Ask

abh commented Mar 5, 2016

Maybe supporting /dist/FooBar.json to return the reviews as JSON?

Again, patches welcome. :-)

Ask

@ap

This comment has been minimized.

Show comment
Hide comment
@ap

ap Mar 7, 2016

Contributor

I’m with @Smylers:

The stars should go away.

Keep the link to CPAN Ratings, by all means – there are some very useful writeups in there. But the stars are completely meaningless. The number of reviews is a much more salient data point. And even that probably needs to be augmented with further qualifications, but that would be a start.

Contributor

ap commented Mar 7, 2016

I’m with @Smylers:

The stars should go away.

Keep the link to CPAN Ratings, by all means – there are some very useful writeups in there. But the stars are completely meaningless. The number of reviews is a much more salient data point. And even that probably needs to be augmented with further qualifications, but that would be a start.

@karenetheridge

This comment has been minimized.

Show comment
Hide comment
@karenetheridge

karenetheridge Mar 8, 2016

Contributor

I'm in favour of removing the link to CPANRatings. It is fundamentally broken by design and I don't see how it can be fixed at this point -- since reviews are anonymous, they can say anything at all without justification; if the review was actually a poorly-written bug report, there is no way for the author (or anyone else) to respond back to the reviewer to request more information or state that a fix is now available. Reviews do not decay, so a bad review on an early alpha will live on even after the distribution has become more robust.

Contributor

karenetheridge commented Mar 8, 2016

I'm in favour of removing the link to CPANRatings. It is fundamentally broken by design and I don't see how it can be fixed at this point -- since reviews are anonymous, they can say anything at all without justification; if the review was actually a poorly-written bug report, there is no way for the author (or anyone else) to respond back to the reviewer to request more information or state that a fix is now available. Reviews do not decay, so a bad review on an early alpha will live on even after the distribution has become more robust.

@ap

This comment has been minimized.

Show comment
Hide comment
@ap

ap Mar 8, 2016

Contributor

If the point of linking to CPAN Ratings was to let people read the reviews, these issues wouldn’t matter that much. You can recognise a misdirected bug report upon reading (and the author can post a rating-less non-review to respond, so there is a sucky way of responding, just barely good enough for the purposes of other people reading the reviews). Reviews prominently state the version that was current at the time of writing. And re anonymity, well same with reviews on Amazon etc., except here you can cross-check with the “product” to an extent that online shopping cannot attain. Etc. All the weaknesses of the service are manageable if the user is looking at the reviews page.

They’re still weaknesses, but they’re much less bad that way. It’s just promoting the rating average to a very prominent position that is grossly unwarranted.

All that said, I would not be sad to see the link go entirely. But it looks like that might not happen – in which case getting away from the ratings average is of the essence. I would be fine with either solution.

Contributor

ap commented Mar 8, 2016

If the point of linking to CPAN Ratings was to let people read the reviews, these issues wouldn’t matter that much. You can recognise a misdirected bug report upon reading (and the author can post a rating-less non-review to respond, so there is a sucky way of responding, just barely good enough for the purposes of other people reading the reviews). Reviews prominently state the version that was current at the time of writing. And re anonymity, well same with reviews on Amazon etc., except here you can cross-check with the “product” to an extent that online shopping cannot attain. Etc. All the weaknesses of the service are manageable if the user is looking at the reviews page.

They’re still weaknesses, but they’re much less bad that way. It’s just promoting the rating average to a very prominent position that is grossly unwarranted.

All that said, I would not be sad to see the link go entirely. But it looks like that might not happen – in which case getting away from the ratings average is of the essence. I would be fine with either solution.

@leejo

This comment has been minimized.

Show comment
Hide comment
@leejo

leejo Mar 8, 2016

@abh - see PR above, any suggestions welcome.

leejo commented Mar 8, 2016

@abh - see PR above, any suggestions welcome.

@Grinnz

This comment has been minimized.

Show comment
Hide comment
@Grinnz

Grinnz Jun 27, 2016

Now that the above functionality has been implemented in CPANratings (see example: http://cpanratings.perl.org/dist/Moo.json) is there any interest in updating metacpan's rating display to be more useful?

Edit: The top level "ratings" could be used directly, either "all" (which averages all helpful reviews only) or "recent" (which averages helpful reviews since the last release up to 1 year ago, it seems).

For this example, the average rating displayed for Moo should be nothing, because there are no reviews anywhere near relevant to the current version, or at best, one from almost two years ago.

Grinnz commented Jun 27, 2016

Now that the above functionality has been implemented in CPANratings (see example: http://cpanratings.perl.org/dist/Moo.json) is there any interest in updating metacpan's rating display to be more useful?

Edit: The top level "ratings" could be used directly, either "all" (which averages all helpful reviews only) or "recent" (which averages helpful reviews since the last release up to 1 year ago, it seems).

For this example, the average rating displayed for Moo should be nothing, because there are no reviews anywhere near relevant to the current version, or at best, one from almost two years ago.

@jberger

This comment has been minimized.

Show comment
Hide comment
@jberger

jberger Jun 27, 2016

Contributor

I wonder how many reviews are not outdated? Also, I know that a major complaint was the weighting of helpful/unhelpful in the show/hide algorithm. Has that been updated?

Contributor

jberger commented Jun 27, 2016

I wonder how many reviews are not outdated? Also, I know that a major complaint was the weighting of helpful/unhelpful in the show/hide algorithm. Has that been updated?

@oalders oalders reopened this Jun 28, 2016

@oalders

This comment has been minimized.

Show comment
Hide comment
@oalders

oalders Jun 28, 2016

Member

If someone wanted to tweak how we handle the ratings, that would be nice. Right now we download a CSV and process that. If we switch to using the API then it might be a lot more overhead, but someone would need to sit down and have a good look at it.

Member

oalders commented Jun 28, 2016

If someone wanted to tweak how we handle the ratings, that would be nice. Right now we download a CSV and process that. If we switch to using the API then it might be a lot more overhead, but someone would need to sit down and have a good look at it.

@Astara

This comment has been minimized.

Show comment
Hide comment
@Astara

Astara Sep 9, 2016

People can talk about fixing reviews, or think they might be helpful, but if they don't even say what version they go with, what's the point? A review about something 40 versions ago isn't very helpful, but someone reading the reviews and ratings wouldn't know that. Also -- no way to answer a review -- you can post another review, but I've seen 'answer reviews' to ratings-spammers be voted down to invisibility so the answer is never seen. Not at all helpful.

Astara commented Sep 9, 2016

People can talk about fixing reviews, or think they might be helpful, but if they don't even say what version they go with, what's the point? A review about something 40 versions ago isn't very helpful, but someone reading the reviews and ratings wouldn't know that. Also -- no way to answer a review -- you can post another review, but I've seen 'answer reviews' to ratings-spammers be voted down to invisibility so the answer is never seen. Not at all helpful.

@ap

This comment has been minimized.

Show comment
Hide comment
@ap

ap Sep 9, 2016

Contributor

People can talk about fixing reviews, or think they might be helpful, but if they don't even say what version they go with, what's the point?

That would be a fair point if the version weren’t automatically recorded, which it is.

A review about something 40 versions ago isn't very helpful, but someone reading the reviews and ratings wouldn't know that.

How can they not? How much more prominent can it be? cpanratings-versions

I've seen 'answer reviews' to ratings-spammers be voted down to invisibility so the answer is never seen. Not at all helpful.

Fair point, that’s a problem.

Contributor

ap commented Sep 9, 2016

People can talk about fixing reviews, or think they might be helpful, but if they don't even say what version they go with, what's the point?

That would be a fair point if the version weren’t automatically recorded, which it is.

A review about something 40 versions ago isn't very helpful, but someone reading the reviews and ratings wouldn't know that.

How can they not? How much more prominent can it be? cpanratings-versions

I've seen 'answer reviews' to ratings-spammers be voted down to invisibility so the answer is never seen. Not at all helpful.

Fair point, that’s a problem.

@Astara

This comment has been minimized.

Show comment
Hide comment
@Astara

Astara Sep 9, 2016

Aristotle Pagaltzis wrote:

People can talk about fixing reviews, or think they might be
helpful, but if they don't even say what version they go with,
what's the point?

That would be a fair point if the version weren’t automatically
recorded, which it is.

A review about something 40 versions ago isn't very helpful, but
someone reading the reviews and ratings wouldn't know that.

How can they not? How much more prominent can it be?
cpanratings-versions
https://cloud.githubusercontent.com/assets/1683/18379796/97c2736e-7674-11e6-9309-472875e87eea.png

I don't recall that ever standing out -- if I saw it, I don't know what
it was supposed to be. It doesn't say it's a version number that the review
applied to. Would it be too much to say "This review was for version
xx.yy.zz?"

Also when I've reviewed something or responded to a review, I was never
asked
what version it applied to.

I've seen 'answer reviews' to ratings-spammers be voted down to
invisibility so the answer is never seen. Not at all helpful.

Fair point, that’s a problem.


There is more than one problem there.  One is voting down "responses" by

the module author -- that's not very useful. Two is the inability to "close
out" bug report or feature deficit comments when they no longer apply. And
the third is reviewing author instead of the technical aspects of the
module.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#1653 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIwTcBmzPgWREmTf5YXcPFiJ-1wZy9Tks5qoRPQgaJpZM4Hn72L.

Astara commented Sep 9, 2016

Aristotle Pagaltzis wrote:

People can talk about fixing reviews, or think they might be
helpful, but if they don't even say what version they go with,
what's the point?

That would be a fair point if the version weren’t automatically
recorded, which it is.

A review about something 40 versions ago isn't very helpful, but
someone reading the reviews and ratings wouldn't know that.

How can they not? How much more prominent can it be?
cpanratings-versions
https://cloud.githubusercontent.com/assets/1683/18379796/97c2736e-7674-11e6-9309-472875e87eea.png

I don't recall that ever standing out -- if I saw it, I don't know what
it was supposed to be. It doesn't say it's a version number that the review
applied to. Would it be too much to say "This review was for version
xx.yy.zz?"

Also when I've reviewed something or responded to a review, I was never
asked
what version it applied to.

I've seen 'answer reviews' to ratings-spammers be voted down to
invisibility so the answer is never seen. Not at all helpful.

Fair point, that’s a problem.


There is more than one problem there.  One is voting down "responses" by

the module author -- that's not very useful. Two is the inability to "close
out" bug report or feature deficit comments when they no longer apply. And
the third is reviewing author instead of the technical aspects of the
module.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#1653 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIwTcBmzPgWREmTf5YXcPFiJ-1wZy9Tks5qoRPQgaJpZM4Hn72L.

@ranguard

This comment has been minimized.

Show comment
Hide comment
@ranguard

ranguard Apr 26, 2017

Member

Search results lost the stars a while ago, kept in left nav of the releases

Member

ranguard commented Apr 26, 2017

Search results lost the stars a while ago, kept in left nav of the releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment