-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
List of route icons at each grade has gone screwy, related to stars changes #2466
Comments
https://www.thecrag.com/climbing/australia/new-south-wales-and-act My guess is that it looks like our recent change in how routes are starred with no minimum ascents has effected the scoring. |
I believe it's the expected behavior, but even just a minimum ascent threshold (5?10?) would help. That list also has some interesting capitalization. Sometimes it displays correctly, others it's lowercase for a climb, despite the climb title itself being capitalised correctly. I haven't managed to narrow it down further... may be related to whether you are at an area/crag level |
So some thoughts here: Do we @scd is the stars article actually accurate for the current new stars algorithm? https://www.thecrag.com/article/Stars Every time I touch this I have to re-learn what we are currently actually doing :) The recent discussion here makes me think that we should actually make a small tweak to the stars, so that volume is a weak signal, but also make volume a stronger signal in the icon'icness rating If we only touch the iconicness algorithm then I think a simple down grade of the stars based on volume would suffice, eg if there has been < 10 ticks then remove 2 stars, if < 100 then remove 1 star. But I'd actually make this a smooth downgrade curve something like downgrade = max(2 - log10 (ticks)), 0). ie once you are over 100 ticks you are working purely on aggregate stars. @scd what do we actually call 'iconicness' under the hood? Can we search on this in the facets? When I see the list of iconic climbs at a node, I kinda want to see what the next X most iconic ones are... |
This seems to have resolved itself? |
I don't think this is resolved, gara is still a complete mess and providing just plain weird results. It ignores multiple 3 star classics that have lots of ascents and instead shows problems that have never been repeated. See also this forum where I list what I think gara's icons for bouldering would be as a comparison point: https://www.thecrag.com/discussion/1244829924/upper-gara-gorge--grading-around-gara |
The Gara example seems ok, Hope has 37 ascents, this is why it is there. Closing this until further evidence of a problem is presented. |
The gara example is still a perfect example of how wrong the current algorithm is. The problem is not Hope, it is everything else in the list, and everything else which is not in the list and should be. Below is the icons right now in a gorge which has 640 routes and boulder problems, and 1788 ascents. And yet despite all that data it comes up with totally unexpected icons, and only 4 icons when it could have icons a much large subset of the entire grade spectrum. If you asked any local what their gut feel would be for iconic routes, none of these would come close except for Hope. https://www.thecrag.com/en/climbing/australia/gara-gorge/upper-gara-gorge Of those 4 route icons, 2 of them Elver Escape and Confined have never been repeated. Sloper traverse has only had 3 ascents. They might be decent enough routes but how it is possible that a route which has never been repeated could possibly be considered an icon, over other routes which have had dozens of consistent ascents for decades at a high rating? I also don't think it's useful to have icons be a mix of routes and problems, it should be one or two lists for the most dominant gear type for a given area. If I asked around here I'd probably come up with a list like this for routes and problems: 16 Hope V0 Prayer Note There will of course be personal preferences but everyone would agree that the ones in this list above are at least in the top 3 at each grade. The original list bears no resemblance to this at all and is essentially useless for someone visiting and wanting a quick taste of the best routes. And this isn't a problem isolated to Gara, Ebor is the same, 3 of its 4 icons have had only 1 or 2 ascents, when there are trade route classics with dozens of ascents which should be on the list instead: https://www.thecrag.com/en/climbing/australia/ebor-gorge Every local crag I looked at is the same, the icons are just pretty useless. At super popular crags like Frog Buttress things still seem off but less so, so I suspect the algorithm is just not performing well at small and medium scale, and its only coming good at very large crags where good routes (even if not the actual most iconic) float to the top regardless of how you rank them. At the moment the algorithm is a bit of a black box and clearly sub optimal. If this was built from scratch the way I'd approach it is:
|
Ranking items by star ratings is also discussed in various places on the web, e. g. https://www.evanmiller.org/ranking-items-with-star-ratings.html I've looked into this briefly during a small project of mine. Overall I think this is not a trivial topic to get it right. |
The main problem is the FA 'froth' factor: everyone thinks their own new problem is the best and they give it lots of stars and it only has 1 ascent. Over time with more ascents the quality rating settles down to a more realistic level. The main issue is that volume of ascents is seemingly not a factor in the algorithm. I think that what happened was that in the past volume was a factor in the quality rating but then it was simplified to the '1 climber 1 vote' (#1511 and #560). I've manually done a couple spreadsheets for selected grades using a super simple formula of quality X ascents. To make sure I didn't skew the results doing this by hand (I only did a tiny subset of routes) I made sure to include the most popular but lowest rated problems in each batch. This very naive approach gives what I consider to be near perfect results. The only difference from the gut feel list I made on the top of my head above was 'The crack' now edges out over 'Pseudoephedrine' and I agree with this in hindsight, but the fact that Pseudoephedrine was such a close second only further shows this metric is pretty close. V3
V4
V5:
V6
|
+1 : Input from another user: https://www.thecrag.com/discussion/6127612239/website-suggestions |
feedback from user
I've noticed strange behavior with how routes are listed as icons, as evidenced by what is seen at the NSW level of the index. Many of the icons seem out of place:
"Montys slot" grade 6, 1 ascent (looks worthless really)
"Lips" grade 11, 2 ascents (looks worthless really)
"Cornerstone Rib Original" grade 8, 2 ascents (far inferior to the normal cornerstone rib)
"Bubblegum beach", grade 14, 1 ascent (looks poor)
"West wall", grade 12, 156 ascents (banned, 2 ascents in 2 years)
I had thought that Icon routes were calculated based on popularity and number of ascents, however somehow the algorithm seems to be weighting towards routes with a 100% rating due to a single well rated ascent.
This may be as you expect, in which case sorry to bother you, just thought it might be worth a look.
The text was updated successfully, but these errors were encountered: