-
Notifications
You must be signed in to change notification settings - Fork 219
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
One-off rank adjustments for overranked Coq Kata #2001
Comments
I agree that ranks should be adjusted for many theorem proving kata. I have created a temporary wiki page which contains a table with suggested kata ranks. Anyone can add new ranks to this table (it now includes @DonaldKellett and my ranks). |
@kazk Any plans on implementing the rank adjustments soon? |
I don't think this will overload the servers. But I don't know if this will work as expected and adjust the honor points/language progress correctly, so I need to test and add the feature to do this if needed. If I remember correctly, it didn't work when the rank was being changed a while ago (but still overloaded the servers).
I currently don't have a plan to do this "soon" since I have other tasks with higher priority. But I agree with the need of adjustments so I'll do it eventually. Also, I'd like to see 2-3 more users to suggest the ranks on the wiki. |
Added my ranking suggestion on the wiki page. I generally agree with monadius' rankings for the katas I remember of. (I've been away from theorem proving for months though.) |
This is a known issue of having only one scale, but it might be better not to translate if the difficulty changes that significantly. So I'm not sure about changing this one to |
@dramforever @Voileexperiments Mind taking the time to add your rank suggestions to the wiki page for those Kata you have completed (just leave the corresponding entry blank if you have not solved it)?
I agree that submitting translations to a Kata with a huge difference in difficulty from the already supported languages is rarely a good idea. In order to prevent similar incidents from happening in the future, I have added a note at the beginning of the Kata Description for the remaining Haskell "theorem-proving" Kata, explicitly asking translators not to submit Coq/Idris/Agda translations.
That was precisely why I mentioned it in the "Additional Remarks" for that particular Kata. But now that I think about it, perhaps it would still be better to downrank it to The rationale for this is that Codewarriors will tend to choose the easy way (it's human nature!), so downranking the Kata to match the difficulty in Coq/Idris/Agda would prevent solvers from getting "easy points" in such a trivial Kata and I don't think many Codewarriors (given this warning) will be stupid enough to do it the hard way in Haskell anyway and perhaps waste hours on a |
I think it's a sensible thing to do. Though I believe 8kyu is too low, maybe increase the kyus by 1 each. |
I realized I can probably just unapprove these kata, so you guys can change your rank assessments and approve them with adjusted rank. I wasn't sure if adjustments to the honor points and the language progress work, but it seems to work since #2015 was caused by it. I can unapprove some of these and see what happens. |
@monadius Fixed the rating. Before the re-calculation, it was 12 "very", 5 "somewhat" and |
I've just edited the wiki page to add a column indicating whether the Kata has been successfully re-ranked. @kazk I am ready for the re-ranking anytime - feel free to un-approve whatever Coq Kata is in the wiki page as you see fit - just give me a heads-up when you do so. @monadius As for the exact rankings used for the re-approval of the overranked Coq Kata mentioned, I suppose we'll just go with your suggested rankings since @Bubbler-4 's suggested rankings agree entirely with yours? |
@DonaldKellett Can you re-approve my 3 kata (1: |
Done - had to perform a bit of vote manipulation for 3 to get the average down to |
Did everything work as expected? It should've decreased the honor points and the language progress. |
@kazk When you unapproved 3 kata my honor points for completed kata and the language progress had decreased. But my honor points for authored kata did not change. When @DonaldKellett re-approved these kata, I got additional honor points for authored kata. |
@monadius Thanks. Yeah, it looks like something had failed when processing site events for authored kata. I just recalculated your stats and will look into the bug. |
I think I was able to fix the bug. @DonaldKellett I unapproved yours. "Simplifying conditional statements", "Simplifying conditional statements, again", "Finding the maximal program", "Multiples of 3, you say?", "A random fact about filtering". |
@kazk @monadius was able to re-approve "Multiples of 3, you say?", "Simplifying conditional statements", "Simplifying conditional statements again" (still working on dragging the average assessed rank of "A random fact about filtering" down to Would you mind checking if there might be a bug in the stats calculation for that Kata like what happened with "Program Verification #1"? Cheers 😄 |
@DonaldKellett Recalculated the rating for "Finding the maximal program". |
Unapproved next 5 kata from the top (skipped DNE for now). |
I just re-cast my rankings for 4 of the 5 Kata (the other one was successfully re-ranked). As it now stands, their average assessed ranks are still way too high so I've opened an Issue on each of them to prevent approval, though Codewars seems a bit slow to update its approval status (from "Awaiting moderator approval" to "Waiting for issues to be resolved"). @kazk Perhaps I'll wait for a while to see if the status update goes through, but you might consider looking into this issue as well, seeing as I've encountered similar issues recently with other Beta Kata (e.g. still seeing "Ranking feedback needed" when it should already be approvable) and the only way to force a status update is by un-publishing and re-publishing the affected Kata (even re-casting votes doesn't seem to work now). |
@DonaldKellett Yeah, looks like the method to adjust the beta status was not called for some of them :/ I just manually triggered it for "A random fact about filtering" and "This square is perfect". |
Unapproved the remaining. |
@kazk My translation honor points were not reduced when you unapproved the remaining kata. |
🤦♂️ It can be fixed by recalculating the site events after reapproving, but I'll look into it. |
Nice, so it seems that we now have 2 Kata to go (excluding "DNE equivalent to PEM? Prove it!"), thanks for all the help @kazk , @monadius and everyone else who has participated in the re-rankings 👍 As for "DNE equivalent to PEM? Prove it!", are we certain that we don't want to do anything about it? AFAICT, its rank is only suitable for the Haskell version |
2 kyu is not suitable even for the Haskell version. In fact, Haskell's solutions are almost exactly the same as in Idris or Agda. So I think that this kata should be re-ranked. Maybe as a 5 kyu kata. |
@kazk May we be informed of your final decision on whether to re-rank "DNE equivalent to PEM"? Once that has been decided, all re-rankings are considered complete and this Issue may be closed. Also, considering that rankings for Coq Kata have finally stabilized and we haven't had any issues with coq_codewars so far, would it be fair to move Coq out of the Beta phase once and for all and officially declare it as a stable supported language on Codewars? |
I don't have a strong opinion about it so I just unapproved it to let you guys decide. Yeah, I think Coq is ready to be out of beta (I haven't been able to train on it, but I haven't heard much issues like you wrote). There are many other languages that's been sitting in beta way longer than necessary. The problem is that if I move them out of beta, it'll be made available as a language to solve "Multiply" kata when signing up. As you can see, there's not much space left because of the fixed size monitor 🤦♂️ The landing page needs major updates in general, but for now I'll get rid of that monitor (looks super outdated too) and will make some changes to fit more languages without looking too crowded. Then I'll start moving some languages out of beta including Coq. |
Coq is now out of beta. |
As power users familiar with Coq know, many of the Coq Kata approved early when Coq was first introduced to Codewars were insanely overranked since
someone wanted to reach(almost) everyone was a n00b in theorem-proving back then and blindly ranked Coq Kata based on similar Kata in Haskell (which wasn't built for theorem-proving in the first place). Considering that such overrankings may have confused new Codewarriors specializing in Coq as to how the ranking system actually works on Codewars (i.e.1 dan
earlier and ended up abusing his Kata approval rights1 kyu
is actually the difficult end and not the easy end) and I do not want the same to happen to Lean when it eventually gets added to Codewars (because by then we would have to translate a few simple Coq Kata into Lean as a demo before inviting in the "big players" to contribute Kata of their own), I would like to propose the following one-off rank adjustments to Coq Kata that I think are overranked:6 kyu
6 kyu
7 kyu
8 kyu
8 kyu
7 kyu
7 kyu
2 kyu
rank (which is still overranked but perhaps not so much so).7 kyu
5 kyu
6 kyu
5 kyu
6 kyu
5 kyu
5 kyu
5 kyu
7 kyu
6 kyu
5 kyu
4 kyu
5 kyu
If anyone has alternative rank suggestions for any of the Kata listed above or would like to point out another overranked Kata not mentioned in the above table then feel free to chime in.
As for concerns about overloading Codewars' servers with these rank adjustments (because Codewars needs to recalculate the rank/honor of each Codewarrior affected, etc.), I don't think it will be a problem since there are only a few Coq completions per Kata (so the total number of Codewarriors affected is probably well under 100). Am I right @kazk?
Special mentions: @monadius, @Voileexperiments, @Bubbler-4, @dramforever
👍 reaction might help
The text was updated successfully, but these errors were encountered: