Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Purging illegal solutions #96

Closed
urticadioica opened this Issue · 23 comments

4 participants

@urticadioica
Collaborator

Excuse me for posting here instead of "pinging one of the moderators on IRC or twitter", but the list of illegal solutions is so huge, posting there would be impractical. Most of the problem is old external program and custom .vimrc solutions, which vimgolf won't let you enter any more, so flagging cheats should be easier in the future.

There are some things I think are cheating that aren't mentioned specifically in the rules:

  1. Using a Vim window that's not 80x25. Using your window manager to change your terminal size is sort of like an external tool, it "hides" strokes, and it makes the solution impossible to replicate based on the strokes given. See here, where I resized my buffer manually and still saved a stroke. If you want a non-standard window size (and to me, 80x25 is the standard), you should have to change it with Vim.
  2. Using :!perl obviously doesn't work, but if your Vim has +perl, :perldo still works. Some might say "builtin feature," but I say "external tool."
  3. [number]D from the beginning of a line doesn't delete the last line if it's blank. On one challenge, some users managed to make it work, somehow.
  4. When you move the cursor with the mouse, it shows in the log as 2 strokes. If you replaced every 3-stroke movement across the site (unless it's in a macro, maybe) with a mouse-click, you'd get tons of 1st places. Those two strokes in the log always appear the same, no matter where you clicked, so it's impossible to replicate the result.

I've come across 531 cheats, by my definition, in first place. More may show up if they're deleted, because only your top solution shows. I listed some more, so a tied-for-first legit solution won't have cheats above it, and because sometimes you can't see the next legit solution above the cut.

@igrigorik
Owner

Thanks a lot for putting this together!

I guess a meta observation here is the fact that the current "admin" list is not as active as it was a year ago.. probably worth revisiting that and adding a few more people to that list.

@gumnos @timvisher @fgalassi would you guys be interested in helping out with this?

@Gumnos

I'd be glad to help out. I'm not as against the :perldo solutions because they can be replicated purely within Vim (though, as I don't care for Perl, I certainly don't object if you want these ousted). Also, I don't much mind the ones that require a specific starting window-size, especially if it's noted in a comment—it's more of a problem if the window-size is changed during the execution of the challenge. I'm quite against the use of external tools, and find the mouse solutions unsavory.

@fgalassi

i can help but i disagree that we should retroactively remove all external tool solutions. The documentation reads "refrain" which is not the same as "must not" or "is prohibited". Back in the days, it was a perfectly accepted practice and good solutions were even praised. It's also a very vim-like way of doing things. The argument "but you can't do the same today" doesn't hold or you should purge for example all old solutions where indentation worked differently. They were perfectly valid but you "can't do the same today". Also the rules are lacking (mouse and terminal resize are not even mentioned). I'd change the rules to be more clear and strict and then i'd very cautiously remove just obvious cheats: non standard tools, vim that mistakenly logs the wrong number of keys and personally i'd go for mouse and terminal resize as well because they're really against vim spirit.

@Gumnos

I think one of the motivating factors behind the refraining from external tools is that it makes the solution more cross-platform. Most of the solutions using external tools don't work across platforms without installing other tools ("sed", "tr", etc may be Posix, but not available on Win32 without installing something like Cygwin; occasionally additional external tools such as the "untag" command are used).

@fgalassi

SPOILER ALERT, for those who have been made admin. You immediately see all solutions even if you didn't submit yours

@igrigorik
Owner

Awesome - thanks guys! Just gave you admin bits on VimGolf.

Re, rules and all that fun stuff:

  • To be honest, I don't fully understand the gotcha with the window size. On the surface though, it seems harmless. The goal is to teach people vim, and if that means you can bend the editor a bit to your will - that's ok, as long as everyone else is able to do this also without relying on any addons/code/etc. In practice (as in, real-life use cases), it seems like the resizing would be impractical anyway.

  • On external tools: I've gone on the record before as against it. Same reasoning as above. The goal is to make people learn Vim. Yes, there may be a faster way to accomplish something with another tool.. but it's called VimGolf for a reason. :-)

  • As far as removing old solutions: that's a tricky one, and I think it's a matter of one by one consideration.

@fgalassi

@igrigorik i think i can put in words the rule which i'd follow and that coincidentally explains why terminal resize is an issue. The point is vimgolf is about being fast, the unit we use to measure speed is the key. We do that because it's easy to count keys, but ideally we would measure time taken, because speed is our focus. So the rule is:
You must not hide time spent outside of typing keys in vim.

  • If you use terminal resize, you get M to move where you want, but that's not free, you're hiding time taken to resize the terminal. You could resize the window with vim itself and that would reveal the cost in keys.
  • If you install a specific external tool to solve a problem, you're hiding the time taken to install the tool.
  • If you use the mouse, you incur in a fixed price of 2 keys, while you're hiding the time taken to move your hand (which is typically costly) ..etc..
@urticadioica
Collaborator

I had a feeling this would start some arguments, and you haven't even started on what to do about "Get rid of html tags"...

Maybe there's a good alternative to removing solutions. It would likely be much harder to implement though, or have some other issues:

  1. Leave them. Top result may be dozens of spots down. New solvers can't tell what the target score is. New solvers can't replicate top scores. Leaderboard is a joke. But no work, no controversy!

  2. Implement some sort of "obsolete" tag. Entries show a score and a special flag, but don't have a place like 1st, 2nd, and they don't count toward the cut. Preferably not the leaderboard either, but that doesn't matter much. Would take time to implement.

  3. Delete them. A couple clever solutions would disappear (majority are obvious/dull). Chaos on the leaderboard. Fair bit of work to finish. Have to agree on which to delete. :)

@fgalassi While I agree with your conclusions on terminal/mouse/etc., the same reasoning also applies to keys outside the main keyboard, like arrow keys. I wasn't happy when I first learned VimGolf doesn't shun arrow keys, but since there was never a controversy, I learned to exploit them to the fullest.

@urticadioica
Collaborator

@fgalassi First: the word refrain. If you used an external tool, you didn't refrain from using it. As long as it's said "refrain", it's been banned.

You mentioned "non standard tools". I'd like to know what standard you propose. I'll nominate POSIX. Calling a GNU utility or something in coreutils "standard" is silly. If we used POSIX, we'd keep cc, nl, tr, iconv, grep, and ls (many of the solutions that use these have other problems though). gcc, rev, seq, indent, tac, primes, and factor would have to go, along with that one-letter command at the top of Reverse characters in a line.

None of that matters of course. Vim is a multi-platform program. None of those programs are standard on every platform Vim comes on. So I'll agree with part of your comment. Let's delete the "obvious cheats: non standard tools". That is, all of them.

Of course, if we don't delete those solutions, we could change the name of the site to GNU/VimGolf.

@Gumnos

I'm not sure I'd call them arguments. More like a friendly clarification of the rules. It seems the consensus (@urticadioica, @igrigorik and myself, with a tempered response from @fgalassi) is that most external tools are right out. Otherwise, one could just build a custom tool that solves the challenge and then call it. The only exception I'd consider is "xxd" which ships with Vim on all platforms.

I agree with Federico & Urtica that actions that don't get recorded in a replicable manner (mousing and resizing the window) are pretty much out too.

A piece of me would also like a way to let the community give input to bump scores. An elegant-but-longer solution might get kudos-points, while one that violates the Vimgolf spirit (e.g. the above two violations) would get down-voted. Then again, a piece of me would also enjoy an "edgolf" or "exgolf" as well, so I may have a screw loose. :-)

(as an aside regarding Frederico's comments on speed, I'll often enter my solution's keystrokes in another terminal window and then paste them into the vim-golf session to replay them accurately, especially if the solution is long/messy, so time is a bit of a bad measure since playback is as fast as the terminal can handle).

@fgalassi

@urticadioica no, refrain means it's your duty to hold back, you decide, versus can not/must not/prohibit etc.. which mean the rules force you to do so. That is why you don't find refrain used in contracts and laws, because it does not explicitly prohibit anything nor it says what happens if you don't refrain from doing something.
But more important that refrain definition is that @igrigorik used intentionally that ambiguous word, because he wanted to be the community to decide. He didn't want to play a dictator role but let it be a social process. Now, i haven't seen mass accusations of cheating. Occasionally cheating has been mentioned when short custom aliases were involved, or personally i attacked utilities i couldn't find on my linux distros, but that's it. So i must conclude that the community accepted external tools as a valid practice.
Taking an example from the other side, terminal resizing has been almost always linked to cheating.
You must start thinking from this perspective. Vimgolf was made not to enforce strict rules but to let people respond. I had to get it myself. I'm the guy who kept annoying @igrigorik with the idea of re-running solutions serverside to remove cheating completely, but in the end he was right. That's just overkill

@fgalassi

I think we're mixing up 2 problems. How vimgolf is right now and how to deal with old solutions when it was different in respect of rules interpretation. I'm going to leave the second very tricky problem alone right now and focus on the first.
1) as of today vim is started with -Z so external tools are not allowed. Not even :grep works. :perldo does because it's compiled into vim, so i guess it's completely acceptable
2) mouse. Let's kill mouse. what about set mouse= in vimrc ?
3) terminal resize. I will pull a trick to cheaply check on submission. I just got an idea that should work.

@fgalassi

4) cd into some known dir (inside ~/.vimgolf?) before running vim, so that you can't play on being in a particular dir

@igrigorik
Owner

@fgalassi why would we want to kill mouse? If there is a valid way to get a faster solution by using a mouse, then that's a perfectly valid solution in my books. Or am I missing something here?

To be honest, I also still don't exactly follow the terminal resizing trick -- clearly, my Vim-foo is weak. ;-)

@urticadioica
Collaborator

@igrigorik Worst thing about the mouse is that it's ugly, and you can beat any solution with a 3-stroke movement. More practically, it shows up as the same two characters in the log, no matter where you clicked. That makes the log impossible to follow and replicate.

As for window resizing, there are three ways to abuse that. First, you can make your window exactly the right size so that H, M, or L go to exactly the line you need. You could also abuse the g$/gj/gk family, which was just recently abused for the first time by KersonHsiao in run of the mill SQL execute, who replaced his mouse solution with this. Lastly, you can set your window to a certain number of columns to work with gq and gw, instead of typing out :se tw=??<CR>, saving 10 strokes. Just like with the mouse, the log on the challenge page shows no trace of what exactly you did, so your solution can't be replicated.

@urticadioica
Collaborator

There are certain problems with the invalid solutions as they are right now. Deleting them is the simplest solution, but maybe we can find other ideas. There are valid arguments to keep them, even though I don't agree. I should have focused more on the problems instead of a harsh but simple solution. Most important thing is that I draw attention to the problem.

  • Standard 1st place: When you don't know what the valid top score is, you can't tell what the standard is. You don't know when you're "done." This issue was raised before. In my first few challenges, I didn't yet know a lot of the top solutions were invalid. I wasted a lot of time looking for those last few strokes that didn't exist. Later, I learned to look at the range of scores, and the usernames next to them, and I could roughly predict the true top score before I started. No first-time user will be able to do that.
  • The cut: If you're a way back from the shown top score, you can't see solutions above a certain point. Is there a valid solution hidden up there? You have to cheat to find out. There are some cases where I beat the previous valid best by a large margin, and I'm not sure someone with the old best score can see me. This applies to invalid solutions too. Even if the top invalid solution is worth seeing, it's possible a new user will never be able to see it. In this case, the lower invalid solutions are the problem, not the top ones. This is the reason I sometimes listed non-"top" scores in the OP.
  • First forever: If the top solution is invalid, it's likely no one will ever be able to match it again. There are a couple cases where I've posted a comment with an improved solution using the same cheat. I can't post it (and I wouldn't if I could). A flawed solution will be on top forever.
  • Recognition: If there's a great solution, people should see it. This should apply even if it uses a banned cheat. It's one of the best arguments for keeping them, but also a good argument for removing them. Sometimes the best solution is hidden deep down the stack, behind a mass of trivial cheats. Being in 1st place gives a certain amount of credit, but @gumnos made a great point earlier. Some solutions that take more strokes are also worthy of extra attention.
  • The leaderboard: Not the most important problem, but getting a boost on the leaderboard with a stroke count that new users can't replicate isn't really fair.

If there's anything less severe, less controversial we can do to fix these problems, I'm for it. I have a couple ideas.

You can tell what entry is legit first if there's a flag next to it. Even less work, if everyone knew a list of "certified firsts", they could go by that (my OP post kinda does that). Less work still, use "the udioica rule". I'm tied for 1st on all 120 VimGolfs I've entered so far. Look for my name, and you'll see the top score. I actually run diffs of my player page every so often to see if I've been beat. People would just need to know about this...

Getting more people to see and learn from great solutions is a bigger problem than just old cheats. On-site, a system of "favorites" or "upvotes" would help, but would take work to implement. I'd really like to see more discussion of VimGolf outside the challenge pages. I started /r/VimGolf, and made a first post avid players might enjoy. Anyone can post, but even if no one does, it can kinda be my VimGolf "blog", which I've wanted to start for a while now. One thing I hope to do is give my perspective on some of my favorite solutions.

@igrigorik
Owner

@urticadioica those are good points.

I agree with your line of thinking: a solution should be reproducible by looking at the actual input log on VimGolf. That's the gold standard. Once that is not true, we're into the gray area: doesn't mean the solution is invalid, but, it requires additional context. In these cases, even a comment on the solution can be the right way to "make it legal" - in other words, as long as it's clear what's happening. Perhaps that's what we should look for?

The column and line hacks are not invalid, in the sense that.. you've bent the tool to match the solution, which is fine, but others need to know how you did it. Of course, I wouldn't want to see all of the top solutions resort to this either.. but at the same time, I wouldn't want to disallow it.

TL;DR: if you're bending the tool, perhaps we should either (a) require that you explain what you did, or (b) other players can provide that context. If we can't reproduce it or figure it out, then we can remove the solution.

There is still a lot of gray area here.. but that's fine, I don't want to be dogmatic about it. I trust the judgement of the admins.

@urticadioica
Collaborator

Yes, adding a comment about what you did resolves the reproducibility problem. That would have solved the problem with custom .vimrc's early in the site's history as well. "I put 'set sw=2' in my .vimrc, now you can reproduce my solution." Or maybe I could use the "* register in my solutions with a comment saying "I put such and such text in my window system's clipboard." Maybe I could get a custom macro to load and do most of my work for me as long as I write it down in the comments? Yes, I can use custom macros or :set or whatever I want to, and the current rules do it the right way: every stroke of it will count against my score.

I have no problem with resizing the window. There are commands for that. Type :se co=70<CR>, and you can have a 70-column window. It'll cost you 10 strokes, and because of that penalty, no one will ever use it. Or you could drag the edge of the window with your mouse for a cost of zero strokes, and put your "phantom strokes" in a comment. That's no better than hiding your indent settings in your .vimrc. If you do an action, it should count against your score.

The way the mouse is currently scored is a bit odd. The two strokes for a mouse click are one for mouse down, and another for mouse up. That's the information that vim -w records. Actually, it's 3 bytes each, but vimgolf recognizes certain sequences of bytes as one "stroke," and that's why we get away with bad practices like arrow keys in our solutions. It would actually make more sense if a mouse click counted as one stroke -- after all, when I press a key on the keyboard, I tend to release those too. Would that make for a good game? Definitely not. No one would use actual Vim commands again.

The VimGolf definition of a "stroke" is pretty funny when you think about it. It's a piece information that shows you how to change the contents of a buffer. If "stroke" was a unit of how effectively the human has interfaced with the computer, arrow keys and such would cost at least 4-5 times more than a home row key, and a mouse click would likely cost at least 10. If it was based on bytes recorded by vim -w, lots of things would count higher, even finger-friendly things like <C-@> or <BS>. As it is, all key chords are 1 stroke, no matter how far your fingers move (better not make fun of emacs any more). Multibyte characters count one stroke per byte, except when they don't (compare number 1 and 3). Mouse clicks count as two, even though neither stroke tells you anything about what happened. And reaching for the mouse, aiming for the window border, carefully dragging to the right place, and bringing your hand back to home row, costs zero.

I recognize the need for simple, easy to master rules when making a good game. That's why I'm not complaining about 1-stroke arrow keys, when they really should count as several. "Everything's a stroke" is a good, simple rule. For the mouse though, 2 strokes is ridiculous, especially since they don't even carry any information. And since vim -w doesn't give you that information, it's hard to make any reasonable adjustment. And allowing a zero-stroke editing command doesn't make any sense at all. Again, since vim -w doesn't tell you anything, you can't make an adjustment.

Anyway, I don't want to sound negative or demanding. I wouldn't care about these issues if I didn't love VimGolf so much. Some things are frustrating for newer users like me, such as wading through massive numbers of old cheats, or a leaderboard algorithm that's so hopelessly biased toward long-time users, it's a miracle I made it as far as I did. But to me, those are footnotes in my user review, next to my 5/5 rating.

@urticadioica
Collaborator

I've got a question for those who have been around longer than I have, regarding Get rid of html tags. Obviously the huge mass of 17s worked at some point in the past, but I can't figure out how. I've tried searching through the commit history, but I can't find anything that would have made those entries work. My only guess is a custom diff program, or a flag that got passed to diff, but there's no way everyone figured out how to do that.

The reason I want to know is I want to propose a better way to deal with all the solutions that don't work across the site. I have ideas that work well for some of the more common cheats (or at least they're cheats now), but for the "ignoring whitespace" entries... I don't know how they got there in the first place.

(OK, there's another reason. If all I had to do was remove tags that don't span lines... I can do that in 15. And I'd like to crush all those clumsy 17s.)

@fgalassi

I don't know what you're talking about. 17 looks a pretty obvious solution and it works for me right now

@urticadioica
Collaborator

Yeah, the 17 is pretty obvious (a lot more obvious than my 22 that actually works). And it certainly appears to work if you don't pay attention to whitespace. But: 1) it leaves one stray space in every line where all the text has been removed; and 2) it leaves a stray blank line at the end.

The 17 you actually posted is amusing: dG:re<Tab> <C-R>+<CR>kdd:wq<CR>

And here's the comment you left on your post: "cheating to obtain 17 which used to be the standard trivial solution. As of today, the challenge is broken: just removing tags doesn't work due to spaces left over. @mike_adolphs please fix it"

@fgalassi

i think he fixed it because now the trivial 17 solution with substitute works

@urticadioica
Collaborator

It definitely doesn't work for me. I'm curious where you're dropping the spaces and the blank line. My best guess is that either your Vim or your diff are doing something strange. I don't suspect my toolset, because I have no idea why the 17s could have ever worked in the first place. The commands to make input and output match just aren't there.

I tried deleting the files from my ~/.vimgolf/put and re-running the challenge. The input and output are the same as before. I don't think this challenge has ever been altered. (Can challenge creators even do that?)

Anyway, maybe whatever is happening on your box is the same thing that happened in the past. If so, you could figure out what the problem was. One of these tests should explain why our results are different:

  • Remove all the files for this challenge from ~/.vimgolf/put to be absolutely sure there's no contamination. Run vimgolf, let it download the challenge, but don't do anything yet. Check your fresh version of ~/.vimgolf/put/4d1a7a05b8cb3409320001b4.output and see if there's a blank line at the end or stray spaces on the blank lines.
  • Run the :s command. Does your buffer still have the extra whitespace?
  • Save it, maybe with :w to a different file. Is the whitespace still there?
  • diff the .output and .work.orig after saving. If they're still different, is diff saying they're the same?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.