Uneven levels of difficulty #222

Open
katox opened this Issue Aug 8, 2012 · 15 comments

5 participants

@katox

After spending considerable time doing the exercises I can say that various people writing problems have wildly different estimates of a problem difficulty for people learning the language.

We can't do much about that but we could gather some data on time actually needed to solve the problem on a successful solution submission. This would allow newcomers (like me two weeks ago) to tackle easier problems first instead of solving older problems first. Averaging this input over larger group of people could help to find outliers.

Another possibility would be a poll on the difficulty of the problem but that could be problematic because people consider problems harder at the beginning. The method I used to pick problems - after getting really stuck on an "easy" one ;) - was to sort the problem list by number of times solved and then to pick by difficulty. But that is also biased towards older problems.

Just thinking aloud. This enhancement could influence the recommendation order only. I don't think that it should result into directly changing difficulty levels. It would be more like fine tuning. In the end difficulty levels are not that off on average. I can remember only a handful of problems which I would move from hard to easy and vice versa.

@djtrack16

+1. I also think problems should go in increasing order of difficulty. (i.e. it should only suggest "medium" problems after you've solved the all the easy/elementary ones). But on the other hand, it'd also be nice to have the option of hiding difficulty level, so you are not worried about how hard/easy others think this problem is when you are trying to solve it -- just try your best to solve it.

With regards to timing solutions, I don't think this is a good idea, because many times I don't solve problems in one sitting, and leave the browser open, or simply get stuck and move on to another problem -- or just walk away for a breather. It seems that timing solutions implies every problem is solved in one sitting.

Just thinking aloud, but how is difficulty level useful, if it is inherently different for everyone, depending on skill level when starting on the site? People will judge for themselves the difficulty level of a problem based on their experience.

I realize this thread is 1.5 years old, but I'm a recent newcomer to the site, and I have similar criticisms.
Thoughts?

@amalloy
4clojure member

You can choose to sort https://www.4clojure.com/problems by how many users have solved the problem. That seems like a reasonable proxy for difficulty, in the case that you're not impressed by the default difficulty guesses. And it looks like it doesn't correspond exactly, but at least there are no Hard problems in the middle of Easy or vice versa, just some confusion about what counts as Medium vs Easy or Hard.

The problem recommendations given after you finish a problem ("you finished X, now try Y") are backed by a much dumber algorithm, as I recall, which is basically just numerical ordering of the problems. I don't remember if it would be hard to change that. But, not much work has gone into 4clojure in a long time; I'm basically content to let it continue on as-is unless something drastic happens.

@djtrack16

@amalloy yes i understand, is it worth doing an investigation and submitting a pull request for this issue and other issues I might find? Or is the project very dormant atm?

I'm also interested in adding problems (heavily math oriented). If I submit a slew of problems via the site within the next few days, how long will it take for someone to look at them and approve/disapprove? And will I get a notification or will I just have to visit the site randomly and the problems will show up? (Just asking because it seems this project hasn't been touched in a while).

Ok I've gotten off-topic zips lips

@amcnamara
4clojure member
@djtrack16

@amcnamara sorry for the delay, i finally submitted the three problems. They should be under my username, djtrack16.

I ran the tests on my own code. Let me know if there are any problems at all with the submission.
Thanks ;)

@amalloy
4clojure member

@djtrack16 Can you include example solutions in this issue conversation?

@djtrack16

@amalloy by example solutions, do you mean actual code, pseudocode, or what is the expected output given certain inputs?

@amalloy
4clojure member

I've solved http://www.4clojure.com/problem/195 myself and marked it approved. It was a fun problem, but editing the problems is a little tedious so I'd be happy if @amcnamara wants to do the rest.

@amalloy
4clojure member

I meant actual code: what solutions did you write that pass your tests. I think that's what @amcnamara meant too. It's just a lot easier to confirm that the test cases are correct if you don't have to solve the problem from scratch youself.

@djtrack16
@amalloy
4clojure member

The parentheses problem is pretty simple really: https://gist.github.com/amalloy/8fdfdda5160581245f23. Your tree-seq/coll?/seq/remove is just a reimplementation of flatten, which is pretty gross anyway, to account for the fact that you forgot to concat in the innermost section of code. It's also much easier if you build a sequence instead of a string, and then just convert it all to strings at the end, just because clojure is so much better at handling sequences.

It's hard to be sure because you've mangled all the variable names, but it looks like we use the same algorithm, modulo your confusion about nested sequences.

@djtrack16
@amalloy
4clojure member

Eh, partial isn't that amazing: I wrote (partial apply str) instead of #(apply str %), and #(cons \( %) instead of (partial cons \() just as a matter of taste; I don't think it matters terribly which one you prefer, and I can't say why I preferred one over the other.

Flatten is just gross in almost all cases, both because it means you produced data of the wrong shape to begin with and because it's too sensitive to the shape of your data: (flatten '(((1) 2 ((3 4) 5)))) probably does what you want, but what if instead of numbers you had [x y] pairs? (flatten '((([1 2]) [3 4] (([5 6] [7 8]) [9 10])))) messes up that internal structure because your data happens to be list-shaped.

As for how to rewrite your solution...well, I'd write mine instead! This is only half a joke: with your minified variable names and over-aggressive indentation it's quite tedious to remember what is going on in your version, even though I know the algorithm is similar to mine. So first, I'd reformat it and remove some extraneous junk: https://gist.github.com/amalloy/2e7af7510d563b4345bd/2d17d2d51e5376ad3a14123bc15ba67ab7bf8952. Now that the two versions are side-by-side, it's not too hard to see what the differences are. The main two are:

  • Your function is inconsistent in its return type. Sometimes it returns a single string, other times a pair of strings. Because of this, each iteration of your function has to go through this awful set/remove/tree-seq rigamarole just to sort out what's what. No wonder you say it's slow! Compare to mine, which always returns a list of strings (or list of lists of characters, which is much the same), and only has to "flatten" by one level.
  • Your recursion is inside-out relative to mine: you're doing something like [(q (concat ...)) (q (concat ...))], where I'm calling (concat (lazy-recur ...) (lazy-recur ...)). What this means is that as your call stack gets deep, each "instance" of q knows all about the characters that came before it in the list, even though most instances do nothing with s but pass it along to the next. Mine, instead, breaks down the problem: each call to lazy-recur knows nothing about the characters that came before it, and never joins any strings together. Instead, I "trust" that the function that called me will know what to do with the partial result I'm returning. eg, (lazy-recur 1 1) will return "incorrect" results: ("())" ")()"). But that's okay, because I only get into those situations when I know there's one open-paren waiting for me at a higher level in the stack, so to speak.

So, all that said, the simplest solution to get rid of your flatten-alike is https://gist.github.com/amalloy/2e7af7510d563b4345bd/80f800b5548af1e4f3811ddf6fa11a219bafb446 - just adding a couple levels of lists to regularize the nesting, followed by an apply concat, and lifting out the set call the to top level, once you've actually got all the answers, instead of constantly going in and out of sets.

@djtrack16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment