Update config.json to match new specification

[kytrinyx]

For the past three years, the ordering of exercises has been done based on gut feelings and wild guesses. As a result, the progression of the exercises has been somewhat haphazard.

In the past few months maintainers of several tracks have invested a great deal of time in analyzing what concepts various exercises require, and then reordering the tracks as a result of that analysis.

It would be useful to bake this data into the track configuration so that we can adjust it over time as we learn more about each exercise.

To this end, we've decided to add a new key exercises in the config.json file, and deprecate the problems key.

See exercism/discussions#60 for details about this decision.

Note that we will not be removing the problems key at this time, as this would break the website and a number of tools.

The process for deprecating the old problems array will be:

• Update all of the track configs to contain the new exercises key, with whatever data we have.
• Simultaneously change the website and tools to support both formats.
• Once all of the tracks have added the exercises key, remove support for the old key in the site and tools.
• Remove the old key from all of the track configs.

In the new format, each exercise is a JSON object with three properties:

• slug: the identifier of the exercise
• difficulty: a number from 1 to 10 where 1 is the easiest and 10 is the most difficult
• topics: an array of strings describing topics relevant to the exercise. We maintain
  a list of common topics at https://github.com/exercism/x-common/blob/master/TOPICS.txt. Do not feel like you need to restrict yourself to this list;
  it's only there so that we don't end up with 20 variations on the same topic. Each
  language is different, and there will likely be topics specific to each language that will
  not make it onto the list.

The difficulty rating can be a very rough estimate.

The topics array can be empty if this analysis has not yet been done.

Example:

"exercises": [
  {
    "slug": "hello-world" ,
    "difficulty": 1,
    "topics": [
        "control-flow (if-statements)",
        "optional values",
        "text formatting"
    ]
  },
  {
    "difficulty": 3,
    "slug": "anagram",
    "topics": [
        "strings",
        "filtering"
    ]
  },
  {
    "difficulty": 10,
    "slug": "forth",
    "topics": [
        "parsing",
        "transforming",
        "stacks"
    ]
  }
]

It may be worth making the change in several passes:

1. Add the exercises key with the array of objects, where difficulty is 1 and topics is empty.
2. Update the difficulty settings to reflect a more accurate guess.
3. Add topics (perhaps one-by-one, in separate pull requests, in order to have useful discussions about each exercise).

[verdammelt]

Done with PR #123

[yurrriq]

I'd suggest re-opening this since the exercises currently have a difficulty of 1 and the empty list for topics.

[verdammelt]

Ah someone is watching!

Hmm we could either keep this issue open or have separate issues for difficulty levels & topics.

Hmm... I'll re-open this issue - we can spawn further issues if it seems like a good idea.

[verdammelt]

BTW @yurrriq do you have any suggestions for topics and/or difficulty levels?

[wobh]

Couple of quick takes.

1. Topics is basically a tagging problem, so we'll need some set of tags to use and identify which one's are appropriate for the exercise. I had the idea that we could tag roughly around the chapters of the Common Lisp Hyperspec where we might expect people to look up things for this exercise. For example, hamming we generally expect people to look up stuff in "Iteration", "Sequences", "Strings", "Characters". For leap, "Data and Control Flow", and "Numbers". For gigasecond, "Environment" and "Data and Control Flow".

2. Difficulty is a ranking problem, but it seems like there are a couple of axes to consider. One is the number of possible solutions we might consider well-formed, and reasonable. For hamming there's a lot of these. For leap there's just two with minor variations, same with gigasecond. Another axis might be the complexity of the different acceptable solutions--that seems like it would track pretty closely to how many iterations, and how much feedback before a student might need before arriving at a decent solution. Lastly there's the question of the "pedagogical intent" with each exercise, which certainly is something that should be reflected in the topics. For example grains and difference-of-squares are basically math problems, compare with gigasecond and space-age which are also math problems, even simpler ones, but one requires learning about CL's time functions and the other is an exercise in first-class function defining.

That's it for now.

[verdammelt]

re: #1 - I think part of the idea of this topics thing is to have topics that span all of exercism along with some that are specific to tracks. So while I sort of like your idea, we can't use CLHS chapters exclusively.

re: #2 - I find that for some of our exercises there are plenty of 'right' ways to solve something. For example you can use recursion or do expressions or loop or map etc. None of them are more right than others and we can discuss their pros and cons. But at their core these exercises (/e.g./ hamming) are actually rather simple. I don't want to have the number of possible solutions to inflate their rank. Perhaps we look at the number of topics involved and some weighting of how 'advanced' some of those topics are to determine a ranking. space-age would be higher rank because it involves procedural functional definition which is more 'advanced' that simple math or even the date/time API.

I think we need to come up with some reasonable but arbitrary ranking and then have a discussion about it, shifting them about a little and then calling it a day. This sort of discussion could go on /ad infinitum/ and we'll have no result. I don't see much harm in setting down some "random" numbers so we can have a more directed discussion.

Maybe we should split the topics & ranking into separate issues and deal with them as such. They are not directly related and we could have topics without ranking and /visa versa/.

[verdammelt]

@exercism/lisp Anyone have any suggestions for topics & difficulty level for the xlisp track?

[verdammelt]

I know there was some discussion on this issue but I am going to close it in favor of #142 & #143. (This issue is tagged on exercism/discussions#60.)