Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update config.json to match new specification #179

Closed
kytrinyx opened this issue Aug 18, 2016 · 4 comments
Closed

Update config.json to match new specification #179

kytrinyx opened this issue Aug 18, 2016 · 4 comments
Assignees

Comments

@kytrinyx
Copy link
Member

For the past three years, the ordering of exercises has been done based on gut feelings and wild guesses. As a result, the progression of the exercises has been somewhat haphazard.

In the past few months maintainers of several tracks have invested a great deal of time in analyzing what concepts various exercises require, and then reordering the tracks as a result of that analysis.

It would be useful to bake this data into the track configuration so that we can adjust it over time as we learn more about each exercise.

To this end, we've decided to add a new key exercises in the config.json file, and deprecate the problems key.

See exercism/discussions#60 for details about this decision.

Note that we will not be removing the problems key at this time, as this would break the website and a number of tools.

The process for deprecating the old problems array will be:

  • Update all of the track configs to contain the new exercises key, with whatever data we have.
  • Simultaneously change the website and tools to support both formats.
  • Once all of the tracks have added the exercises key, remove support for the old key in the site and tools.
  • Remove the old key from all of the track configs.

In the new format, each exercise is a JSON object with three properties:

  • slug: the identifier of the exercise
  • difficulty: a number from 1 to 10 where 1 is the easiest and 10 is the most difficult
  • topics: an array of strings describing topics relevant to the exercise. We maintain
    a list of common topics at https://github.com/exercism/x-common/blob/master/TOPICS.txt. Do not feel like you need to restrict yourself to this list;
    it's only there so that we don't end up with 20 variations on the same topic. Each
    language is different, and there will likely be topics specific to each language that will
    not make it onto the list.

The difficulty rating can be a very rough estimate.

The topics array can be empty if this analysis has not yet been done.

Example:

"exercises": [
  {
    "slug": "hello-world" ,
    "difficulty": 1,
    "topics": [
        "control-flow (if-statements)",
        "optional values",
        "text formatting"
    ]
  },
  {
    "difficulty": 3,
    "slug": "anagram",
    "topics": [
        "strings",
        "filtering"
    ]
  },
  {
    "difficulty": 10,
    "slug": "forth",
    "topics": [
        "parsing",
        "transforming",
        "stacks"
    ]
  }
]

It may be worth making the change in several passes:

  1. Add the exercises key with the array of objects, where difficulty is 1 and topics is empty.
  2. Update the difficulty settings to reflect a more accurate guess.
  3. Add topics (perhaps one-by-one, in separate pull requests, in order to have useful discussions about each exercise).
@IanWhitney IanWhitney self-assigned this Sep 2, 2016
@IanWhitney
Copy link
Contributor

I'm working my way through this.

IanWhitney pushed a commit to IanWhitney/xrust that referenced this issue Sep 2, 2016
Working on exercism#179

The new exercises collection is in place and I'm populating the topics
values.
petertseng added a commit that referenced this issue Jan 8, 2017
History:

Thanks to the hard work of #167 and #127, we have topics in problems.md
that tell why we ordered problems a certain way.

After #229 (motivated by #179), we copied the topics into config.json,
since now all tracks can take advantage of the "topics" concept.

To reduce duplication, we can remove the topics and specific problems.

This commit uses something more abstract, that states our values without
going into the exact implementation. This requires less maintenance, and
is less likely to go out of date than specifically listing problems.

Since it only has the ordering philosophy and not the actual topics,
it's time to rename it to problem_ordering.md.

Closes #236
@petertseng
Copy link
Member

Observation made in #195

The problem I ran in to is that I find the difficulty rating impossible. What does 'difficult' mean? Is it hard because it introduces Lifetimes? Because it's algorithmically hard? And guessing at the difficulty of exercises I haven't done was even harder.

One possible benefit of adding difficulty and topics to config.json was to make sure our exercises are ordered in a sane way. But we have been paying attention to ordering ever since #127, so we miss that benefit.

I managed to add topics in #229 because we already had the topics ready to go.

So now one of our few incentives for adding difficulties is "we get to close one more issue", or something. That wasn't enough incentive for me to go through and rate all our exercises. If the rating showed on the website, then that'd be another story.

@kytrinyx
Copy link
Member Author

I think it would be fine to leave all the difficulties at 1 until we feel like we have a use for them. I'd just close the issue.

@petertseng
Copy link
Member

I have a compromise solution to propose to the track maintainers in #252.

petertseng added a commit that referenced this issue Jan 30, 2017
As noted in #195, difficulty rating is hard.

This commit proposes a difficulty rating scale that should be completely
unambiguous, requiring no judgment:

In the last version of problems.md before it was rewritten in #242:

* Everything in "Introduction" stays at 1.
* Everything in "Getting Rusty" gets a 4.
* Everything in "Rust Gets Strange" gets a 7.
* Everything in "Putting it all Together" gets a 10.

It is acknowledged that this is not a perfect difficulty scheme. See,
for example, how many exercises are at 4.
However, it is more accurate than leaving them all at 1 because it is
generally true that those exercises in "Getting Rusty" are more
difficult than those in "Introduction".
It is a starting point from which further adjustments can be made.
I posit that there is little motivation to adjust further until the
difficulty ratings actually show up on the website, since until then
they are just arbitrary numbers with no meaning.

Closes #179.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants