Check ordering of core exercises

[ErikSchierboom]

I just received some feedback from a student that he thought the sum-of-multiples exercise was way harder than the previous exercises. We should look into the current order and see if the difficulty is progressing as smoothly as possible.

[pdmoore]

Similarly, "Two Fer" comes after Bob and Raindrops. On the Java track "Two Fer" is the first exercise after Hello World. On C# track "Two Fer" is the 6th exercise and doesn't really add to what has already been covered.

[ErikSchierboom]

Thanks for the feedback! I'll come up with a new proposal soon.

[ErikSchierboom]

Some initial thoughts on this:

• two-fer should be much higher in the list, as it is actually quite easy.
• we should remove sum-of-multiples as a core exercise (see this thread).
• maybe accumulate isn't the best of core exercises, as people don't really learn much from it (except for the yield keyword).
• maybe markdown isn't a great core exercise, as it is primarily about refactoring and as such, does not teach any new skills.

[ErikSchierboom]

@jpreese @robkeim Your thoughts on this?

[ErikSchierboom]

This is the current list of core exercises and their topics:

• hello-world: strings
• leap: control_flow_if_else_statements, integers
• bob: control_flow_if_else_statements, strings
• sum-of-multiples: arrays, math, transforming
• raindrops: filtering, strings
• two-fer: optional_values, strings
• nucleotide-count: dictionaries, strings
• accumulate: extension_methods, sequences, transforming
• grade-school: lists, sorting
• kindergarten-garden: enumerations, parsing
• clock: classes, structural_equality
• binary-search: arrays, searching
• allergies: bitwise_operations, filtering
• saddle-points: arrays, matrices, tuples
• markdown: parsing, refactoring, transforming
• book-store: recursion

And these are the topics and their counts:

• strings: 5
• arrays: 3
• transforming: 3
• control_flow_if_else_statements: 2
• filtering: 2
• parsing: 2
• integers: 1
• math: 1
• optional_values: 1
• dictionaries: 1
• extension_methods: 1
• sequences: 1
• lists: 1
• sorting: 1
• enumerations: 1
• classes: 1
• structural_equality: 1
• searching: 1
• bitwise_operations: 1
• matrices: 1
• tuples: 1
• refactoring: 1
• recursion: 1

[robkeim]

Do we have data that we could use from before the release of Exercism v2 that would help us determine this order? Some values such as:

• The number of students that completed an exercise
• The number of iterations on average per exercise
• The order in which students completed the exercise

Might help give us more insight and the true "difficulty" of exercises as perceived by the students. With the release of v2 we can't trust the stats as much for the existing core exercises since we're forcing students into a specific order, but that wasn't the case in the first version.

[ErikSchierboom]

I don't think we have that data unfortunately. We'll have to rely on gut feeling, and what our experience as mentors tell us. I hope to make a proposal for a new order today.

[ErikSchierboom]

While working on the re-ordering, here are some of my thoughts on the current track's status:

• Some exercises unlock way more side exercises than others. This should be more spread out.
• We could probably add some more core exercises, to not unlock massive numbers of side exercises each time.
• We should probably add some of the more difficult exercises as core exercises.
• We could improve upon how core exercises unlock related side exercises, this is a bit random at times.

[ErikSchierboom]

This is my proposal for the new track progression:

Difficulty: easy

two-fer (core): optional_values, strings
-> reverse-string: strings
-> pangram: strings
-> isogram: filtering, strings
-> gigasecond (core): dates

space-age (core): floating point numbers
-> difference-of-squares: control_flow_loops, integers, math
-> sum-of-multiples: arrays, math, transforming
-> collatz-conjecture: algorithms, control_flow_conditionals, control_flow_loops, integers, math
-> grains: integers
-> armstrong-numbers: math

hamming (core): filtering, strings
-> error-handling: exception_handling
-> accumulate: extension_methods, sequences, transforming
-> strain: filtering, sequences
-> scrabble-score: transforming
-> protein-translation: lists, strings, transforming

nucleotide-count (core): dictionaries, strings
-> etl: dictionaries, lists, transforming
-> sieve: filtering, math
-> binary-search: arrays, searching
-> word-count: dictionaries, strings, transforming

bob (core): control_flow_if_else_statements, strings
-> raindrops: filtering, strings
-> rna-transcription: strings, transforming
-> phone-number: parsing, transforming
-> triangle: enumerations, integers

allergies (core): bitwise_operations, filtering
-> secret-handshake: arrays, bitwise_operations
-> perfect-numbers: integers, math
-> queen-attack: classes

robot-simulator (core): classes, enumerations
-> kindergarten-garden: enumerations, parsing
-> clock: classes, structural_equality
-> robot-name: classes, randomness, strings
-> grade-school: lists, sorting

beer-song (core): string building
-> proverb: string building
-> house: string building
-> twelve-days: string building
-> food-chain: string building

Difficulty: medium

grep (core): files, searching, strings
-> anagram: filtering, strings
-> series: arrays, strings, transforming
-> isbn-verifier: conditionals, loops, pattern_matching, strings
-> markdown: parsing, refactoring, transforming

yacht (core): games, parsing, sorting
-> poker: games, parsing, sorting
-> bowling: algorithms, control_flow_loops
-> dominoes: arrays, tuples
-> tournament: parsing, strings

simple-linked-list (core): classes, lists
-> linked-list: classes, lists
-> circular-buffer: classes, queues
-> custom-set: sets
-> sublist: lists
-> binary-search-tree: overloading, searching, trees
-> tree-building: refactoring, trees

meetup (core): dates
-> bank-account: classes, concurrency
-> saddle-points: arrays, matrices, tuples
-> complex-numbers: math, tuples
-> rational-numbers: math
-> prime-factors: integers, math

roman-numerals (core): control_flow_loops, transforming
-> wordy: parsing, strings, transforming
-> say: strings, transforming
-> acronym: strings, transforming
-> largest-series-product: integers, math, strings, transforming
-> ledger: globalization, refactoring, strings
-> pig-latin: strings, transforming

spiral-matrix (core): algorithms, matrices
-> scale-generator: algorithms, parsing
-> bracket-push: parsing, strings
-> ocr-numbers: parsing, pattern_recognition
-> matrix: matrices, parsing
-> minesweeper: parsing, transforming

rotational-cipher (core): algorithms, strings, transforming
-> simple-cipher: algorithms, strings, transforming
-> atbash-cipher: algorithms, strings, transforming
-> crypto-square: algorithms, strings, transforming
-> rail-fence-cipher: algorithms, strings, transforming

run-length-encoding (core): algorithms, transforming
-> nth-prime: math
-> diffie-hellman: algorithms, integers, math, transforming
-> luhn: algorithms, strings, transforming
-> parallel-letter-frequency: dictionaries, parallellism, transforming
-> pythagorean-triplet: integers, math, overloading
-> transpose: strings, transforming

book-store (core): recursion
-> all-your-base: integers, math, transforming
-> list-ops: lists, recursion
-> flatten-array: lists, recursion
-> pascals-triangle: arrays, control_flow_loops, math
-> dot-dsl: classes, domain_specific_languages, equality

Difficulty: hard

diamond (core): algorithms, strings
-> sgf-parsing: parsing, strings
-> connect: algorithms, strings
-> go-counting: algorithms, strings
-> rectangles: parsing, transforming
-> word-search: searching, tuples

zebra-puzzle (core): logic, laziness
-> two-bucket: logic
-> hangman: reactive programming
-> react: events

forth (core): stacks
-> alphametics: algorithms
-> change: arrays, integers

zipper (core): trees
-> pov: trees

The core progression path would become:

1. two-fer: optional_values, strings
2. space-age: floating point numbers
3. hamming: filtering, strings
4. nucleotide-count: dictionaries, strings
5. bob: control_flow_if_else_statements, strings
6. allergies: bitwise_operations, filtering
7. robot-simulator: classes, enumerations
8. beer-song: string building
9. grep: files, searching, strings
10. yacht: games, parsing, sorting
11. simple-linked-list: classes, lists
12. meetup: dates
13. roman-numerals: control_flow_loops, transforming
14. spiral-matrix: algorithms, matrices
15. rotational-cipher: algorithms, strings, transforming
16. run-length-encoding: algorithms, transforming
17. book-store: recursion
18. diamond: algorithms, strings
19. zebra-puzzle: logic, laziness
20. forth: stacks
21. zipper: trees

Here is my rationale for this progression:

• I've tried to select those exercises as core exercises that are easier than the ones they unlock. This gives students the ability to "dig deeper," if they want to.
• I've tried to select core exercises that will probably have solutions that are interesting to mentor/receive feedback on.
• The core exercises unlock similar side exercises, such that they are better related to the core. This will allow students to further hone their skills on a particular subject through the use of side exercises.
• I've added some more core exercises, to allow a smoother progression where each core exercise unlocks a small set of side exercises, to not overwhelm the user with side exercises.
• The core exercises are quite diverse in their topics, with little overlap.
• Move core exercises that often don't have an interesting solution (such as leap and accumulate) to side exercises.
• Ensure that all math-y exercises are side exercises (see this issue).

[ErikSchierboom]

@robkeim @pdmoore @jpreese The comment above contains my suggestion for an updated core exercise path. Any thoughts?

[ErikSchierboom]

@robkeim @jpreese Could you perhaps chime in with your thoughts on this thread: https://exercism-team.slack.com/messages/CASDGG84B/convo/CASDGG84B-1543225722.030400/

[robkeim]

@ErikSchierboom how do we get an access to that slack workspace?

[ErikSchierboom]

Hmmm, you're not an exercism mentor, are you? My bad. I'll just post the question here then:

I'm currently doing some thinking on how to better structure the C# track (which exercises are core, which order, etc.). One of the problem I have is determining what are the key, basic types one has to learn in the first few exercises. Currently I have string, int, bool and datetime. I was wondering if some collection type should also be introduced as a basic type, as collections are very important. To me, the most obvious choice would be an array. However, there are not many array-based exercises at the moment. One array exercise that is fairly simple, is high-scores. However, the most simple implementation would require one to use LINQ, which in itself is actually a bit more advanced concept as it builds on extension methods. Do you think we could get away with introducing high-scores as a basic exercise and point people to LINQ very soon in the track (they wouldn't have to understand how it works, just be able to use it), or would that be too complicated?

There was one response so far:

I've only done a handful of exercises, so take this with a grain of salt, but here are my initial thoughts:

1. Sum of Multiples (the fourth exercise in the track) ends up using LINQ pretty heavily. If we're worried about introducing LINQ, then we should move that one back further.
2. I'm frankly not all that afraid to introduce LINQ early on: it's possible that people won't grasp the nuances of extension methods and delegates, but the syntax and concepts are fairly straightforward: I would say more straightforward than new object construction and for loops. And it's used far more often in real C# code (in my experience) than for loops are.
3. Arrays are certainly very basic data structures, in the sense that they're closer to the metal, but in real life I think List<>s are more "basic" in the sense of being more commonly used and less likely to surprise you. If we're borrowing from a common set of exercises that assumes arrays are more fundamental than lists, then I'm not opposed to starting with arrays, but I wouldn't necessarily favor arrays as an earlier concept than lists, all else being equal.

To which I responsed:

Thanks for the very detailed response!

1. The sum-of-multiples exercise won't be a core exercise anymore, as there is a new policy that math-related exercises should not be core exercises.
2. Good to hear! I hope to hear from more people what their take on this is, but this is very hopeful to me.
3. I was really struggling with this, as I also initially considered List as the basic type. We can totally determine for ourselves if we prefer List over Array, so there is no pressure there.

[robkeim]

No I didn't join as a mentor when v2 was released.

Here's my $0.02 after having some time to think this over:

I agree that the notion of an enumerable should be introduced as a fundamental "data type" in C#. I don't think it makes a big difference whether we do that via an array or a list. That said, list might be slightly easier to understand since the notion of a list is something everyone is familiar with whereas the notion of an array is used only in mathematics.

I would be hesitant to introduce LINQ and particularly LINQ heavy exercises too early in the track. I think there are a few different "levels" of understanding LINQ:

1. I use LINQ but don't really understand the syntax
2. I use LINQ and understand concepts it uses (extension methods, lambda expressions, delegates)
3. I use LINQ and understand "how" it works (could implement it if asked to)

In my opinion, the jump to even get to the first level is pretty high particularly for someone that doesn't have SQL or functional programming experience. That said, if the "non-LINQ" equivalent solution isn't too complicated then leaving it early in the track shouldn't pose a problem as students can decide themselves whether they want to leverage LINQ or "write it by hand" so to speak.

[ErikSchierboom]

Thanks for the thoughtful feedback! Currently, the most basic list-based exercise is probably high-scores: https://github.com/exercism/problem-specifications/blob/master/exercises/high-scores/canonical-data.json. When this exercise is implemented, users could use the Sum() and Take() and OrderBy() LINQ methods to solve it. Do you think that would be too hard? If so, we might try to come up with a new, basic, list-based exercise.

[robkeim]

I think this exercise should be fine to solve without LINQ... although much more succinct using LINQ :)

Here's an untested proof of concept implementation I through together in 10 minutes only using List from the Collections namespace no other dependencies:

public class HighScores
{
    private List<int> _scores;

    public HighScores(List<int> scores)
    {
        _scores = new List<int>(scores);
    }

    public IEnumerable<int> Scores()
    {
        return _scores;
    }

    public int Latest()
    {
        return _scores[_scores.Count - 1];
    }

    public int PersonalBest()
    {
        var best = int.MinValue;

        foreach (var score in _scores)
        {
            if (score > best)
            {
                best = score;
            }
        }

        return best;
    }

    public List<int> PersonalTop()
    {
        var scores = new List<int>();
        scores.AddRange(_scores);
        scores.Sort();

        var result = new List<int>();

        var index = scores.Count - 1;

        while (result.Count < 3 && index >= 0)
        {
            result.Add(scores[index]);
            index--;
        }

        return result;
    }
}

[ErikSchierboom]

In my opinion, the jump to even get to the first level is pretty high particularly for someone that doesn't have SQL or functional programming experience.

This was also my gut feeling.

Thanks for putting the example non-LINQ solution together. It has helped me see how difficult a non-LINQ implementation would be. Seeing this implementation, I think that this exercise is great to introduce LINQ, but it should probably not be a basic exercise (which means one of the five first exercises).

CC @F3PiX

[ErikSchierboom]

The first stage of the re-ordering has been merged in #740.

[ErikSchierboom]

With #744 merged, I'm gonna close this issue. There will be continuous re-evaluation of the track's ordering, but the major re-ordering is done.