Implementing multi-step exercises?

[IanWhitney]

Today my Kata group discussed solutions for Queen Attack. We all thought it would be interesting to have some follow-up exercises. The idea is that the student would build-on and modify their code from the earlier exercises to solve later exercises.

One possible set of exercises could be:

1. Can 2 queens attack each other? (this is the standard set of queen attack tests)
2. Can 2 knights attack each other?
3. Given a board with a queen and knight, who can attack whom?
4. Given a board with 2 queens and 2 knights, who can attack whom?

Each exercise would introduce a new wrinkle into the existing code. Moving from Exercise 1 to 2 would get students thinking about different piece types, what behavior they have in common and what behavior they do not.

2 -> 3 Would introduce the ideas of non-reciprocal attacks. If queen A can attack queen B, then the reverse is true. Same for knights. But if a queen can attack a knight, then the reverse is false.

3 -> 4 Is a big leap. You have to introduce a board concept, because you have to start thinking about other pieces being in your way. Yes, that queen might be diagonal from your queen, but there's a knight in your path. Etc.

And so on. You can take this pretty far, obviously.

[kytrinyx]

I think this is a great idea, and that it would be an excellent extension of the iterative approach that Exercism already has. Off the top of my head I have no idea how we would implement it, nor what it would look like in terms of the user experience, and I would love to see a discussion here about how we might go about it, or perhaps how we might go about introducing a naive version of it in order to experiment with it.

[petertseng]

This is interesting. And even without any special support for it, it's already possible to add new exercises that are related to one another, it's just that there's no way to express to students their relation. Currently we have series which is a potential precursor to largest series product (Depending on how you choose to implement LSP). Obviously it would be better if there were some way to show they're related.

So when we decide a way to do this, it would likely be applicable to those two exercises.

Wasn't there a previous discussion somewhere of a kind of track-within-a-track kind of thing? I am struggling to find the issue now, but I was going to see if it was suitable to used for this kind of thing.

E: Found it. exercism/problem-specifications#71 . May be back with some more thoughts in a day or so after I've had time to read that and related discussions.

[IanWhitney]

I think there are two big hurdles:

1. Showing that they are related
2. Giving students starter code should they start in the middle of the series

I think we can address the first point through the Readme and the tests. The readme part is pretty self-explanitory. For the tests I think the tests in the later exercises should contain the tests from the earlier exercises.

Using my Chess suggestion as an example. The 2nd exercise (implementing knights) should also run the tests for the first part (implementing queens). Since the goal is to show the growth of code as new requirements are added, we want to also ensure that students don't introduce regression bugs.

I think we can handle the 2nd hurdle by linking to the example code from the previous exercise. So if a student starts with Knights, we link to the example code for implementing Queens.

[kotp]

Showing that they are related should not be such a big deal. We already have the code generate the test file. Just have it refer to their version of the prior exercise. The README and the tests are our very capable ways of communicating with the person any thing that needs to be communicated.

We already have them modify the tests by having them delete the 'skip' or comment it out, etc. Have a line that communicates that they can require their prior solution. The prior tests for the previous exercise already drives the API.

I don't see where this would be a problem., except for when someone moves to a new machine, and doesn't have their old solutions. Still, I think the cli already has a way to pull down last solutions?

[kytrinyx]

Right now each individual exercise is stand-alone. It doesn't assume anything about which exercises you've solved or in which order.

If you skip the first in a series, should it automatically skip you past all the related ones?

Should follow-on exercises be displayed on the website as being available to solve? Or should you just get them as a "step 2" after you've solved the first one?

[IanWhitney]

I know students can choose to fetch specific exercises, but isn't the default to simply fetch the next exercise in that language's config.json?

As for starting in the middle, I think that's handled by a link to starter code.

[kytrinyx]

Yeah, the default is to fetch the next one. Maybe that's good enough.

[IanWhitney]

Putting some of this conversation together into an example, here's what I think we have. I'll use Ruby as the example language.

config.json

"problems": [
  ...
  "queen-attack",
  "knight-attack",
  "queen-knight-attack",
  ...
]

The description of Knight Attack has some text like:

This exercise follows Queen Attack. If you have not done that problem, you can do so before starting this one or you can get the code from _link_to_ruby_queen_attack_example_

The test file for Knight Attack would include the tests from Queen Attack. I'm not sure the best way to do this. If the student has fetched Queen Attack, then we could require_relative it in. But I'm not sure if all languages can do that (I would think they could, but don't know). If a student hasn't fetched Queen Attack then this would fail. Perhaps we'd have to require that students at least fetch Queen Attack before doing Knight Attack?

Alternatively, the entire test suite of Queen Attack could be copied into Knight Attack's tests. But that seems like a maintenance problem. Every change to the Queen Attack test suite would have to be ported to each subsequent test suite.

[IanWhitney]

I have a semi-working version of this in Rust as well. If there's interest, I can put together a PR over the next few days and then we'd have an actual implementation to look at and discuss.

[kytrinyx]

I think you're right that copying the queen tests into knight tests would add a potential maintenance burden.

Can we get away with instructing people to (a) fetch the queen exercise if they haven't already and (b) copy the queen tests (and perhaps their solution) into the new place before getting started?

Would the code they submit include the queens-only code, or just the new stuff? (Would they be writing it in the same file?)

[kotp]

Of course there is a maintenance cost. But the queen tests and the knight tests may be individual. Nothing yet to say that Knights attack depends on something from Queens attack.

A follow on exercise where it is Knights Vs Queens may use behavior previously established, those contracts being testable and loadable, due to knowing the folder structure. A test that fails to load the require library should report that the exercise should be completed first, as this one relies on the prior exercise.

I don't think there should be any copying going on. Not to the other exercise folder. This is an anti-pattern in design anyway, we don't want duplicate code. We can use the code that is already there, or that is required to be where it was to get a 'passing' exercise.

From the users standpoint, the require_relative is '../queen_attack_test' and is "known" so there is no copying required, just a reference to where it is expected to be. If the require fails (returns false) then we can assume that it is not there, and can suppose that it may be not completed. However, if it is completed, it could be fetched.

Fetching the problem if it has been already solved is not a problem, we either indicate for them to use the exercism fetch <track> queen_attack, if that is the one that is missing, or refer them to use exercism --help with a hint that they will need to fetch their solution, or fetch the exercise and complete it.

The "maintenance cost" then is the burden of the nitpiker, to complete the prior exercise to specifications, the "development cost" is to design it so that it complies with anything else that it relies on.

[IanWhitney]

I think I initially said that it would be nice if running the Knight tests also ran the Queen tests. But thinking about it more, it is probably totally unnecessary since the Knight tests will implicitly test Queen behavior. For example:

test_queen_attack.rb

def test_horizontal_attack
  white_queen = Queen.new([2,4])
  black_queen = Queen.new([2,6])
  assert white_queen.can_attack?(black_queen)
end

Then in test_knight_attack

def test_knight_can_attack_queen
  knight = Knight.new([2,4])
  queen = Queen.new([3,5])
  assert knight.can_attack?(queen)
  refute queen.can_attack?(knight)
end

If the student has broken the queen's implementation, that test will raise an error or fail. Hopefully.

So there's probably no reason to run the Queen tests while working on the Knight implementation.

[kotp]

@IanWhitney That is what I am thinking.

The exercism client side structure is such that this line should be sufficient to find out if we have a Queen to even use, so in the test, may be something like this:

(Since your code looks like it may be Ruby influenced)...

begin
  require_relative '../queen_attack/queen_attack'
rescue LoadError => e
  STDERR.puts "I can not load the Queen Attack exercise files, please either fetch your solution for that exercise, or complete that exercise before attempting this one."
  STDERR>puts e
end

Not tested, but just a model.

[kytrinyx]

A follow on exercise where it is Knights Vs Queens may use behavior previously established, those contracts being testable and loadable, due to knowing the folder structure.

I don't know if all languages will allow loading based on knowing relative paths.

[kotp]

Neither do I, but if there is any way to load code that is not in the same file, then that is something the ones familiar with their tracks are likely to be able to do. The problem is common and so I would be surprised if there is not a mechanism.

Even if the 'relative path' type functionality isn't built in, I think all languages have a way to get the current working path. From there it may be (and should be) trivial.