Ensuring good canonical test coverage: List exercises where it'd be good to anti-verify

[petertseng]

I just made that term up, I guess.

It means, what exercises have potentially incorrect implementations that are important enough we should make sure the canonical data contains a test case that that incorrect implementation would fail?

Give priority to any exercises where it is difficult to tell through simple visual inspection whether a potential implementation would fail (for example, it's pretty easy to tell whether the acronym cases cover all interesting incorrect implementations)

• book-store: Greedy.
• bracket-push: Simple counter, only checks balancing not nesting. Forget to check for closing, only check nesting not balancing.
• change: greedy, Change: Add test for incorrect greedy algorithm #882
• circular-buffer: Always dump a value on overwrite
• connect: impl that connects on +1/-1. Impls that omit any one of the needed directions.
• dominoes: dominoes: Check that last can match first; Check that output dominoes = input dominoes ocaml#140: always [1, 2]; always [1, 1]; always [1, 1] times expected length
• isbn-verifier: isbn-verifier: missing test case for non-digit character in the middle #1212 and isbn-verifier: Tune too-long inputs to catch more incorrect algorithms #1199 - this is my highest priority, because there are too many incorrect implementations and it's not possible to tell by hand which implementations accept or reject a given case, since it needs more math than can be done mentally.
• leap: leap: Improve test suite to cover corner case. #955 and leap: Fix loophole in unit tests #971. This is probably fine to check visually, but I'm probably doing this one first anyway, to test that the anti-verify code works.
• list-ops: see if the "foldl' isn't just foldr . flip" test is applicable here
• luhn: checking from the wrong direction
• pangram: Checks for 26 unique characters, for some variable definition of character: add pangram tests #228, fix pangram test for duplicate mixed-case chars #852
• react: calling callbacks twice if the value changes because of both dependencies changing. or twice if the value doesn't change but both dependencies changed.
• word-search: Implementations that omit any of the eight directions.

Exercises that do have incorrect implementations, but I've chosen not to automatically anti-verify for the following reasons:

• bowling: There are probably a few here, but I've forgotten them. Probably look at Update bowling canonical-data #391, [Bowling] Add edge case with fill balls in a last-frame strike #418, bowling: Add tests for more than 10 after last frame #444, bowling: add tests for rolling after bonus rolls #1189. Need to think about how to actually write a broken impl first.
• dominoes: impl that only checks eulerian possibility (sum of all cardinalities is even). Such an impl can't actually make a chain.
• sum-of-multiples: double-counting. Not worth the effort to implement. Any reasonable set of test cases already tests double-counting.
• transpose: Transpose: add test-case #1046 ? I need a concrete incorrect implementation.

Closing this issue since this repo cannot usefully take action on it (but please still list any incorrect implementations that are important to you).

[petertseng]

TODO: add dominoes unconditional [1, 2], for exercism/ocaml#140 done