Skip to content

Add test command to run any unit test#1092

Merged
ErikSchierboom merged 23 commits intoexercism:mainfrom
xavdid:main
Jul 28, 2023
Merged

Add test command to run any unit test#1092
ErikSchierboom merged 23 commits intoexercism:mainfrom
xavdid:main

Conversation

@xavdid
Copy link
Copy Markdown
Contributor

@xavdid xavdid commented Jun 25, 2023

Per discussion in this forum post, I've added a test command to the exercism CLI. Much like the universal test runner it's inspired by, its goal is to be able to run the basic unit test suite for any exercism exercise.

It does this by:

  1. reading .exercism/metadata.json to get the track name
  2. doing a lookup in a table which maps track name => a test command (and some configuration; see below)
  3. maybe reading .exercism/config.json to get test file name(s) and including them
  4. including any args from exercism test
  5. Using go's exec.Command to run and return info about the exit code

Types of Tests

Based on my (very brief) sampling of exercism tracks, there are 2 main ways tests are run:

  1. Using a language-wide common command (e.g cargo test, go test, lien test, etc)
  2. A single test file invoked like any program of that language (e.g. ruby lasagna_test.rb)

So, the TestConfiguration struct covers both of those cases (by including an option to append the test file(s) to the command).

The other configuration option aids in arg passing. Some language tools (namely cargo) expect arguments to the test command to be after a double dash (--). Unfortunately, so does Cobra, the go arg parser. So, I was running into the following trying to run rust tests with --include-ignored:

  1. exercism test --include-ignored: --include-ignored is not a valid flag for the test command
  2. exercism test -- --include-ignored: --include-ignored is not a valid flag for cargo, put it behind a --
  3. exercism test -- -- --include-ignored: works as expected

This isn't great UX though, so I added an option for languages to be able to ask that args to their test runner be put behind an implied --. This way, the rust command works when run as option 2 above (which is in line with most other languages).

TODO

  • add tests
  • add more languages (I don't know that we need to ship with 100% coverage, but the top 20 most popular languages would be great)
  • add docs / changelog

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

x:rep/massive Massive amount of reputation x:size/massive Massive amount of work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants