Skip to content

Conversation

@sanderploegsma
Copy link
Contributor

@sanderploegsma sanderploegsma commented Aug 22, 2023

I was investigating why the bug in #2330 didn't cause a failure in CI, and apparently the exercises aren't tested at all due to an error in the test script!

>>> solve_all_exercises(exercism_exercises_dir="/home/runner/work/java/java/build/exercism")
+ solve_all_exercises /home/runner/work/java/java/build/exercism
+ local exercism_exercises_dir=/home/runner/work/java/java/build/exercism
+ echo '>>> solve_all_exercises(exercism_exercises_dir="/home/runner/work/java/java/build/exercism")'
++ pwd
+ local track_root=/home/runner/work/java/java
++ jq '.exercises[].slug + " "' --join-output
++ cat config.json
jq: error (at <stdin>:2069): Cannot index array with string "slug"

It appears there are multiple things wrong with this script:

  • It ignores errors from commands in a pipe, causing it to swallow the failure
  • Apparently the bin/journey-test.sh script wasn't updated to parse config.json correctly after the addition of concept exercises, so none of the tests even ran

After some fiddling with the script I noticed that the exercises/build.gradle script is configured to do exactly what is required in CI: it pre-processes the test files to remove any @Ignore tags and then runs the tests against the exemplar/example implementation. So I figured the easiest way to fix the CI pipeline is to just run gradle test.

Note that because the CI pipeline has probably been broken for a while, there were a few exercises of which the example implementation doesn't pass their own test cases. I fixed the easy ones, but the following practice exercises seem to require a bit more attention to get their tests passing:

  • Pythagorean Triplet
  • SGF Parsing

Reviewer Resources:

Track Policies

@sanderploegsma sanderploegsma changed the title Sync CI test script with Kotlin track Fix CI silently not running exercise tests Aug 22, 2023
This is not the best solution, probably it would be better if the CI script is able to parse the config.json file and copy all files listed in `files.editor` and `files.exemplar`, but for now duplicating the editor files into the exemplar implementation should help to unblock the CI build.
By making `test` depend on `copyTestsFilteringIgnores`, Gradle would run `compileTestJava` -> `copyTestFilteringIgnores` -> `Test`. This fails because there is nothing to compile in the test sourceset, since it has not been generated yet.

The correct task order should be `copyTestsFilteringIgnores` -> `compileTestJava` -> `Test`.
@ErikSchierboom
Copy link
Member

@sanderploegsma Could you look into the CI failure please?

@sanderploegsma
Copy link
Contributor Author

Copy link
Member

@ErikSchierboom ErikSchierboom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants