Fix unit tests to be fully portable #14
Conversation
…ly after checking out the project. - Enforce en-US locale for test execution : fixes 331 tests - Remove android logger dependency (as a library, jsword should only rely on the slf4j api) : fixes 78 tests - Automatically skip tests that rely on externally-installed data if this data is not available, and remove download of ESV2011 which is missing from crosswire repo : fixes 46 tests
UnasZole
force-pushed
the
fixUnitTests
branch
from
2fd514d
to
2844135
Compare
|
The second commit of this PR brings a change to BackendTest.testBackendStrongsGreekRawLd() which might require additional action to pass on pipeline builds. Indeed, it relies on StrongsGreek, but it seems that the StrongsGreek package currently available in CrossWire has been updated compared to the original test, as the content no longer matched. If we notice that the test fails on the pipeline, two solutions :
|
| if(project.hasProperty("tests")) { | ||
| implementation("org.slf4j:slf4j-api:1.7.6") | ||
| } else { | ||
| implementation("de.psdev.slf4j-android-logger:slf4j-android-logger:1.0.5") |
There was a problem hiding this comment.
When using in Android app (AndBible) we need this logger implementation.
There was a problem hiding this comment.
Could you provide PR for AndBible how to use this if we make this change?
There was a problem hiding this comment.
Well, I'm not familiar with android development (I'm a very experienced Java developer, but only for backend applications, not frontends so far) so I'm not equipped to run tests on and-bible. (I could take time to try it later on, though).
But the change is very simple : you can just add the dependency directly in this list : https://github.com/AndBible/and-bible/blob/develop/app/build.gradle.kts#L335
This should be enough :-)
…eferenced by tests and still available on CrossWire, in order to un-skip the corresponding tests
UnasZole
force-pushed
the
fixUnitTests
branch
from
2844135
to
7e47409
Compare
|
I've just updated the PR to simplify the testBackendStrongsGreekRawLd, to avoid the possible issue I mentioned above (due to the fact that the StrongsGreek module this test relies on has changed between what's currently available on CrossWire, and what's in your pipeline's testmods.zip). Now, the test should pass in both cases, so you won't need to update testmods at all :-) Could you approve the Workflow that is pending, so we can check if all the unit tests still pass in your test environment ? I'll open the PR to and-bible to update the dependencies to this new jsword version (and to the slf4j android logger) once it's published, if you want :-) |
|
It looks good to me. Will merge and see if I get it working in AndBible side. |
|
Interesting. It looks like it works without any changes in AndBible (and checked that Slf4jAndroidLogger log lines get into logcat). I wonder if de.psdev.slf4j-android-logger is obsoleted by standard android libs or more recent version of slf4j. |
|
The android logger slf4j implementation is probably already included transitively in some other dependencies of and-bible ! Thank a lot for the merge ! By the way, I saw that the CI build on the pull request had failed (cf. https://github.com/AndBible/jsword/actions/runs/7560282128/job/20605220349 ), apparently because it didn't manage to retrieve the URL of the testmods file (the secrets.DOWNLOAD_TEST_MODULES_URL was empty). |
Good point. Then it would be better to put dependency explicitly. AndBible/and-bible#3174
I noticed that too. I have a feeling it has worked in the past, but not any more for some reason. It works OK when running it for local branch (and I think also OK for local pull requests). I can give you write access to repository (master and develop branches are protected) so you can create branches locally and make prs from there. |
After checking first checking out the project, I originally got 457 unit test failures due to environmental issues.
This PR attempts at fixing most of them.
The numbers are approximate, because in the middle of that there are also some flaky tests remaining (that pass or fail a bit randomly) : specifically the JobTest seems to consistently pass within my IDE, but randomly fail when run from Gradle.
Notes :