You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current commit 3b13fc6 on the release/0.3.0 branch is failing on
the CI server. After some investigation I have found what is causing
the error. I'll write up the details here as this might help with similar
problems when debugging the verification functionality of the command line
interface.
The verification part of the CLI runs cucumber feature files which are
maintained in the biobox_cli/verification folder. Each feature file file
specifies how a how biobox type should behave - for example a feature file
for a short read assembler, which tests out different user inputs - both
correct and incorrect.
These verification feature files are part of the UX and so must be tested to
ensure they do not break between builds. Therefore these feature files have
corresponding feature files in the feature directory. This essentially is
testing the testing code.
In commit 3b13fc6 the cucumber features for verification fail for
the short read assembler biobox on the CI server but work when run locally.
This indicates a problem specifically for the CI server environment.
I logged into circleci using the ssh button. This reruns the same failing
build and also allows ssh access during the build. This makes interactive
debugging possible as I can log in, rerun the feature tests, and inspect
the outputs.
Each time a feature test is run, any created files are stored in the
temporary directory tmp under the project root. The verification test
output files are however not stored in this folder because they are being
run indirectly by the top level feature tests. Instead the tmp folder
contains the JSON output for each verification test.
This JSON contains the reason why a verification test fails. The
verification code in the CLI parses this file for the failing scenario
message. If there is an unsual reason why a verification test failed, this
JSON file will contain the output. Specifically in the instance of this bug
the error message is being written on the stderr when the feature test
expects none to be produced:
"error_message": [
"Assertion Failed: The stderr should be empty but contains:",
"",
"WARNING: Your kernel does not support memory swappiness capabilities, memory swappiness discarded."
],
"status": "failed"
I contacted Circle CI and they say the LXC environment is responsible for this
error message.
The text was updated successfully, but these errors were encountered:
The current commit 3b13fc6 on the release/0.3.0 branch is failing on
the CI server. After some investigation I have found what is causing
the error. I'll write up the details here as this might help with similar
problems when debugging the verification functionality of the command line
interface.
maintained in the
biobox_cli/verification
folder. Each feature file filespecifies how a how biobox type should behave - for example a feature file
for a short read assembler, which tests out different user inputs - both
correct and incorrect.
ensure they do not break between builds. Therefore these feature files have
corresponding feature files in the
feature
directory. This essentially istesting the testing code.
the short read assembler biobox on the CI server but work when run locally.
This indicates a problem specifically for the CI server environment.
ssh
button. This reruns the same failingbuild and also allows ssh access during the build. This makes interactive
debugging possible as I can log in, rerun the feature tests, and inspect
the outputs.
temporary directory
tmp
under the project root. The verification testoutput files are however not stored in this folder because they are being
run indirectly by the top level feature tests. Instead the
tmp
foldercontains the JSON output for each verification test.
verification code in the CLI parses this file for the failing scenario
message. If there is an unsual reason why a verification test failed, this
JSON file will contain the output. Specifically in the instance of this bug
the error message is being written on the stderr when the feature test
expects none to be produced:
I contacted Circle CI and they say the LXC environment is responsible for this
error message.
The text was updated successfully, but these errors were encountered: