Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing CI feature tests caused by docker warning message about the kernel #59

Closed
michaelbarton opened this issue Oct 28, 2015 · 1 comment

Comments

@michaelbarton
Copy link
Contributor

The current commit 3b13fc6 on the release/0.3.0 branch is failing on
the CI server. After some investigation I have found what is causing
the error. I'll write up the details here as this might help with similar
problems when debugging the verification functionality of the command line
interface.

  • The verification part of the CLI runs cucumber feature files which are
    maintained in the biobox_cli/verification folder. Each feature file file
    specifies how a how biobox type should behave - for example a feature file
    for a short read assembler, which tests out different user inputs - both
    correct and incorrect.
  • These verification feature files are part of the UX and so must be tested to
    ensure they do not break between builds. Therefore these feature files have
    corresponding feature files in the feature directory. This essentially is
    testing the testing code.
  • In commit 3b13fc6 the cucumber features for verification fail for
    the short read assembler biobox on the CI server but work when run locally.
    This indicates a problem specifically for the CI server environment.
  • I logged into circleci using the ssh button. This reruns the same failing
    build and also allows ssh access during the build. This makes interactive
    debugging possible as I can log in, rerun the feature tests, and inspect
    the outputs.
  • Each time a feature test is run, any created files are stored in the
    temporary directory tmp under the project root. The verification test
    output files are however not stored in this folder because they are being
    run indirectly by the top level feature tests. Instead the tmp folder
    contains the JSON output for each verification test.
  • This JSON contains the reason why a verification test fails. The
    verification code in the CLI parses this file for the failing scenario
    message. If there is an unsual reason why a verification test failed, this
    JSON file will contain the output. Specifically in the instance of this bug
    the error message is being written on the stderr when the feature test
    expects none to be produced:
"error_message": [
"Assertion Failed: The stderr should be empty but contains:",
"",
"WARNING: Your kernel does not support memory swappiness capabilities, memory swappiness discarded."
],
"status": "failed"

I contacted Circle CI and they say the LXC environment is responsible for this
error message.

@michaelbarton
Copy link
Contributor Author

Fixed in #60

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant