PRJenkins

Testing OMPI with Jenkins

Every Pull Request (PR) submitted to the ompi or ompi-release repositories is monitored by the Mellanox Jenkins server which will start a smoke-test.

• The smoke-test script repository is located here:

https://github.com/mellanox-hpc/jenkins_scripts

• The smoke-test script for OMPI is located here:

https://github.com/mellanox-hpc/jenkins_scripts/blob/master/jenkins/ompi/ompi_jenkins.sh

The smoke-test consists of tests organized by topics:

• build - runs autogen/configure/make sequence (default: on)
• examples - builds and run examples from ompi source tree (default: on)
• threads - builds ompi with threads support and run thread tests (default: on)
• src_rpm - create ompi.src.rpm (default: on) - (run make distclean, make distcheck, create src.rpm)
• check - runs "make check" after build (default: on)
• help_txt - runs contrib/check-help-strings.pl and test help files (default: off)
• oshmem - runs oshmem tests (default: on)
• vader - runs vader tests (default: on)
• known_issues - run tests for known issues (default: off)
• cov - runs coverity tests and generate html report (default: on)
• all - all of the above (default: off)

In order to select specific test topic for submitted pull request - one can:

• set the PR title (check "Edit" button next to the title) with a jenkins commands. (see below)
• add as PR comment in the special format (see below)

jenkins: check,help_txt,threads,src_rpm

and add comment to the PR, in form:

bot:retest

Also, it is possible to unselect specific test topics and re-trigger jenkins on PR. To unselect specific tests - please add "no" to the test token name, i.e.

jenkins: nosrc_rpm,nocheck
bot: retest

Please note - only last comment in PR is consulted by Jenkins script for user-selected action.

Viewing Jenkins tests result online

• The Jenkins test results for Pull requests for v1.8 branch can be found here:

http://bgate.mellanox.com/jenkins/job/gh-ompi-v1.8-pr/

• The Jenkins test results for Pull requests for master branch can be found here:

http://bgate.mellanox.com/jenkins/job/gh-ompi-master-pr/

• The jenkins test results for Merge commits to the master branch can be found here:

http://bgate.mellanox.com/jenkins/job/gh-ompi-master-merge/

• When viewing results online, you can access following byproducts for every job:

• Coverity report (code static analysis, reports number of detected error events per component in PR comment)

Hint: to suppress false positives coverity errors - use code annotations as described here: https://doclazy.wordpress.com/2011/07/14/coverity-suppressing-false-positives-with-cod/

• Check job console for error details
• Access job workspace and extract files from the build area (src.rpm, tarball, config.log,...)
• Workspace for master PRs - http://bgate.mellanox.com/jenkins/job/gh-ompi-master-pr/ws/
• Workspace for v1.8 PRs - http://bgate.mellanox.com/jenkins/job/gh-ompi-release-v1.8-pr/ws/

How to re-trigger Jenkins testing

Every newly submitted PR will be tested by Jenkins automatically, based on PR title/comment Jenkins directives (if not present - default tests topics will be used).

Jenkins will retest PR if it was updated with new code, or if a new comment added.

Also, one can re-trigger Jenkins manually with the following PR comment:

bot:retest

TODO

• Run valgrind with OMPI and track failures
• Run smoke tests with different compilers (now gcc, add intel, pgi)
• Add tests for PMI (srun) based execution (now using orte)
• Add extreme feedback device: http://www.jensjaeger.com/2010/04/extreme-feedback-device-the-batman-lamp/

Happy testing!