add support for testing easyconfig PRs from eb command line #920

boegel · 2014-05-08T15:44:26Z

example usage (see also easybuilders/easybuild-easyconfigs#857 (comment)):

$ eb --test-easyconfigs-pr=857 --robot --github_user=jenkins1 --force
== temporary log file in case of crash /tmp/easybuild-FqdQKW/easybuild-1aiA1p.log
== resolving dependencies ...
== processing EasyBuild easyconfig /tmp/easybuild-FqdQKW/files_pr857/SOAPaligner-2.21_Linux-x86_64.eb
== building and installing SOAPaligner/2.21_Linux-x86_64...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== packaging...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file /user/scratchdelcatty/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/SOAPaligner/2.21_Linux-x86_64/easybuild/easybuild-SOAPaligner-2.21-20140508.174314.log
== Build succeeded for 1 out of 1
== temporary log file /tmp/easybuild-FqdQKW/easybuild-1aiA1p.log has been removed.
== temporary directory /tmp/easybuild-FqdQKW has been removed.
== Test report, uploaded to https://gist.github.com/b7db9f056ef8eb2c83e8 and mentioned in a comment in easyconfigs PR#857:
Test result:
    Build succeeded for 1 out of 1 (1 easyconfigs in this PR)

Overview of tested easyconfigs (in order):
    [OK] SOAPaligner-2.21_Linux-x86_64.eb

Time info:
    start: Thu, 08 May 2014 15:43:00 +0000 (UTC)
    end: Thu, 08 May 2014 15:43:14 +0000 (UTC)

EasyBuild info:
    easybuild-framework version: 1.13.0dev
    easybuild-easyblocks version: 1.13.0dev
    command line:
        eb --test-easyconfigs-pr=857 --robot --github_user=jenkins1 --force
    full configuration (includes defaults):
        --buildpath=/user/home/gent/vsc400/vsc40023/.local/easybuild/build
        --config=/user/scratchdelcatty/gent/gvo000/gvo00002/vsc40023/easybuild_easy_installed/lib/python2.6/site-packages/easybuild_framework-1.13.0dev-py2.6.egg/easybuild/easybuild_config.py
        --disable-allow-modules-tool-mismatch
        --disable-avail-easyconfig-constants
        --disable-avail-easyconfig-licenses
        --disable-avail-easyconfig-params
        --disable-avail-easyconfig-templates
        --disable-avail-module-naming-schemes
        --disable-avail-modules-tools
        --disable-avail-repositories
        --disable-debug
        --disable-dry-run
        --disable-dry-run-short
        --disable-experimental
        --disable-ignore-osdeps
        --disable-info
        --disable-job
        --disable-list-toolchains
        --disable-logtostdout
        --disable-pretend
        --disable-quiet
        --disable-recursive-module-unload
        --disable-regtest
        --disable-regtest-online
        --disable-sequential
        --disable-set-gid-bit
        --disable-show-default-moduleclasses
        --disable-skip
        --disable-skip-test-cases
        --disable-sticky-bit
        --force
        --github_user=jenkins1
        --ignore-dirs=.git,.svn
        --installpath=/user/home/gent/vsc400/vsc40023/.local/easybuild
        --logfile-format=easybuild,easybuild-%(name)s-%(version)s-%(date)s.%(time)s.log
        --module-naming-scheme=EasyBuildModuleNamingScheme
        --moduleclasses=bio,chem,data,devel,ide,lib,mpi,perf,system,tools
        --modules-tool=EnvironmentModulesC
        --oldstyleconfig
        --repository=FileRepository
        --repositorypath=/user/home/gent/vsc400/vsc40023/.local/easybuild/ebfiles_repo
        --robot=/user/scratch/gent/vsc400/vsc40023/easybuild_easy_installed/lib/python2.6/site-packages/easybuild_easyconfigs-1.13.0.0dev-py2.6.egg/easybuild/easyconfigs
        --sourcepath=/user/home/gent/vsc400/vsc40023/.local/easybuild/sources
        --strict=warn
        --subdir-modules=modules
        --subdir-software=software
        --test-easyconfigs-pr=857
        --tmp-logdir=/tmp

System info:
    core_count: 16
    cpu_model: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
    cpu_speed: 2600.175
    cpu_vendor: Intel
    gcc_version: gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4); Copyright (C) 2010 Free Software Foundation, Inc.; This is free software; see the source for copying conditions.  There is NO; warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.; ; 
    hostname: node2154.delcatty.os
    kernel_name: Linux
    libc_version: ldd (GNU libc) 2.12; Copyright (C) 2010 Free Software Foundation, Inc.; This is free software; see the source for copying conditions.  There is NO; warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.; Written by Roland McGrath and Ulrich Drepper.; 
    os_name: SL
    os_type: Linux
    os_version: 6.5
    platform_name: x86_64-unknown-linux
    python_version: 2.6.6 (r266:84292, Jan 22 2014, 05:06:49) ; [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)]
    system_gcc_path: /usr/bin/gcc
    system_python_path: /usr/bin/python

List of loaded modules:
    cluster/delcatty

Environment:
    EASYBUILDBUILDPATH: /tmp/vsc40023/easybuild_build
    EASYBUILDPREFIX: /user/scratchdelcatty/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge
    EASYBUILDSOURCEPATH: /user/data/gent/vsc400/vsc40023/EasyBuild/sources:/apps/gent/source:/nfsmuk/apps/gent/source/
    G_BROKEN_FILENAMES: 1
    HISTCONTROL: ignoredups
    HISTSIZE: 1000
    HOME: /user/home/gent/vsc400/vsc40023
    HOSTNAME: node2154.delcatty.os
    LANG: en_US.utf8
    LC_ALL: en_US.utf8
    LC_CTYPE: UTF-8
    LESSOPEN: |/usr/bin/lesspipe.sh %s
    LOADEDMODULES: cluster/delcatty
    LOGNAME: vsc40023
    LS_COLORS: 
    MAIL: /var/spool/mail/vsc40023
    MODULEPATH: /etc/modulefiles/vsc:/user/scratchdelcatty/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/modules/all
    MODULESHOME: /usr/share/Modules
    PATH: /user/home/gent/vsc400/vsc40023/lmod/5.1.5/libexec:/user/home/gent/vsc400/vsc40023/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/user/scratch/gent/vsc400/vsc40023/easybuild_easy_installed/bin:/user/home/gent/vsc400/vsc40023/scratch/easybuild_easy_installed/bin
    PBS_DEFAULT: master15.delcatty.gent.vsc
    PWD: /user/home/gent/vsc400/vsc40023
    PYTHONPATH: /user/scratch/gent/vsc400/vsc40023/easybuild_easy_installed/lib/python2.6/site-packages:/user/scratch/gent/vsc400/vsc40023/easybuild_easy_installed/lib64/python2.6/site-packages:/user/home/gent/vsc400/vsc40023/.local/lib/python2.6/site-packages:/user/home/gent/vsc400/vsc40023/scratch/easybuild_easy_installed/lib/python2.6/site-packages:/user/home/gent/vsc400/vsc40023/scratch/easybuild_easy_installed/lib64/python2.6/site-packages
    SHELL: /bin/bash
    SHLVL: 2
    SSH_CLIENT: 172.24.13.3 42501 22
    SSH_CONNECTION: 172.24.13.3 42501 172.24.12.154 22
    SSH_TTY: /dev/pts/0
    TERM: xterm-256color
    USER: vsc40023
    VSCPROFILELOADED: 1
    VSC_ARCH_LOCAL: sandybridge
    VSC_DATA: /user/data/gent/vsc400/vsc40023
    VSC_DATA_VO: /user/data/gent/gvo000/gvo00002
    VSC_DATA_VO_USER: /user/data/gent/gvo000/gvo00002/vsc40023
    VSC_HOME: /user/home/gent/vsc400/vsc40023
    VSC_INSTITUTE: gent
    VSC_INSTITUTE_CLUSTER: delcatty
    VSC_INSTITUTE_LOCAL: gent
    VSC_OS_LOCAL: SL6
    VSC_SCRATCH: /user/scratch/gent/vsc400/vsc40023
    VSC_SCRATCHGULPIN: /user/scratchgulpin/gent/vsc400/vsc40023
    VSC_SCRATCH_CLUSTER: /user/scratchdelcatty/gent/gvo000/gvo00002/vsc40023
    VSC_SCRATCH_DELCATTY: /user/scratchdelcatty/gent/vsc400/vsc40023
    VSC_SCRATCH_DELCATTY_VO: /user/scratchdelcatty/gent/gvo000/gvo00002
    VSC_SCRATCH_DELCATTY_VO_USER: /user/scratchdelcatty/gent/gvo000/gvo00002/vsc40023
    VSC_SCRATCH_GENGAR: /user/scratchgengar/gent/vsc400/vsc40023
    VSC_SCRATCH_GENGAR_VO: /user/scratchgengar/gent/gvo000/gvo00002
    VSC_SCRATCH_GENGAR_VO_USER: /user/scratchgengar/gent/gvo000/gvo00002/vsc40023
    VSC_SCRATCH_GULPIN: /user/scratchgulpin/gent/vsc400/vsc40023
    VSC_SCRATCH_GULPIN_VO: /user/scratchgulpin/gent/gvo000/gvo00002
    VSC_SCRATCH_GULPIN_VO_USER: /user/scratchgulpin/gent/gvo000/gvo00002/vsc40023
    VSC_SCRATCH_NODE: /local
    VSC_SCRATCH_SITE: /user/scratch/gent/vsc400/vsc40023
    VSC_SCRATCH_VO: /user/scratch/gent/gvo000/gvo00002
    VSC_SCRATCH_VO_USER: /user/scratch/gent/gvo000/gvo00002/vsc40023
    VSC_VO: gvo00002
    _: /usr/bin/python
    _LMFILES_: /etc/modulefiles/vsc/cluster/delcatty
    module: () {  eval `/usr/bin/modulecmd bash $*`
}

…ine options

…gs, etc.

…ersion in test report

Conflicts: easybuild/main.py

wpoely86 · 2014-05-08T15:48:52Z

easybuild/tools/agithub.py

+    try:
+        conn.request(method, url, body, headers)
+    except socket.gaierror, err:
+        fancylogger.getLogger().raiseException("Failed to issue HTTP request.")


Maybe say HTTP request for the github API?

stdweird · 2014-05-08T16:23:00Z

easybuild/main.py

+        test_report = '\n'.join(test_report)
+        descr = "EasyBuild test report for easyconfigs PR #%s" % pr_nr
+        fn = 'easybuild_test_report_easyconfigs_pr%s_%s.txt' % (pr_nr, strftime("%Y%M%d-UTC-%H-%M-%S", gmtime()))
+        gist_url = create_gist(test_report, descr=descr, fn=fn, github_user=options.github_user, github_token=github_token)


for all i care, this report can be always created with each build, and added to the easybuildlogdir. only in case of a pr create the gist and comment?

creating the test report for each build separately doesn't make sense, since it's a general overview of the whole run (which may include multiple builds)

I can (and will) always mention the test report in the overall debug log though, and provide a --dump-test-report option.

stdweird · 2014-05-15T07:27:16Z

easybuild/tools/agithub.py

@@ -28,12 +28,13 @@
 Interface to GitHub.


@JensTimmerman or @boegel can you make sure this is in sync with hpcugent/vsc-base#119? also when that PR is merged, a new PR should remove this module

@JensTimmerman made some changes in there, that may affects things in here

but I can tackle that as soon hpcugent/vsc-base#119 is merged in

we'll still need agithub.py though, since hpcugent/vsc-base#119 doesn't feature the GitHub interface (but we should reuse Client)

wpoely86 · 2014-05-15T10:01:40Z

easybuild/tools/systemtools.py

+    if os_type == LINUX:
+        glibc_tool = 'ldd'
+        glibc_ver_str = get_tool_version(glibc_tool)
+        glibc_ver_regex = re.compile(r"^%s.*?(\d[\d.]*).*$" % glibc_tool)


The output of ldd is always: ldd (system string) version. What if their is a version number in the system string?

wpoely86 · 2014-05-15T11:05:20Z

easybuild/tools/systemtools.py

+
+    if os_type == LINUX:
+        glibc_ver_str = get_tool_version('ldd')
+        glibc_ver_regex = re.compile(r"^ldd \([^)]*\) (\d[\d.]*).*$")


maybe a last nit pic remark: I've always understood that unless you will use a regex very often, it's no point in compiling it yourself? re already keeps a cache.

I'm also using glibc_ver_regex.pattern deeper down, so I need this (I don't want to store the regex in a string first to use in re.search or something like that)

stdweird · 2014-05-15T11:56:43Z

@boegel some minor remark, but looks ok overall...

boegel · 2014-05-15T12:36:09Z

Jenkins: test this please

boegel · 2014-05-15T12:40:36Z

Jenkins: test this please

boegel · 2014-05-15T12:54:06Z

Jenkins: test this please

wdpypere · 2014-05-15T14:34:29Z

Jenkins: test this please

boegel · 2014-05-15T14:38:54Z

Jenkins: test this please

boegel · 2014-05-15T14:57:59Z

Jenkins: test this please

wdpypere · 2014-05-15T15:15:30Z

@boegel fixed

boegel · 2014-05-15T15:24:54Z

Thanks for the extensive reviewing @stdweird, and @wpoely86!

Going in!

add support for testing easyconfig PRs from eb command line

fgeorgatos · 2014-05-15T15:37:33Z

Congrats! This may prove to be the single most important step to help with
QA of the PRs!
On 15 May 2014 17:24, "Kenneth Hoste" notifications@github.com wrote:

Merged #920 #920.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/920#event-121512285
.

boegel added 16 commits May 7, 2014 18:29

add support for fetching easyconfigs from a PR via eb --from-pr

a645732

add unit test for --from-pr

8fd37e7

make sure PR is in a 'clean' mergeable state

b811093

fix using post with authentication in agithub module

c472de3

add --github-token, --github-user and --test-easyconfigs-pr command l…

6943879

…ine options

add functions to create a gist and post a comment in an issue

ba76adb

implement support for reporting back with PR test result

1bda0e6

fix typo

266e9de

fix another typo

a45b96c

add support for fetching GitHub token from keyring

985f8f9

get rid of github_token cmdline option to avoid leaking tokens via lo…

4247f56

…gs, etc.

further flesh out reporting back on tested easyconfigs PR

aed807a

include time, system and EasyBuild info in test report

f5d71e1

include short system info in comment, reenable posting comment

e1438dd

add module list, dump of environment, EasyBuild configuration, libc v…

1d6d806

…ersion in test report

Merge branch 'develop' into test_pr

0cafb49

Conflicts: easybuild/main.py

wpoely86 reviewed May 8, 2014
View reviewed changes

get rid of unused --regtest-online option

635d0b5

stdweird reviewed May 8, 2014
View reviewed changes

boegel added 2 commits May 14, 2014 21:43

fix remarks

58270da

fix last remarks

f9c0b44

stdweird reviewed May 15, 2014
View reviewed changes

fix help message for --test-easyconfigs-pr

09ddd62

boegel mentioned this pull request May 15, 2014

Libc version #921

Merged

stdweird mentioned this pull request May 15, 2014

added a rest api tool hpcugent/vsc-base#119

Merged

boegel added 3 commits May 15, 2014 11:36

use proper function for determining gcc/glibc versions

ea0b729

enhance systemtools unit tests

f0c9625

fix regex for determining version number from ldd --version output

2a68c5e

wpoely86 reviewed May 15, 2014
View reviewed changes

boegel added 3 commits May 15, 2014 12:53

fix regex used to fetch ldd version

d3a9c15

remove --test-easyconfigs-pr, add --upload-test-report instead

a8873b3

avoid greedy regex on ')'

b8af09b

wpoely86 reviewed May 15, 2014
View reviewed changes

don't convert specific PR to int, already handled via options parser

db19c35

boegel added a commit that referenced this pull request May 15, 2014

Merge pull request #920 from boegel/test_pr

5df522a

add support for testing easyconfig PRs from eb command line

boegel merged commit 5df522a into easybuilders:develop May 15, 2014

boegel deleted the test_pr branch May 15, 2014 15:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support for testing easyconfig PRs from eb command line #920

add support for testing easyconfig PRs from eb command line #920

boegel commented May 8, 2014

wpoely86 May 8, 2014

boegel May 9, 2014

stdweird May 8, 2014

boegel May 9, 2014

stdweird May 15, 2014

boegel May 15, 2014

wpoely86 May 15, 2014

boegel May 15, 2014

wpoely86 May 15, 2014

boegel May 15, 2014

stdweird commented May 15, 2014

boegel commented May 15, 2014

boegel commented May 15, 2014

boegel commented May 15, 2014

wdpypere commented May 15, 2014

boegel commented May 15, 2014

boegel commented May 15, 2014

wdpypere commented May 15, 2014

boegel commented May 15, 2014

fgeorgatos commented May 15, 2014

add support for testing easyconfig PRs from eb command line #920

add support for testing easyconfig PRs from eb command line #920

Conversation

boegel commented May 8, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stdweird commented May 15, 2014

boegel commented May 15, 2014

boegel commented May 15, 2014

boegel commented May 15, 2014

wdpypere commented May 15, 2014

boegel commented May 15, 2014

boegel commented May 15, 2014

wdpypere commented May 15, 2014

boegel commented May 15, 2014

fgeorgatos commented May 15, 2014