Setup buildbot for OpenBLAS

[xianyi]

Hi all,

I plan to setup buildbot for OpenBLAS. Dose anybody have some experiences?
Just go through the buildbot manual. It looks like I need to buy a VPS to run build-master. Then, I setup the build-slaver on OpenBLAS dev machines.

[xianyi]

http://build.openblas.net/

Setup the following builder

• core2-linux
• nehalem-linux
• sandybridge-linux
• haswell-linux
• barcelona-linux
• bulldozer-linux
• piledriver-linux

[matthew-brett]

Sorry to be slow to get back to you. I have a lot of buildbot experience; let me know if I can help.

[xianyi]

@matthew-brett , Do you know the reason for the following error? unstable network?

[Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.

[matthew-brett]

I don't think I've seen these, at least, not as consistently as you have. Are the the slave machines on the same network as the master?

For my setup, I run no builds on the master machine - do you have the same?

[xianyi]

@matthew-brett , the master and slave are not at same network.

[matthew-brett]

@stefanv @yarikoptic - have you seen this error on your setups?

@xianyi - I assume there was no reason the buildbot slaves could have restarted during the build?

What versions of masters / slaves do you have?

[xianyi]

The slaves are in China. The master is a VPS in US. I don't think the slave has restarted during the build.

[matthew-brett]

Could this be relevant? http://trac.buildbot.net/ticket/2147

Is it possible the master has got overloaded?

Which buildslave / buildmaster versions are you running?

[xianyi]

build master

  buildbot --version
Buildbot version: 0.8.12
Twisted version: 13.2.0

build slave

buildslave --version
Buildslave version: 0.8.8
Twisted version: 13.2.0

[xianyi]

I just login to the slave. Although the twisted connection is closed, the test scripts (docker) are still running.

[matthew-brett]

Maybe worth upgrading the buildslaves to latest?

[brada4]

@xianyi would it be possible to set up clang scan-build (easy to set up with ubuntu/debian and "official" clang-3.7 clang-3.8). it is quite reveling regarding uninitialized variables

[xianyi]

@brada4 , OK.
I will try to add clang

[xianyi]

@brada4 , I added a clang build step at quick-xxx-linux builder. You can check the output http://build.openblas.net/builders/quick-core2-linux/builds/43/steps/shell_1/logs/stdio

Meanwhile, on OS X, the default compiler is clang.

[brada4]

I did not mean alias gcc=clang, i meant alias make='scan-build-3.8 make'

e.g. within limits of my understanding

Index: kernel/generic/trsm_ltcopy_8.c
===================================================================
--- kernel/generic/trsm_ltcopy_8.c      (revision 6263)
+++ kernel/generic/trsm_ltcopy_8.c      (working copy)
@@ -798,7 +798,7 @@
        *(b +  1) = data02;
        *(b +  2) = data03;
        *(b +  3) = data04;
-       *(b +  4) = data05;
+       // *(b +  4) = data05;
       }

       if (ii < jj) {
Index: kernel/generic/zgemm_tcopy_4.c
===================================================================
--- kernel/generic/zgemm_tcopy_4.c      (revision 6263)
+++ kernel/generic/zgemm_tcopy_4.c      (working copy)
@@ -293,8 +293,8 @@

        aoffset1 += 8;
        aoffset2 += 8;
-       aoffset3 += 8;
-       aoffset4 += 8;
+       // aoffset3 += 8;
+       // aoffset4 += 8;

        boffset1 += m * 8;
        i --;
Index: kernel/generic/zneg_tcopy_4.c
===================================================================
--- kernel/generic/zneg_tcopy_4.c       (revision 6263)
+++ kernel/generic/zneg_tcopy_4.c       (working copy)
@@ -293,8 +293,8 @@

        aoffset1 += 8;
        aoffset2 += 8;
-       aoffset3 += 8;
-       aoffset4 += 8;
+       // aoffset3 += 8;
+       // aoffset4 += 8;

        boffset1 += m * 8;
        i --;
Index: .
===================================================================
--- .   (revision 6263)
+++ .   (working copy)

[xianyi]

Oh, I see.

However, when I ran scan-build-3.8 make, I met the following error.

make[1]: /usr/share/clang/scan-ar: Command not found
make[1]: *** [libs] Error 127
make[1]: Leaving directory `/home/traits/project/OpenBLAS_tmp/interface'
make: *** [libs] Error 1
scan-build: 48 bugs found.
scan-build: Run 'scan-view /tmp/scan-build-2016-03-31-034631-41225-1' to examine bug reports.

[brada4]

ln -s `which ar` /usr/share/clang/scan-ar
ln -s `which ranlib` /usr/share/clang/scan-ranlib

You need some browser like w3m or firefox to see the result...

[martin-frbg]

@xianyi is buildbot setup still an open issue ?