add tensor flow package

[davidlange6]

add tensorflow for slc7_amd64, make theano the default backend for Keras. glibc on slc6 is too old to support it there.

[cmsbuild]

A new Pull Request was created by @davidlange6 (David Lange) for branch IB/CMSSW_9_0_X/gcc530.

@cmsbuild, @smuzaffar, @iahmad-khan, @davidlange6 can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

[davidlange6]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17773/console

[cmsbuild]

+1
Tested at: d26f1a1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2824/17773/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2824/17773/summary.html

The workflows 1003.0, 1001.0, 1000.0, 140.53, 136.731, 4.22 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

[davidlange6]

need to confirm slc7 is ok in the IBs...

[mharrend]

Sorry to step into this conversation, but since we are also interested in using tensorflow within our framework I have some questions:

• Are you planning to compile and provide the tensorflow C++ library, too?
  E.g this can be done via
  git clone https://github.com/tensorflow/tensorflow
  cd tensorflow
  ./configure
  bazel build -c opt //tensorflow:libtensorflow_cc.so

• Can this be backported to 8_0_X?

[davidlange6]

Hi @mharrend - I was hoping the C++ API came with the PIP wheel, but it seems it does not (the headers are there..) - I think supporting bazel is beyond what we can do today, but I'll investigate. We can eventually think about how to back port things to analysis releases, though we need to implement some integration testing and get some experience first.

[davidlange6]

hi again

fwiw, I've been slowly getting tensor flow to work - the available wheel requires changes to our python build. Its now available in the 'devel' version of our centos7 builds. At least import tensorflow works now..

[dlange@lxplus031 dlange]$ scram p CMSSW_9_1_DEVEL_X_2017-03-23-1100
WARNING: Release CMSSW_9_1_DEVEL_X_2017-03-23-1100 is not available for architecture slc7_amd64_gcc493.
Developer's area is created for available architecture slc7_amd64_gcc630.
[dlange@lxplus031 dlange]$ cd CMSSW_9_1_DEVEL_X_2017-03-23-1100
[dlange@lxplus031 CMSSW_9_1_DEVEL_X_2017-03-23-1100]$ cmsenv
[dlange@lxplus031 CMSSW_9_1_DEVEL_X_2017-03-23-1100]$ python
Python 2.7.11 (default, Mar 23 2017, 15:35:17)
[GCC 6.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import tensor flow

now onto the C++ interface..

[mharrend]

Dear David, we are working on the C++ API of Tensorflow in parallel at the moment. In https://github.com/mharrend/tensorflow-c--api I have a Jenkinsfile which partly allows to build the API. Hopefully, next week it will work completely. At the moment I am preparing a PR to allow a non-interactive Tensorflow configure step. Best regards Marco Am Freitag, 24. März 2017 10:18:28 CET schrieb David Lange <notifications@github.com>:

…

hi again fwiw, I've been slowly getting tensor flow to work - the available wheel requires changes to our python build. Its now available in the 'devel' version of our centos7 builds. At least import tensorflow works now.. ***@***.*** dlange]$ scram p CMSSW_9_1_DEVEL_X_2017-03-23-1100 WARNING: Release CMSSW_9_1_DEVEL_X_2017-03-23-1100 is not available for architecture slc7_amd64_gcc493. Developer's area is created for available architecture slc7_amd64_gcc630. ***@***.*** dlange]$ cd CMSSW_9_1_DEVEL_X_2017-03-23-1100 ***@***.*** CMSSW_9_1_DEVEL_X_2017-03-23-1100]$ cmsenv ***@***.*** CMSSW_9_1_DEVEL_X_2017-03-23-1100]$ python Python 2.7.11 (default, Mar 23 2017, 15:35:17) [GCC 6.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tensor flow now onto the C++ interface..

-- Marco A. Harrendorf KIT - IEKP

[mharrend]

Dear @davidlange6,

I have now managed to build the Tensorflow C++ API library and also a Tensorflow PIP wheel package for SLC6.
Please checkout my recipe in
https://github.com/mharrend/tensorflow-c--api/blob/master/Jenkinsfile
and let me know if I can help you further, so that we get Tensorflow backported to CMSSW_8_0_X.

If you follow the instructions you can install obtained PIP wheel file via
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py --user
pip install wheel-file.name

Best regards,

Marco

[mharrend]

Dear @davidlange6,

for at least the CMSSW_8_0_26_patch1 version it is now quite easy to use Tensorflow in the following way:

export PYTHONPATH=/afs/cern.ch/work/m/mharrend/public/tensorflow-cmssw8-0-26-patch1/site-packages:$PYTHONPATH

For a check if Tensorflow works:

python -m tensorflow

or better check:

git clone https://gitlab.cern.ch/mrieger/TensorflowTest.git
scram b -j
run_tftest

Hopefully, this is useful.

Best regards,

Marco