Add a SCRAM toolfile for Intel VTune #2681

fwyzard · 2016-11-25T13:52:15Z

No description provided.

cmsbuild · 2016-11-25T13:52:29Z

A new Pull Request was created by @fwyzard (Andrea Bocci) for branch IB/CMSSW_9_0_X/gcc530.

@cmsbuild, @smuzaffar, @iahmad-khan, @davidlange6 can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

fwyzard · 2016-11-25T14:17:46Z

please test

cmsbuild · 2016-11-25T14:18:01Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/cmsdist-test-pr/414/console

fwyzard · 2016-11-25T14:22:24Z

This should be included in CMSSW only on top of cms-sw/cmssw-config#56 .

smuzaffar · 2016-11-25T14:38:32Z

intel-vtune.spec

+  <client>
+    <environment name="INTEL_VTUNE_BASE" default="/afs/cern.ch/sw/IntelSoftware/linux/x86_64/xe2017/vtune_amplifier_xe_%{realversion}"/>
+    <environment name="INCLUDE" default="$INTEL_VTUNE_BASE/include"/>
+    <environment name="LIBDIR" default="$INTEL_VTUNE_BASE/lib64"/>


this directory contains libstdc++. this can cause problems. Looks like vtune executables linked with full path e.g.

libstdc++.so.6 => /afs/cern.ch/sw/IntelSoftware/linux/x86_64/xe2017/vtune_amplifier_xe_2017.0.2.478468/bin64/./../lib64/libstdc++.so.6 (0x00007f4003603000)

So I guess we can drop it and instead add

<flags LDFLAGS="$(INTEL_VTUNE_BASE)/lib64"/>

fwyzard · 2016-11-25T15:03:00Z

@smuzaffar I see your point, however that does not seem to happen: after setting up the intel-vtune tool in CMSSW, I do not see its library path added to LD_LIBRARY_PATH:

$ cmsenv
$ echo $LD_LIBRARY_PATH | tr : \\n
/data/user/fwyzard/IntelWorkshop/CMSSW_9_0_X_2016-11-24-2300/biglib/slc6_amd64_gcc530
/data/user/fwyzard/IntelWorkshop/CMSSW_9_0_X_2016-11-24-2300/lib/slc6_amd64_gcc530
/data/user/fwyzard/IntelWorkshop/CMSSW_9_0_X_2016-11-24-2300/external/slc6_amd64_gcc530/lib
/cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_0_X_2016-11-24-2300/biglib/slc6_amd64_gcc530
/cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_0_X_2016-11-24-2300/lib/slc6_amd64_gcc530
/cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_0_X_2016-11-24-2300/external/slc6_amd64_gcc530/lib
/cvmfs/cms-ib.cern.ch/2016-48/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_X_2016-11-22-2300/biglib/slc6_amd64_gcc530
/cvmfs/cms-ib.cern.ch/2016-48/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_X_2016-11-22-2300/lib/slc6_amd64_gcc530
/cvmfs/cms-ib.cern.ch/2016-48/slc6_amd64_gcc530/external/llvm/4.0.0-oenich/lib64
/cvmfs/cms-ib.cern.ch/2016-48/slc6_amd64_gcc530/external/gcc/5.3.0/lib64
/cvmfs/cms-ib.cern.ch/2016-48/slc6_amd64_gcc530/external/gcc/5.3.0/lib

cmsbuild · 2016-11-25T15:17:43Z

+1
Tested at: 5360e5c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2681/414/summary.html

cmsbuild · 2016-11-25T15:17:45Z

Comparison job queued.

smuzaffar · 2016-11-25T15:27:40Z

@fwyzard , ah right. SCRAM only adds the path in LD_LIBRARY_PATH if tool has explicitly ask it to do so

<runtime name="LD_LIBRARY_PATH" value="$TOOL_BASE/lib64" type="path"/>

so this toolfile should be fine.

smuzaffar · 2016-11-25T15:34:10Z

intel-vtune.spec

+</tool>
+EOF_TOOLFILE
+
+## IMPORT scram-tools-post


I am not sure, if this is going to create the correct init.sh and init.csh files. As you do not have an explicit -toofile, so I would suggest to not import scram-tools-post but just add explicit %post section like https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_9_0_X/gcc530/scram-tools-post.file#L19 (use INTEL_VTUNE instead of %{BaseTool}_TOOLFILE).

You mean like

%post if [ "$CMS_INSTALL_PREFIX" = "" ] ; then CMS_INSTALL_PREFIX=$RPM_INSTALL_PREFIX; export CMS_INSTALL_PREFIX; fi %{relocateConfig}etc/scram.d/*.xml echo "INTEL_VTUNE_ROOT='$CMS_INSTALL_PREFIX/%{pkgrel}'" > $RPM_INSTALL_PREFIX/%{pkgrel}/etc/profile.d/init.sh echo "set INTEL_VTUNE_ROOT='$CMS_INSTALL_PREFIX/%{pkgrel}'" > $RPM_INSTALL_PREFIX/%{pkgrel}/etc/profile.d/init.csh

?

But there is nothing to relocate, and there is nothing that makes use of $INTEL_VTUNE_ROOT .
Maybe I should just have an empty %post section ?

scramv1-tool-conf.file (which collects all toolfiles and bundle them in tool-conf) needs to access the <tool>_ROOT env. So each tool should set it.
If you provide an empty %post section then cmsBuild is anyway going to generate init.sh with default values for ROOT, VERSION, PATH, LD_LIBRARY_PATH etc (just a standard init.sh file). As for toofiles we do not need any of these that is why we re-create init.* with only _ROOT variable in them.

smuzaffar · 2016-11-25T15:34:36Z

intel-vtune.spec

+  </client>
+  <runtime name="PATH" value="$INTEL_VTUNE_BASE/bin64" type="path"/>
+  <runtime name="VTUNE_AMPLIFIER_XE_2017_DIR" value="$INTEL_VTUNE_BASE"/>
+  <runtime name="INTEL_LICENSE_FILE" value="28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch" type="path" handler="warn"/>


drop this one too, I will set it via intel-license toolfile

OK - but there is not such toolfile for the time being. Do you want me to add one ?

I have added a one here
7cfbfa5
it should be available in next IB

cmsbuild · 2016-11-25T16:02:51Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2681/414/summary.html

fwyzard · 2016-11-25T16:57:26Z

Understood, thanks. I'll fix it.

cmsbuild · 2016-11-25T21:57:04Z

Pull request #2681 was updated.

fwyzard · 2016-11-25T21:57:38Z

please test

cmsbuild · 2016-11-25T21:57:53Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/cmsdist-test-pr/415/console

fwyzard · 2016-11-26T07:31:32Z

The error does not seem to come from this PR ?

smuzaffar · 2016-11-26T09:51:10Z

@fwyzard , forgot to mention that you need to add intel-vtune dependency in cmssw-tool-conf.spec file. Without that this tool is not going to be part of cmssw. You should remove INTEL_LICENSE_FILE too which is now available in cmssw 90C IBs via intel-license tool.

fwyzard · 2016-11-28T17:32:17Z

please test

cmsbuild · 2016-11-28T17:32:20Z

The tests are being triggered in jenkins.

cmsbuild · 2016-11-28T17:32:21Z

Pull request #2681 was updated.

fwyzard · 2016-11-28T17:37:44Z

please test

cmsbuild · 2016-11-28T17:37:49Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/cmsdist-test-pr/416/console

cmsbuild · 2016-11-28T17:37:50Z

Pull request #2681 was updated.

fwyzard · 2016-11-28T17:39:23Z

@fwyzard , forgot to mention that you need to add intel-vtune dependency in cmssw-tool-conf.spec file. Without that this tool is not going to be part of cmssw. You should remove INTEL_LICENSE_FILE too which is now available in cmssw 90C IBs via intel-license tool.

Both should be done.

Should there be an explicit dependency on intel-license ? How does one make sure it ends up in the "selected" tools instead of the "available" ones ?

smuzaffar · 2016-11-28T18:21:16Z

All tools build via cmssw-tool-conf are by default selected unless explicitly ignored. intel-license is not ignored , so it should be selected and should already be visible in 90X IBs.

cmsbuild · 2016-11-28T20:48:29Z

+1
Tested at: 5bc1acd
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2681/416/summary.html

cmsbuild · 2016-11-28T20:48:32Z

Comparison job queued.

cmsbuild · 2016-11-28T21:21:08Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2681/416/summary.html

The workflows 1003.0, 1001.0, 1000.0, 140.53, 136.731, 4.22 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

cmsbuild added externals-pending orp-pending pending-signatures tests-pending labels Nov 25, 2016

fwyzard force-pushed the add_intel_vtune_90x_530 branch from a85bdc2 to 5360e5c Compare November 25, 2016 14:16

cmsbuild added tests-started and removed tests-pending labels Nov 25, 2016

fwyzard mentioned this pull request Nov 25, 2016

Instrument cmssw for Intel VTune cms-sw/cmssw#16763

Closed

smuzaffar reviewed Nov 25, 2016

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Nov 25, 2016

smuzaffar reviewed Nov 25, 2016

View reviewed changes

fwyzard force-pushed the add_intel_vtune_90x_530 branch from 5360e5c to f16f42a Compare November 25, 2016 21:56

cmsbuild added tests-pending and removed tests-approved labels Nov 25, 2016

cmsbuild removed the tests-pending label Nov 25, 2016

cmsbuild added the tests-rejected label Nov 25, 2016

fwyzard added 2 commits November 28, 2016 18:31

Add a SCRAM toolfile for Intel VTune

f018ff4

Add Intel VTune to CMSSW

934d0a4

fwyzard force-pushed the add_intel_vtune_90x_530 branch from f16f42a to 934d0a4 Compare November 28, 2016 17:32

cmsbuild added tests-pending and removed tests-rejected labels Nov 28, 2016

cmsbuild added tests-started and removed tests-pending labels Nov 28, 2016

move INTEL_LICENSE_FILE to the intel-license tool

5bc1acd

cmsbuild added tests-pending and removed tests-started labels Nov 28, 2016

cmsbuild added tests-started and removed tests-pending labels Nov 28, 2016

cmsbuild added tests-approved and removed tests-started labels Nov 28, 2016

smuzaffar merged commit 69f22df into cms-sw:IB/CMSSW_9_0_X/gcc530 Nov 28, 2016

fwyzard deleted the add_intel_vtune_90x_530 branch October 4, 2017 15:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a SCRAM toolfile for Intel VTune #2681

Add a SCRAM toolfile for Intel VTune #2681

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016 •

edited

fwyzard commented Nov 25, 2016

smuzaffar Nov 25, 2016 •

edited

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016

cmsbuild commented Nov 25, 2016

smuzaffar commented Nov 25, 2016

smuzaffar Nov 25, 2016

fwyzard Nov 25, 2016

smuzaffar Nov 25, 2016 •

edited

smuzaffar Nov 25, 2016

fwyzard Nov 25, 2016

smuzaffar Nov 25, 2016

cmsbuild commented Nov 25, 2016

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016 •

edited

fwyzard commented Nov 26, 2016 via email

smuzaffar commented Nov 26, 2016 •

edited

fwyzard commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

fwyzard commented Nov 28, 2016

cmsbuild commented Nov 28, 2016 •

edited

cmsbuild commented Nov 28, 2016

fwyzard commented Nov 28, 2016

smuzaffar commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

Add a SCRAM toolfile for Intel VTune #2681

Add a SCRAM toolfile for Intel VTune #2681

Conversation

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016 • edited

fwyzard commented Nov 25, 2016

smuzaffar Nov 25, 2016 • edited

Choose a reason for hiding this comment

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016

cmsbuild commented Nov 25, 2016

smuzaffar commented Nov 25, 2016

smuzaffar Nov 25, 2016

Choose a reason for hiding this comment

fwyzard Nov 25, 2016

Choose a reason for hiding this comment

smuzaffar Nov 25, 2016 • edited

Choose a reason for hiding this comment

smuzaffar Nov 25, 2016

Choose a reason for hiding this comment

fwyzard Nov 25, 2016

Choose a reason for hiding this comment

smuzaffar Nov 25, 2016

Choose a reason for hiding this comment

cmsbuild commented Nov 25, 2016

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016

fwyzard commented Nov 25, 2016

cmsbuild commented Nov 25, 2016 • edited

fwyzard commented Nov 26, 2016 via email

smuzaffar commented Nov 26, 2016 • edited

fwyzard commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

fwyzard commented Nov 28, 2016

cmsbuild commented Nov 28, 2016 • edited

cmsbuild commented Nov 28, 2016

fwyzard commented Nov 28, 2016

smuzaffar commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

cmsbuild commented Nov 28, 2016

cmsbuild commented Nov 25, 2016 •

edited

smuzaffar Nov 25, 2016 •

edited

smuzaffar Nov 25, 2016 •

edited

cmsbuild commented Nov 25, 2016 •

edited

smuzaffar commented Nov 26, 2016 •

edited

cmsbuild commented Nov 28, 2016 •

edited